added ntscu-u support

README.md

@@ -23,6 +23,8 @@ python3 project-solana build ~/Games/ratchet.iso   # once: about 95 MB of downlo
 ./project-solana desktop                     # adds it to the applications menu; from there, Steam > Add a Non-Steam Game
 ```
 
+With both the PAL and the NTSC-U version built, `play` starts the one built or played last; `play --game pal` or `play --game us` picks one (`log` and `report` take `--game` too).
+
 It needs `python3` and the **32-bit** SDL2 and OpenGL libraries. On SteamOS (Steam Deck, Steam Machine) nothing needs installing: the launcher uses the copy of SDL2 that comes with Steam. Elsewhere, `./project-solana status` says what to install for your distribution. Everything is kept in `~/.local/share/project-solana`. On a Steam Deck, run the build once in Desktop Mode (Konsole), then add the game to Steam with `desktop`; it starts full screen there (and on any SteamOS device), with the Deck's controls as the controller.
 
 ### Supported games
@@ -30,7 +32,8 @@ It needs `python3` and the **32-bit** SDL2 and OpenGL libraries. On SteamOS (Ste
 | Game | Region | Disc executable | Status |
 |---|---|---|---|
 | Ratchet & Clank | PAL (v2.00) | `SCES_509.16`, MD5 `d86429e2eca4e1da7d7f1cc656af2574` | Playable start to finish |
-| Ratchet & Clank | NTSC-U, NTSC-J | `SCUS_971.99`, `SCPS_150.37` | Recognised, not supported yet |
+| Ratchet & Clank | NTSC-U | `SCUS_971.99`, MD5 `0c081d4e0353b419d95e5f95eb0ebacb` | New: boots, menus and all 19 levels load; less tested than PAL |
+| Ratchet & Clank | NTSC-J | `SCPS_150.37` | Recognised, not supported yet |
 | Going Commando, Up Your Arsenal, Deadlocked | all | | Recognised, not supported yet |
 
 ### Requirements
@@ -74,7 +77,9 @@ The launcher fetches from the release named after the game (`releases/download/r
 | `toolchain-linux64.zip` | the same for Linux x86-64 (the Linux launcher's) |
 | `rac1-build.zip` | `games/rac1` plus the decomp's source as `decomp/` (no `baserom/`) |
 
-It checks both zips against their SHA-256s, unpacks them next to itself (`toolchain\`, `games\rac1\`), runs `build.py --platform windows --iso <your disc>`, and then deletes the intermediate files (about 1.3 GB) and the bytes it read from the disc, keeping `games\rac1\build\windows` (the game, its saves and logs).
+It checks both zips against their SHA-256s, unpacks them next to itself (`toolchain\`, `games\rac1\`), runs `build.py --platform windows --region pal|us --iso <your disc>`, and then deletes the intermediate files (about 1.3 GB) and the bytes it read from the disc, keeping `games\rac1\build\windows` (PAL) or `games\rac1\build\windows_us` (NTSC-U): the game, its saves and logs. Both versions come from the same kit.
+
+The NTSC-U build is made from the same sources: `tools/region.py` rewrites the decomp and the host engine from PAL addresses into NTSC-U ones with the address map `region/us.json` (functions paired by their code, data by the code that points at it; made by `dev/region_map.py`), and the code itself is translated from the NTSC-U executable and levels. `host/region/us/` holds the little the NTSC-U game needs that PAL's does not.
 
 `solana.ini` `[build]` `release_url=` (a URL or a local folder laid out like the release) and `levels=` override where it fetches from and which levels it builds, for testing.
 

games/rac1/README.md

@@ -1,6 +1,6 @@
-# Ratchet & Clank (PAL v2.00): build kit
+# Ratchet & Clank (PAL v2.00, NTSC-U): build kit
 
-A static recompilation of Ratchet & Clank (`SCES_509.16`) for Windows (and Linux, for development).
+A static recompilation of Ratchet & Clank (PAL `SCES_509.16`, and NTSC-U `SCUS_971.99` with `--region us`) for Windows and Linux.
 
 `build.py` reads the game's executable and level programs from your disc image, generates C from them (the decomp's own C where it has it, everything else translated instruction by instruction by `tools/recomp.py`, the VU microcode by `tools/vu_recomp.py`), and compiles it with the host engine in `host/` into `build/windows/rac1_game.dll` and `rac1_wrapper.exe`.
 
@@ -8,6 +8,8 @@ A static recompilation of Ratchet & Clank (`SCES_509.16`) for Windows (and Linux
 python build.py --platform windows --iso "Ratchet & Clank (Europe).iso" --repo ../rac1-decomp --levels 0,1,...,18
 ```
 
+`--region us` builds the NTSC-U version into `build/<platform>_us` from the same sources: `tools/region.py` first writes the decomp and `host/` in NTSC-U addresses (`build/<platform>_us/region`, with `region_report.txt` listing what it changed and what had no counterpart), using the address map `region/us.json` (made by `dev/region_map.py` from both executables and both sets of levels). Code only the NTSC-U game has goes in `host/region/us/`, already in its addresses.
+
 Needs Python 3, `pip install rabbitizer ziglang`. The Project Solana launcher runs this for players with its own copies of both.
 
 | Path | What |
@@ -16,6 +18,7 @@ Needs Python 3, `pip install rabbitizer ziglang`. The Project Solana launcher ru
 | `tools/recomp.py` | MIPS (R5900) to C, the executable and each level |
 | `tools/vu_recomp.py` | VU0/VU1 microprograms to C |
 | `tools/level_dump.py` | a level's program from the disc image |
+| `tools/region.py`, `region/us.json` | the NTSC-U build: the sources rewritten in its addresses |
 | `host/src/gs/` | the Graphics Synthesizer: software and OpenGL 3.3 (upscaling: `UYA_SCALE`) |
 | `host/src/gfx/` | VIF, GIF and the VU runtime |
 | `host/src/sdk/`, `host/src/rac1/` | Sony library entry points, 989snd and the SPU2, levels, boot |

games/rac1/build.py

@@ -25,13 +25,19 @@ ap.add_argument("-j", type=int, default=os.cpu_count() or 2)
 ap.add_argument("-O", default="2")
 ap.add_argument("--iso", default=None)
 ap.add_argument("--levels", default="00")
+ap.add_argument("--region", default="pal", choices=("pal", "us"),
+                help="pal: SCES-50916 (the decomp's own); us: SCUS-97199, made from the same sources by tools/region.py")
 a = ap.parse_args()
+sys.path.insert(0, os.path.join(HERE, "tools"))
+import region as _region
+RC = _region.consts(a.region)
+EXE_NAME = RC["exe"]
 
 zig = [shutil.which("zig")] if shutil.which("zig") else [sys.executable, "-m", "ziglang"]
 # Linux: 32-bit glibc 2.17 or newer (any distribution since 2013), linked dynamically so it can load
 # the system's SDL2 and OpenGL driver (32-bit: i386 libraries)
 target = {"linux": "x86-linux-gnu.2.17", "windows": "x86-windows-gnu"}[a.platform]
-out = os.path.join(HERE, "build", a.platform)
+out = os.path.join(HERE, "build", a.platform if a.region == "pal" else "%s_%s" % (a.platform, a.region))
 gen = os.path.join(out, "gen")
 stage = os.path.join(out, "gen.new")
 obj = os.path.join(out, "obj")
@@ -90,14 +96,22 @@ def iso_file(iso, name):
     return None
 
 
-env = dict(os.environ, UYA_PLATFORM=a.platform, RAC_PLATFORM=a.platform)
+env = dict(os.environ, UYA_PLATFORM=a.platform, RAC_PLATFORM=a.platform, UYA_REGION=a.region)
+# another region: the decomp and the host sources rewritten in its addresses (tools/region.py)
+HOST = os.path.join(HERE, "host")
+if a.region != "pal":
+    rdir = os.path.join(out, "region")
+    subprocess.run([sys.executable, os.path.join(HERE, "tools", "region.py"), a.region, a.repo, rdir], check=True)
+    a.repo = os.path.join(rdir, "repo")
+    HOST = os.path.join(rdir, "host")
+env["UYA_HOST"] = HOST
 # the static recompilation is built from the game itself: the executable comes
 # from the disc image (the decomp repo only lends its symbol and function lists)
 if find_iso():
-    _exe = iso_file(find_iso(), "SCES_509.16")
+    _exe = iso_file(find_iso(), EXE_NAME)
     if _exe:
         os.makedirs(os.path.join(HERE, "build", "disc"), exist_ok=True)
-        _p = os.path.join(HERE, "build", "disc", "SCES_509.16")
+        _p = os.path.join(HERE, "build", "disc", EXE_NAME)
         if not os.path.exists(_p) or open(_p, "rb").read() != _exe: open(_p, "wb").write(_exe)
         env["UYA_EXE"] = _p
         print("executable read from the disc image:", find_iso())
@@ -108,7 +122,7 @@ def progress(phase, i, n):
     print("[progress] %s %d %d" % (phase, i, n), flush=True)
 progress("prepare", 0, 3)
 subprocess.run([py, os.path.join(HERE, "tools", "gen_host.py"), a.repo, stage], check=True, env=env)
-dumps = os.path.join(HERE, "build", "levels")
+dumps = os.path.join(HERE, "build", "levels" if a.region == "pal" else "levels_" + a.region)
 # the level programs first: the executable's translation looks at the core
 # addresses their code calls (tools/recomp.py)
 for lv in levels:
@@ -146,7 +160,7 @@ with open(os.path.join(stage, "level_table.c"), "w") as f:
 sync(stage, gen)
 
 common = ["-target", target, "-O" + a.O, "-g", "-march=nehalem", "-mfpmath=sse", "-malign-double", "-fno-strict-aliasing", "-fwrapv",
-          "-fno-PIE", "-fno-stack-protector", "-I", os.path.join(gen, "include"), "-I", os.path.join(HERE, "host", "include")]
+          "-fno-PIE", "-fno-stack-protector", "-I", os.path.join(gen, "include"), "-I", os.path.join(HOST, "include")]
 if a.platform == "linux": common += ["-D_FILE_OFFSET_BITS=64"]   # (32-bit glibc: disc images are over 2 GB)
 # frontbin code: keep it permissive, it was written for gcc 2.95
 menu_flags = ["-w", "-Wno-error=incompatible-pointer-types", "-Wno-error=int-conversion",
@@ -168,9 +182,9 @@ jobs = []   # (source, extra flags, object name, forced header)
 for c in sorted(glob.glob(os.path.join(gen, "*.c"))):
     jobs.append((c, menu_flags if os.path.basename(c) not in ("tramp_table.c", "level_table.c") else [], None, None))
 jobs.append((os.path.join(gen, "abs_syms.s"), [], None, None))
-for c in sorted(glob.glob(os.path.join(HERE, "host", "overrides", "*.c"))):
+for c in sorted(glob.glob(os.path.join(HOST, "overrides", "*.c"))):
     jobs.append((c, menu_flags, None, None))
-for c in sorted(glob.glob(os.path.join(HERE, "host", "src", "**", "*.c"), recursive=True)):
+for c in sorted(glob.glob(os.path.join(HOST, "src", "**", "*.c"), recursive=True)):
     jobs.append((c, ["-Wall", "-Wno-unused"], None, None))
 for LV, *_ in level_info:
     lg = os.path.join(gen, "l" + LV)
@@ -181,7 +195,7 @@ for LV, *_ in level_info:
     jobs.append((os.path.join(lg, "abs_syms.s"), [], os.path.join("l" + LV, "abs_syms.s.o"), None))
     # the executable's main-segment overrides that the level has: the level's copies
     for nm in open(os.path.join(lg, "overrides.txt")).read().split():
-        c = os.path.join(HERE, "host", "overrides", nm + ".c")
+        c = os.path.join(HOST, "overrides", nm + ".c")
         jobs.append((c, menu_flags + ["-include", hdr], os.path.join("l" + LV, "override_" + nm + ".c.o"), hdr))
 
 
@@ -194,13 +208,13 @@ if not os.path.exists(_stamp) or open(_stamp).read() != _flags:
 flags_time = os.path.getmtime(_stamp)
 
 # headers every generated file sees: a change rebuilds them all
-headers_time = max([flags_time] + [os.path.getmtime(h) for h in glob.glob(os.path.join(gen, "include", "*.h")) + glob.glob(os.path.join(HERE, "host", "include", "*.h"))])
+headers_time = max([flags_time] + [os.path.getmtime(h) for h in glob.glob(os.path.join(gen, "include", "*.h")) + glob.glob(os.path.join(HOST, "include", "*.h"))])
 
 
 def cc(job):
     src, extra, oname, hdr = job
     o = os.path.join(obj, oname or (os.path.basename(src) + ".o"))
-    if os.path.exists(o) and src.endswith(".c") and os.path.join(HERE, "host") not in src:
+    if os.path.exists(o) and src.endswith(".c") and HOST not in src:
         t = os.path.getmtime(o)
         if t > os.path.getmtime(src) and t > headers_time and (not hdr or t > os.path.getmtime(hdr)):
             return o, None

games/rac1/dev/region_map.py

@@ -0,0 +1,267 @@
+#!/usr/bin/env python3
+"""region_map.py DECOMP_REPO PAL_EXE US_EXE PAL_LEVELS US_LEVELS OUT_JSON
+
+Developer tool (needs both executables): the PAL-to-US address map that lets the
+build make the US version (SCUS-97199) from the same sources as PAL (SCES-50916).
+The output is addresses only (no game bytes) and ships in the build kit as
+region/us.json; tools/region.py reads it.
+
+  functions  the executable's: OpenRAC's config/overlays/us_map.tsv (code alone),
+             PAL name -> US address and size, with how it was paired; and each
+             level's, from the same table and the catalogue's places
+             (config/overlays/functions.tsv), PAL place -> US place
+  data       the executable's data sections: PAL and US words aligned (an address
+             in either one counts as equal: the pointers moved), giving runs of
+             PAL addresses that share one shift. Sections whose size did not
+             change move as one. Checked against the data addresses the paired
+             functions form with lui/addiu (both builds load the same data).
+  levels     each level's data records (lit, bss, data, vtbl, camvtbl, sndvtbl)
+             aligned the same way (PAL_LEVELS and US_LEVELS: level_dump.py output)
+"""
+import json, os, re, struct, sys
+from collections import Counter, defaultdict
+
+REPO, PAL_EXE, US_EXE, PAL_LV, US_LV, OUT = sys.argv[1:7]
+pal, us = open(PAL_EXE, "rb").read(), open(US_EXE, "rb").read()
+
+
+def sections(e):
+    shoff, _, shnum, shstr = struct.unpack_from("<I", e, 32)[0], *struct.unpack_from("<HHH", e, 46)
+    S = [struct.unpack_from("<10I", e, shoff + i * 40) for i in range(shnum)]
+    names = e[S[shstr][4]:S[shstr][4] + S[shstr][5]]
+    out = {}
+    for s in S:
+        n = names[s[0]:names.index(b"\0", s[0])].decode()
+        if s[3]: out[n] = {"addr": s[3], "off": s[4], "size": s[5], "nobits": s[1] == 8}
+    return out
+
+
+SP, SU = sections(pal), sections(us)
+
+
+def seg_bytes(e, a, n):
+    phoff, phnum = struct.unpack_from("<I", e, 28)[0], struct.unpack_from("<H", e, 44)[0]
+    for i in range(phnum):
+        t, off, va, pa, fs = struct.unpack_from("<5I", e, phoff + 32 * i)
+        if va <= a and a + n <= va + fs: return e[off + a - va: off + a - va + n]
+
+
+# ---- functions -----------------------------------------------------------------------------------
+fsize = {}
+for u in json.load(open(os.path.join(REPO, "progress", "report.json")))["units"]:
+    for f in u.get("functions", []):
+        md = f.get("metadata") or {}
+        if "virtual_address" in md: fsize[f["name"]] = (int(md["virtual_address"]), int(f["size"]))
+places = {}   # catalogue name -> {level: [PAL addresses]}
+for l in open(os.path.join(REPO, "config", "overlays", "functions.tsv")):
+    if l.startswith("#") or not l.strip(): continue
+    f = l.rstrip("\n").split("\t")
+    d = defaultdict(list)
+    for x in f[5].split(","):
+        if ":" in x: d[x.split(":")[0]].append(int(x.split(":")[1], 16))
+    places[f[0]] = d
+
+exe_funcs, exe_us_only = {}, []
+lv_rows = defaultdict(list)
+for l in open(os.path.join(REPO, "config", "overlays", "us_map.tsv")):
+    if l.startswith("#") or not l.strip(): continue
+    prog, ua, sz, name, method, sim = l.rstrip("\n").split("\t")[:6]
+    ua, sz = int(ua, 16), int(sz)
+    if prog == "boot":
+        q = re.fullmatch(r"func_([0-9A-F]{8})", name)
+        if q: exe_funcs["%08X" % int(q.group(1), 16)] = [ua, sz, method]
+        else: exe_us_only.append([ua, sz])
+    else:
+        lv_rows[prog[6:]].append((ua, sz, name, method))
+levels = {}
+for lv, rows in sorted(lv_rows.items()):
+    by_name = defaultdict(list)
+    only = []
+    for ua, sz, name, method in sorted(rows):
+        if name in places and places[name].get(lv): by_name[name].append((ua, sz, method))
+        else: only.append([ua, sz])
+    m = {}
+    for name, us_pl in by_name.items():
+        for pa, (ua, sz, method) in zip(sorted(places[name][lv]), us_pl):   # several copies: paired in address order
+            m["%08X" % pa] = [ua, sz, method]
+    levels[lv] = {"funcs": m, "us_only": only}
+
+# ---- data ----------------------------------------------------------------------------------------
+def words(b): return list(struct.unpack("<%dI" % (len(b) // 4), b[:len(b) // 4 * 4]))
+def tok(w): return -1 if 0x00100000 <= w < 0x02000000 else w
+
+
+def align(a_words, b_words, a_base, b_base, W=8):
+    """runs [pal_lo, pal_hi, shift] (bytes) of a PAL range onto a US range"""
+    a, b = [tok(x) for x in a_words], [tok(x) for x in b_words]
+    d, runs, i = 0, [], 0
+    while i < len(a):
+        if 0 <= i + d < len(b) and a[i] == b[i + d]:
+            if runs and runs[-1][2] == d and runs[-1][1] == i: runs[-1][1] = i + 1
+            else: runs.append([i, i + 1, d])
+            i += 1; continue
+        best = None
+        for k in range(0, 1024):
+            for dd in (d - k, d + k):
+                if 0 <= i + dd and i + dd + W <= len(b) and a[i:i + W] == b[i + dd:i + dd + W] and \
+                        sum(1 for x in a[i:i + W] if x not in (0, -1)) >= 3:
+                    best = dd; break
+            if best is not None: break
+        if best is None: i += 1; continue
+        d = best
+    # runs of a few words that only match zeros or pointers are noise: drop the short ones
+    # between two runs of one shift
+    out = []
+    for s, e, dd in runs:
+        sh = (b_base + 4 * (s + dd)) - (a_base + 4 * s)
+        if out and out[-1][2] == sh: out[-1][1] = a_base + 4 * e
+        else: out.append([a_base + 4 * s, a_base + 4 * e, sh])
+    clean = []
+    for k, r in enumerate(out):
+        if 0 < k < len(out) - 1 and r[1] - r[0] <= 16 and out[k - 1][2] == out[k + 1][2] != r[2]:
+            continue
+        if clean and clean[-1][2] == r[2]: clean[-1][1] = r[1]
+        else: clean.append(r)
+    # close the gaps: a gap takes the shift of the run before it
+    for k in range(len(clean) - 1): clean[k][1] = clean[k + 1][0]
+    if clean: clean[0][0], clean[-1][1] = a_base, a_base + 4 * len(a_words)
+    return clean
+
+
+data = []
+for name in ("core.data", "core.rdata", "core.bss", "core.lit", ".lit", ".bss", ".data", "lvl.vtbl", "lvl.camvtbl", "lvl.sndvtbl"):
+    p, u = SP[name], SU[name]
+    if p["nobits"]:   # (no bytes to compare: it moves as one; checked against the code below)
+        data.append([p["addr"], p["addr"] + p["size"], u["addr"] - p["addr"], name])
+        continue
+    for lo, hi, sh in align(words(pal[p["off"]:p["off"] + p["size"]]), words(us[u["off"]:u["off"] + u["size"]]), p["addr"], u["addr"]):
+        data.append([lo, hi, sh, name])
+
+
+def data_shift(a):
+    for lo, hi, sh, _ in data:
+        if lo <= a < hi: return sh
+
+
+# the data addresses paired functions build (lui + addiu/ori/load/store): both builds load the
+# same data, so each is a PAL -> US pair. The executable's functions and every level's (level code
+# uses the executable's data a lot).
+def refs(b):
+    w = struct.unpack("<%dI" % (len(b) // 4), b); hi = {}; out = {}
+    for i, x in enumerate(w):
+        op, rs, rt, imm = x >> 26, (x >> 21) & 31, (x >> 16) & 31, x & 0xFFFF
+        if op == 0x0F: hi[rt] = imm << 16; continue
+        if rs in hi and (op == 0x09 or 0x20 <= op <= 0x3F or op in (0x1A, 0x1B, 0x1E, 0x1F)):
+            out[i] = (hi[rs] + (imm - 0x10000 if imm & 0x8000 else imm)) & 0xFFFFFFFF
+        if op in (0x09, 0x0D, 0x0F) or 0x20 <= op <= 0x27: hi.pop(rt, None)
+    return out
+
+
+anchor_votes = defaultdict(Counter)
+RESIDENT = 0x0015F000   # PAL: from here up, each level program has its own lit/bss/data/tables
+def collect(b1, b2, top):
+    if not b1 or not b2: return
+    r1, r2 = refs(b1), refs(b2)
+    for i, x in r1.items():
+        if i in r2 and SP["core.data"]["addr"] <= x < top:
+            anchor_votes[x][r2[i] - x] += 1
+
+
+for pa_s, (ua, sz, method) in exe_funcs.items():
+    if method == "fingerprint":
+        pa = int(pa_s, 16); collect(seg_bytes(pal, pa, sz), seg_bytes(us, ua, sz), SP[".text"]["addr"])
+lv_text = {}
+for lv in levels:
+    try:
+        tp = [r for r in json.load(open(os.path.join(PAL_LV, "level_" + lv, "manifest.json")))["records"] if r["name"] == "text"][0]
+        tu = [r for r in json.load(open(os.path.join(US_LV, "level_" + lv, "manifest.json")))["records"] if r["name"] == "text"][0]
+    except (OSError, IndexError):
+        continue
+    bp = open(os.path.join(PAL_LV, "level_" + lv, "text.bin"), "rb").read(); bu = open(os.path.join(US_LV, "level_" + lv, "text.bin"), "rb").read()
+    lv_text[lv] = (tp["address"], bp, tu["address"], bu)
+    for pa_s, (ua, sz, method) in levels[lv]["funcs"].items():
+        if method != "fingerprint": continue
+        pa = int(pa_s, 16)
+        collect(bp[pa - tp["address"]:pa - tp["address"] + sz], bu[ua - tu["address"]:ua - tu["address"] + sz], RESIDENT)
+
+# data sections move down (by 0x100 to 0x400): a "shift" outside that is a constant, not an address
+def align_shift(a):
+    for lo, hi, sh, _ in data:
+        if lo <= a < hi: return sh
+
+
+anchors = {}
+for x, c in anchor_votes.items():
+    sh, n = c.most_common(1)[0]
+    if not (-0x600 <= sh <= -0x40 and sh % 4 == 0): continue
+    # trusted when several references agree and none disagrees, or when the bytes agree
+    # (a lone or contested pair is often base + a constant the compiler folded differently)
+    if (n >= 2 and len(c) == 1) or sh == align_shift(x): anchors[x] = sh
+
+
+def build(sec_runs):
+    """the final runs for one section: anchors where the code has them; between two anchors of
+    one shift, that shift; between anchors of different shifts, the alignment's boundary if it
+    has one there, else halfway"""
+    lo, hi = sec_runs[0][0], sec_runs[-1][1]
+    pts = sorted((a, s) for a, s in anchors.items() if lo <= a < hi)
+    if not pts: return [r[:3] for r in sec_runs]
+    def align_at(a):
+        for r in sec_runs:
+            if r[0] <= a < r[1]: return r[2]
+    out = []
+    def put(a0, a1, sh):
+        if a0 >= a1: return
+        if out and out[-1][2] == sh and out[-1][1] == a0: out[-1][1] = a1
+        else: out.append([a0, a1, sh])
+    # before the first anchor: the alignment, as long as it agrees with that anchor, else the anchor's
+    put(lo, pts[0][0], pts[0][1])
+    for (a, s), (b, t) in zip(pts, pts[1:]):
+        if s == t: put(a, b, s); continue
+        # the boundary: the first address from a where the alignment says t
+        cut = None
+        for r in sec_runs:
+            if r[2] == t and a < r[0] <= b: cut = r[0]; break
+        if cut is None: cut = (a + (b - a) // 2) & ~3
+        put(a, cut, s); put(cut, b, t)
+    put(pts[-1][0], hi, pts[-1][1])
+    return out
+
+
+final = []
+for name in ("core.data", "core.rdata", "core.bss", "core.lit", ".lit", ".bss", ".data", "lvl.vtbl", "lvl.camvtbl", "lvl.sndvtbl"):
+    rs = [r for r in data if r[3] == name]
+    for r in build(rs): final.append(r + [name])
+data = final
+
+
+def data_shift(a):
+    for lo, hi, sh, _ in data:
+        if lo <= a < hi: return sh
+
+
+agree = sum(1 for x, s in anchors.items() if data_shift(x) == s)
+print("data map: %d runs; %d code anchors (%d agree with the map)" % (len(data), len(anchors), agree))
+for lo, hi, sh, n in data[:80]: print("   %-11s %08X-%08X %+X" % (n, lo, hi, sh))
+
+# ---- level data ----------------------------------------------------------------------------------
+level_data = {}
+for lv in sorted(levels):
+    pm = os.path.join(PAL_LV, "level_" + lv, "manifest.json"); um = os.path.join(US_LV, "level_" + lv, "manifest.json")
+    if not (os.path.exists(pm) and os.path.exists(um)): continue
+    P = {r["name"]: r for r in json.load(open(pm))["records"]}; U = {r["name"]: r for r in json.load(open(um))["records"]}
+    runs = []
+    for rec in ("lit", "bss", "data", "vtbl", "camvtbl", "sndvtbl"):
+        p, u = P[rec], U[rec]
+        if p["bytes"] == u["bytes"] or rec == "bss":
+            runs.append([p["address"], p["address"] + p["bytes"], u["address"] - p["address"], rec]); continue
+        pb = open(os.path.join(PAL_LV, "level_" + lv, rec + ".bin"), "rb").read()
+        ub = open(os.path.join(US_LV, "level_" + lv, rec + ".bin"), "rb").read()
+        for lo, hi, sh in align(words(pb), words(ub), p["address"], u["address"]): runs.append([lo, hi, sh, rec])
+    level_data[lv] = runs
+
+json.dump({"region": "us", "exe": "SCUS_971.99", "pal_exe": "SCES_509.16",
+           "sections": {n: [SP[n]["addr"], SP[n]["size"], SU[n]["addr"], SU[n]["size"]] for n in SP if n in SU},
+           "exe_funcs": exe_funcs, "exe_us_only": exe_us_only, "data": data,
+           "levels": levels, "level_data": level_data}, open(OUT, "w"), separators=(",", ":"))
+print("wrote %s: %d executable functions (%d US-only), %d levels" % (OUT, len(exe_funcs), len(exe_us_only), len(levels)))

games/rac1/host/region/us/mc_us.c

@@ -0,0 +1,11 @@
+/* NTSC-U (SCUS_971.99) only: library functions the PAL game does not have.
+ * Written in US addresses; tools/region.py copies this folder into the US
+ * host tree as it is (no rewriting) and the PAL build never sees it. */
+#include "uya_host.h"
+#include "uya_runtime.h"
+
+s32 rac1_mc_chdir(s32 port, s32 slot, const char *dir, char *prev);
+
+/* 0x001240C0: sceMcChdir(port, slot, newDir, curDir), RPC 0x0C (its sync
+ * callback 0x00124038 copies the previous directory into curDir) */
+s32 func_001240C0(s32 port, s32 slot, char *dir, char *cur) { return rac1_mc_chdir(port, slot, dir, cur); }

games/rac1/host/src/rac1/core_map.c

@@ -107,7 +107,7 @@ s32 func_00118E70(s32 reg) { (void)reg; return 0x00070000; }         /* sceSifGe
  *    game sends the data with sceSifSetDma (above). Without it every stash
  *    slot was refused (size 0) and the pause menu unpacked the weapon data
  *    from a scrubbed buffer (a crash reading 0xDEE6BEEF). ---------- */
-#define STASH_BASE 0x00160000u
+#define STASH_BASE 0x00160000u                       /* an IOP address (region:keep) */
 #define STASH_SIZE 0x00094000u
 void func_0011AE20(s32 mode) { (void)mode; }                         /* sceSifInitRpc */
 s32 func_0011B2F8(void *cd, u32 server, s32 mode)                    /* sceSifBindRpc */

games/rac1/host/src/rac1/rc_runtime.c

@@ -144,7 +144,7 @@ const char *rc_stack_scan(void)
     t[0] = 0;
     for (int i = 0; sp && i < 1024 && k < 12; i += 2) {
         u32 v = sp[i];
-        if (v >= 0x00112380u && v < 0x00300000u && !(v & 3) && (v & 0xFFFF) && v != 0x00160000u && v != 0x00170000u) {
+        if (v >= 0x00112380u && v < 0x00300000u && !(v & 3) && (v & 0xFFFF) && v != 0x00160000u && v != 0x00170000u) { /* region:keep */
             n += snprintf(t + n, sizeof t - n, " %08X@%d", v, i * 4); k++;
         }
     }

games/rac1/host/src/rac1/snd989.c

@@ -410,7 +410,7 @@ static u32 bank_add(const u8 *src, u32 size)
         b->nsounds = (s16)rd16(b->sblk + 0x16);
         b->first_sound = rd32(b->sblk + 0x1C);
         b->first_grain = rd32(b->sblk + 0x20);
-        b->handle = 0x00120000u + (u32)i * 0x1000u + ((++bank_serial) & 0xFFF);
+        b->handle = 0x00120000u + (u32)i * 0x1000u + ((++bank_serial) & 0xFFF);   /* region:keep */
         if (rd32(b->sblk + 4) != 1) uya_log("989snd: SBlk version %u (only 1 is expected)", rd32(b->sblk + 4));
         uya_log("989snd: bank %08X loaded: %d sounds, %u KB of samples", b->handle, b->nsounds, b->smp_size / 1024);
         if (snd_log > 0 && getenv("UYA_DUMP")) {               /* for checking: the bank as loaded */

games/rac1/host/src/sdk/memcard.c

@@ -96,11 +96,15 @@ static int glob_match(const char *p, const char *s)
 /* Card path ("/DIR/file" or "DIR/file") to a host path. Wildcards in the
  * directory part pick the first directory that matches. Returns 0 if a
  * wildcard directory had no match. */
+static char mc_cwd[2][512] = {"/", "/"};   /* sceMcChdir's current directory (the NTSC-U game sets it) */
 static int host_path(int port, const char *name, char *out, size_t n, char *last, size_t nlast)
 {
     char tmp[512], cur[1024];
     snprintf(cur, sizeof cur, "%s", slot_dir(port));
-    snprintf(tmp, sizeof tmp, "%s", name ? name : "");
+    if (name && name[0] != '/' && strcmp(mc_cwd[port & 1], "/"))
+        snprintf(tmp, sizeof tmp, "%s/%s", mc_cwd[port & 1], name);   /* relative to the current directory */
+    else
+        snprintf(tmp, sizeof tmp, "%s", name ? name : "");
     char *parts[16]; int np = 0;
     for (char *t = strtok(tmp, "/"); t && np < 16; t = strtok(0, "/")) parts[np++] = t;
     for (int i = 0; i < np - 1; i++) {
@@ -177,6 +181,32 @@ s32 uyafn_12E400(s32 port, s32 slot, s32 *type, s32 *free_kb, s32 *format)
     return 0;
 }
 
+/* sceMcChdir(port, slot, new, cur*) (RPC 0x0C): the card's current directory
+ * for relative names; cur, if given, gets the previous one. PAL's game never
+ * calls it; NTSC-U's sets "/" before listing (host/region/us maps it). */
+s32 rac1_mc_chdir(s32 port, s32 slot, const char *dir, char *prev)
+{
+    (void)slot;
+    if (mclog()) uya_log("memory card: chdir %s", dir ? dir : "(null)");
+    if (!card_present(port)) { finish(0xC, -10); return 0; }
+    if (prev) snprintf(prev, 1024, "%s", mc_cwd[port & 1]);
+    char want[512];
+    if (!dir || !*dir) { finish(0xC, 0); return 0; }
+    if (dir[0] == '/') snprintf(want, sizeof want, "%s", dir);
+    else if (!strcmp(dir, "..")) {
+        snprintf(want, sizeof want, "%s", mc_cwd[port & 1]);
+        char *sl = strrchr(want, '/'); if (sl && sl != want) *sl = 0; else snprintf(want, sizeof want, "/");
+    } else snprintf(want, sizeof want, "%s%s%s", mc_cwd[port & 1], strcmp(mc_cwd[port & 1], "/") ? "/" : "", dir);
+    size_t l = strlen(want);
+    while (l > 1 && want[l - 1] == '/') want[--l] = 0;
+    char hp[1024];
+    snprintf(hp, sizeof hp, "%s%s", slot_dir(port), want);
+    if (!is_dir(hp)) { finish(0xC, -4); return 0; }   /* no such directory */
+    snprintf(mc_cwd[port & 1], sizeof mc_cwd[0], "%s", want);
+    finish(0xC, 0);
+    return 0;
+}
+
 /* sceMcGetDir table entry (sceMcTblGetDir), 0x40 bytes */
 typedef struct {
     u8 create[8], modify[8];

games/rac1/tools/gen_host.py

@@ -48,9 +48,15 @@ while _a:
 LV = "%02d" % LEVEL if LEVEL is not None else None
 SFX = "_l%s" % LV if LEVEL is not None else ""
 HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+# the region (build.py --region): its host sources (tools/region.py writes them) and constants
+REGION = os.environ.get("UYA_REGION", "pal")
+HOSTDIR = os.environ.get("UYA_HOST") or os.path.join(HERE, "host")
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import region as _region
+RC = _region.consts(REGION)
 PLAT = os.environ.get("RAC_PLATFORM", os.environ.get("UYA_PLATFORM", "linux"))  # linux | windows (COFF i386 adds a leading _)
 os.makedirs(OUT, exist_ok=True)
-overrides = {os.path.basename(p)[:-2] for p in glob.glob(os.path.join(HERE, "host", "overrides", "func_*.c"))}
+overrides = {os.path.basename(p)[:-2] for p in glob.glob(os.path.join(HOSTDIR, "overrides", "func_*.c"))}
 ASM_MACRO = re.compile(r"^\s*(INCLUDE_ASM|ASM_FUNC|LINKER_REMNANT|INCLUDE_RODATA|TEXT_PADDING)\s*\(([^()]*)\)\s*;?\s*(?:/\*.*?\*/|//.*)?\s*$", re.M)
 
 
@@ -82,7 +88,7 @@ def strip_comments_keep_len(s):
 
 host_defined = set()
 STATIC = os.environ.get("UYA_STATIC", "1") != "0"   # RAC1_STATICRECOMP: translate the whole game from its bytes
-for p in glob.glob(os.path.join(HERE, "host", "src", "**", "*.c"), recursive=True):
+for p in glob.glob(os.path.join(HOSTDIR, "src", "**", "*.c"), recursive=True):
     host_defined |= set(re.findall(r"^[\w \t\*]*\b(func_(?:L\d\d_)?[0-9A-Fa-f]{6,8})\s*\([^;{]*\)\s*\{", strip_comments_keep_len(open(p).read()), re.M))
 
 
@@ -171,8 +177,8 @@ def rewrite_types(s):
 DECL = re.compile(r"^[ \t]*(?:extern[ \t]+)?(?P<t>[A-Za-z_][\w \t\*]*?)[ \t\*]*\b(?P<n>(?:func|D)_(?:L\d\d_)?[0-9A-Fa-f]{6,8}\w*)[ \t]*(?P<s>\((?:[^;{}]*)\)|\[[^;{}]*\])?[ \t]*;", re.M)
 
 
-MAIN_TEXT = (0x001E9080, 0x0023E740)   # the executable's game program (.text of the main segment)
+MAIN_TEXT = RC["main_text"]   # the executable's game program (.text of the main segment); PAL 0x001E9080-0x0023E740
-RESIDENT = 0x0015F000                   # below: core segment and core_lit, the same in every program
+RESIDENT = RC["resident"]     # below: core segment and core_lit, the same in every program; PAL 0x0015F000
 PLACES = {}                             # level mode: catalogue name -> its addresses in this level
 CAT_KIND = {}
 CAT_SIZE = {}
@@ -333,7 +339,7 @@ if LEVEL is None: os.makedirs(INC, exist_ok=True)
 for h in (glob.glob(os.path.join(REPO, "include", "*.h")) if LEVEL is None else []):
     bn = os.path.basename(h)
     if bn == "common.h":
-        t = open(os.path.join(HERE, "host", "include", "rac1_common.h")).read()
+        t = open(os.path.join(HOSTDIR, "include", "rac1_common.h")).read()
     elif bn == "include_asm.h":
         t = "/* host: assembly inclusion has no meaning on the PC */\n"
     else:
@@ -344,7 +350,7 @@ for h in (glob.glob(os.path.join(REPO, "include", "*.h")) if LEVEL is None else
 
 stubbed, defined, asm_entries, report = [], set(), [], {}
 gen_texts = {}
-drafts = {os.path.basename(p)[:-2]: p for p in glob.glob(os.path.join(HERE, "host", "drafts", "func_*.c"))}
+drafts = {os.path.basename(p)[:-2]: p for p in glob.glob(os.path.join(HOSTDIR, "drafts", "func_*.c"))}
 used_drafts = set()
 files = sorted(os.path.relpath(p, REPO) for p in glob.glob(os.path.join(REPO, "src", "game", "*.c")))
 files += sorted(os.path.relpath(p, REPO) for p in glob.glob(os.path.join(REPO, "src", "game", "movie", "*.c")))
@@ -359,6 +365,9 @@ dropped = []      # level mode: functions of other levels' programs
 # the EE ABI but not for x86 (each entry: file, old text, new text)
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 from source_patches import PATCHES
+if REGION != "pal":   # (written against the PAL sources: in the region's names, as tools/region.py rewrote them)
+    _rw = _region.Rewriter(_region.RegionMap(_region.REGIONS[REGION]["map"]))
+    PATCHES = [(pf, _rw.text(old, "patch:" + pf), _rw.text(new, "patch:" + pf)) for pf, old, new in PATCHES]
 patches_applied = 0
 # names.h: readable names are macros for the address names; expand them in
 # the C (outside string literals) so every later pass sees func_/D_ names.
@@ -453,7 +462,8 @@ for rel in files:
     # UYA_FORCE_ASM="" compiles all the C; or a list: file substrings, re:regex,
     # !file (keep that file's C), e.g. "help_0020CDF0,help_00232560" (the old default).
     _fa = os.environ.get("UYA_FORCE_ASM", "re:.")
-    if LEVEL is not None and _fa:
+    # (another region: the executable's C too, so all of it comes from the region's own bytes)
+    if (LEVEL is not None or REGION != "pal") and _fa:
         _ex = [t[1:] for t in _fa.split(",") if t.startswith("!")]
         if any(x and x in rel for x in _ex): _fa = ""            # "!file-substring": that file keeps its C
         for tok in _fa.split(","):
@@ -483,7 +493,9 @@ for rel in files:
             seg = "/* host: top-level asm removed */"
         else:
             name, bpos = extra
-            if LEVEL is not None and base_func(name) == name and (name.startswith("func_L") or is_main_func(name)) and name not in PLACES:
+            # (another region: func_palonly_X, a PAL function with no counterpart found in this region,
+            # is not in any of its programs either; its C would be defined once per level)
+            if LEVEL is not None and (name.startswith("func_palonly_") or base_func(name) == name and (name.startswith("func_L") or is_main_func(name)) and name not in PLACES):
                 dropped.append(name)
                 out.append("/* host: %s is not in level %s's program */" % (name, LV)); pos = e
                 continue
@@ -602,7 +614,7 @@ for fn, t in gen_texts.items():
         rets[q.group(1)] = ret_type(t[q.start():q.start(1) + len(q.group(1))], q.group(1))
 # host functions (host/src, host/overrides) too: a decomp call that leaves out
 # arguments the EE would still pass in registers gets them filled in
-for hp in glob.glob(os.path.join(HERE, "host", "src", "**", "*.c"), recursive=True) + glob.glob(os.path.join(HERE, "host", "overrides", "*.c")):
+for hp in glob.glob(os.path.join(HOSTDIR, "src", "**", "*.c"), recursive=True) + glob.glob(os.path.join(HOSTDIR, "overrides", "*.c")):
     ht = open(hp, errors="replace").read()
     hm = strip_comments_keep_len(ht)
     for q in HDR.finditer(hm):
@@ -808,7 +820,7 @@ for n, p in defs.items():
     a = ps2_addr(n)
     if a is not None and re.fullmatch(r"func_(?:L\d\d_)?[0-9A-Fa-f]{8}", n) and not any(x[0] == "..." for x in p):
         fdefs[a] = p
-for op_ in glob.glob(os.path.join(HERE, "host", "overrides", "func_*.c")):
+for op_ in glob.glob(os.path.join(HOSTDIR, "overrides", "func_*.c")):
     ot = open(op_, errors="replace").read()
     for q in HDR.finditer(strip_comments_keep_len(ot)):
         n = q.group(1); a = ps2_addr(n)
@@ -1224,7 +1236,7 @@ for fn_, t_ in gen_texts.items():
 # value in an int (`int mode = GetEffectTex(1);` then passed on as the texture),
 # gets 64 bits: on the EE the register keeps all of them whatever the C says.
 _r64 = sorted(n for n, s in sigs.items() if s.get("src") == "decl" and s["ret"] == "l" and re.fullmatch(r"func_(?:L\d\d_)?[0-9A-Fa-f]{8}", n)
-              and (s.get("addr") or 0) >= 0x0015F000)     # game code (the core's library calls are the host's)
+              and (s.get("addr") or 0) >= RESIDENT)     # game code (the core's library calls are the host's)
 if _r64:
     _names = "|".join(re.escape(n) for n in _r64)
     _INT = r"(?:int|s32|u32|unsigned int|unsigned|signed int|signed)"

games/rac1/tools/recomp.py

@@ -43,6 +43,11 @@ rabbitizer.config.pseudos_enablePseudos = False
 
 GEN, REPO = sys.argv[1], sys.argv[2]
 HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+REGION = os.environ.get("UYA_REGION", "pal")
+HOSTDIR = os.environ.get("UYA_HOST") or os.path.join(HERE, "host")
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import region as _region
+RC = _region.consts(REGION)
 LEVEL = DUMP = EXEGEN = None
 _a = sys.argv[3:]
 while _a:
@@ -53,14 +58,15 @@ while _a:
     else: sys.exit("recomp.py: unknown option " + _o)
 LV = "%02d" % LEVEL if LEVEL is not None else None
 P = "rc_" if LEVEL is None else "rc%s_" % LV        # prefix of the generated symbols
-RESIDENT = 0x0015F000
+RESIDENT = RC["resident"]   # PAL 0x0015F000
 
 # ---- inputs -----------------------------------------------------------------
 # testing: UYA_RC_TRACE="func_A,func_B" (build time) logs every call of those translated functions
 TRACE_FUNCS = set(x for x in os.environ.get("UYA_RC_TRACE", "").split(",") if x)
-EXE = os.environ.get("UYA_EXE") or os.path.join(REPO, "baserom", "SCES_509.16")   # build.py: the copy read from the disc image
+EXE = os.environ.get("UYA_EXE") or os.path.join(REPO, "baserom", RC["exe"])   # build.py: the copy read from the disc image
 img = open(EXE, "rb").read()
-TEXT_LO, TEXT_HI = 0x00100080, 0x00100080 + 0x13E6B0
+TEXT_LO = 0x00100080   # the executable's one loadable segment (PAL size 0x13E6B0)
+TEXT_HI = TEXT_LO + struct.unpack_from("<I", img, struct.unpack_from("<I", img, 28)[0] + 16)[0]
 LTEXT_LO = LTEXT_HI = 0
 if LEVEL is not None:
     _ld = os.path.join(DUMP, "level_%s" % LV)
@@ -86,16 +92,15 @@ for u in report["units"]:
             fsize[f["name"]] = (int(md["virtual_address"]), int(f["size"]))
 sigs = json.load(open(os.path.join(GEN, "sigs.json")))
 entries = json.load(open(os.path.join(GEN, "asm_entries.json")))
-overrides = {os.path.basename(p)[:-2] for p in os.listdir(os.path.join(HERE, "host", "overrides")) if p.endswith(".c")} if os.path.isdir(os.path.join(HERE, "host", "overrides")) else set()
+overrides = {os.path.basename(p)[:-2] for p in os.listdir(os.path.join(HOSTDIR, "overrides")) if p.endswith(".c")} if os.path.isdir(os.path.join(HOSTDIR, "overrides")) else set()
 
 todo = [n for n in entries if n in fsize and n not in overrides and re.fullmatch(r"func_[0-9A-F]{8}", n)
         and not (n in sigs and sigs[n].get("src") == "def")
         and TEXT_LO <= fsize[n][0] < TEXT_HI]
 # Sony libraries that are pure computation are translated too (libvu0: the
 # sceVu0 matrix and vector routines, core_text 0x1252A0-0x125630)
-EXTRA_RANGES = [(0x001252A0, 0x00125630),
+EXTRA_RANGES = RC["extra_ranges"]   # PAL 0x001252A0-0x00125630, and libgcc 0x0011DF10-0x001207B8
-                # libgcc (64-bit integer and soft-float helpers; level code calls some the executable's C does not)
+                # (libgcc: 64-bit integer and soft-float helpers; level code calls some the executable's C does not)
-                (0x0011DF10, 0x001207B8)]
 todo += [n for n, (a, sz) in fsize.items() if any(lo <= a < hi for lo, hi in EXTRA_RANGES)
          and re.fullmatch(r"func_[0-9A-F]{8}", n) and not (n in sigs and sigs[n].get("src") == "def") and n not in overrides]
 base_of = {}
@@ -259,7 +264,7 @@ else:
     _is_start_x = {a for a, _ in _starts_x}
     _abs = os.path.join(GEN, "abs_syms.s")
     _cand = {int(x, 16) for x in re.findall(r"\.set\s+_?D_([0-9A-F]{8})\w*,", open(_abs).read())} if os.path.exists(_abs) else set()
-    _cand = {v for v in _cand if 0x001E9080 <= v < 0x0023E740}
+    _cand = {v for v in _cand if RC["main_text"][0] <= v < RC["main_text"][1]}
     # and functions the C (the executable's and the levels') calls by an address
     # inside an assembly-only function: func_001252C0 is 0x20 into libvu0's
     # func_001252A0 (the levels' hero code calls it)
@@ -277,7 +282,7 @@ else:
             _w = struct.unpack_from("<I", _d, _i)[0]
             if _w >> 26 == 3:
                 _t = (_w & 0x3FFFFFF) << 2
-                if TEXT_LO <= _t < 0x0015F000: _cand.add(_t)
+                if TEXT_LO <= _t < RESIDENT: _cand.add(_t)
     for _v in sorted(_cand):
         if not (TEXT_LO <= _v < TEXT_HI) or _v % 4 or _v in _is_start_x: continue
         _own = [(a, e) for a, e in _starts_x if a < _v < e]

games/rac1/tools/region.py

@@ -0,0 +1,298 @@
+#!/usr/bin/env python3
+"""region.py: build another region's version of the game from the PAL sources.
+
+  python tools/region.py us DECOMP_REPO OUT_DIR
+
+The decomp, the host sources and every tool here work in the PAL executable's
+addresses (SCES-50916), and they name functions and data by address
+(func_0012F4A8, D_0013A548). region/us.json (made by tools/region_map.py from both
+executables; addresses only) maps every PAL address to the US one (SCUS-97199).
+This writes, under OUT_DIR:
+
+  repo/   the decomp as the tools read it, in US addresses: its catalogues
+          (progress/report.json, config/*, config/overlays/functions.tsv), headers and
+          sources with every func_/D_ name and address literal rewritten
+  host/   the host sources (host/src, host/include, host/overrides) the same way
+  region_report.txt   every literal rewritten, and what could not be mapped
+
+A PAL function the US build does not have is renamed func_palonly_X, so nothing is
+ever placed at its PAL address; a US function the PAL build does not have is added to
+the catalogue (and translated from the bytes like the rest). The build then runs the
+usual tools on these trees (build.py --region us)."""
+import bisect, json, os, re, shutil, sys
+
+HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+REGIONS = {"us": {"map": os.path.join(HERE, "region", "us.json"), "exe": "SCUS_971.99", "pal_exe": "SCES_509.16",
+                  "video": "ntsc"}}
+RESIDENT_PAL = 0x0015F000
+CORE_TEXT = (0x00112380, 0x0012F580)   # PAL core.text: the Sony libraries
+
+
+class RegionMap:
+    def __init__(self, path):
+        m = json.load(open(path))
+        self.m = m
+        self.exe = {int(k, 16): v for k, v in m["exe_funcs"].items()}
+        self.exe_starts = sorted(self.exe)
+        self.data = sorted(m["data"])
+        self.lv = {lv: {int(k, 16): v for k, v in d["funcs"].items()} for lv, d in m["levels"].items()}
+        self.lv_starts = {lv: sorted(d) for lv, d in self.lv.items()}
+        self.lv_data = {lv: sorted(r) for lv, r in m["level_data"].items()}
+        self.sections = m["sections"]
+        self.us_only = m["exe_us_only"]
+        self.lv_us_only = {lv: d["us_only"] for lv, d in m["levels"].items()}
+
+    @staticmethod
+    def _func(table, starts, a, interior=True):
+        if a in table: return table[a][0]
+        if not interior: return None
+        k = bisect.bisect_right(starts, a) - 1
+        if k >= 0:
+            s = starts[k]; ua, sz, method = table[s]
+            if s < a < s + sz and method == "fingerprint": return ua + (a - s)   # same instructions: same offsets
+        return None
+
+    @staticmethod
+    def _runs(runs, a):
+        for r in runs:
+            if r[0] <= a < r[1]: return a + r[2]
+        return None
+
+    def section_edge(self, a):
+        """a PAL section's first address or its end: the US section's"""
+        if a == 0x0023E740:                      # the end of PAL's game program as the tools give it (consts)
+            t = self.sections[".text"]; return (t[2] + t[3] + 15) & ~15
+        for n, (pa, ps, ua, us_) in self.sections.items():
+            if not pa: continue
+            if a == pa: return ua
+            if a == pa + ps: return ua + us_
+            if a == (pa + ps + 15) & ~15: return (ua + us_ + 15) & ~15     # (an end rounded up to 16)
+        return None
+
+    def exe_addr(self, a, interior=True):
+        """the US address of a PAL executable address, or None"""
+        if 0x00100000 <= a < 0x00112380: return a                # .reginfo, .vutext: the same in both
+        f = self._func(self.exe, self.exe_starts, a, interior)
+        if f is not None: return f
+        return self._runs(self.data, a)
+
+    def level_addr(self, lv, a, interior=True):
+        if a < RESIDENT_PAL: return self.exe_addr(a, interior)  # the core: shared by every program
+        f = self._func(self.lv.get(lv, {}), self.lv_starts.get(lv, []), a, interior)
+        if f is not None: return f
+        return self._runs(self.lv_data.get(lv, []), a)
+
+    def exe_func(self, a): return self.exe[a][0] if a in self.exe else None
+    def level_func(self, lv, a): return self.lv[lv][a][0] if lv in self.lv and a in self.lv[lv] else None
+
+
+def consts(region):
+    """what the tools need to know about the executable, for this region"""
+    pal = {"exe": "SCES_509.16", "main_text": (0x001E9080, 0x0023E740), "resident": 0x0015F000,
+           "core_text": CORE_TEXT, "extra_ranges": [(0x001252A0, 0x00125630), (0x0011DF10, 0x001207B8)]}
+    if region in (None, "", "pal"): return pal
+    rm = RegionMap(REGIONS[region]["map"])
+    t = rm.sections[".text"]; lit = rm.sections[".lit"]; ct = rm.sections["core.text"]
+    return {"exe": REGIONS[region]["exe"],
+            "main_text": (t[2], (t[2] + t[3] + 15) & ~15),
+            "resident": lit[2],
+            "core_text": (ct[2], ct[2] + ct[3]),
+            "extra_ranges": [(rm.exe_addr(lo), rm.exe_addr(hi) if rm.exe_addr(hi) else rm.exe_addr(hi - 4) + 4) for lo, hi in pal["extra_ranges"]]}
+
+
+# ---- rewriting ---------------------------------------------------------------------------------------
+NAME = re.compile(r"\b(func|D)_(?:L(\d\d)_)?([0-9A-Fa-f]{6,8})(?![0-9A-Fa-f])")
+LIT = re.compile(r"\b0x(00[1-9][0-9A-Fa-f]{5}|0[1-9][0-9A-Fa-f]{5}|[1-9][0-9A-Fa-f]{5})(?=[uUlL]*\b)")
+# a line marked region:keep holds values that only look like addresses (IOP addresses, handles, lui values)
+
+
+class Rewriter:
+    def __init__(self, rm):
+        self.rm = rm
+        self.log, self.unmapped = [], []
+        self.taken = {}
+
+    def name(self, kind, lv_name, hexpart, lv_ctx, where):
+        a = int(hexpart, 16)
+        up = hexpart.isupper() or not any(c.isalpha() for c in hexpart)
+        fmt = "%08X" if len(hexpart) == 8 else ("%06X" if up else "%06x")
+        if kind == "func":
+            if lv_name is not None:   # a level catalogue name: its address in that level
+                ua = self.rm.level_func(lv_name, a)
+                if ua is None: self.unmapped.append((where, "func_L%s_%s" % (lv_name, hexpart))); return "func_palonly_L%s_%s" % (lv_name, hexpart)
+                return "func_L%s_%s" % (lv_name, fmt % ua)
+            ua = self.rm.exe_func(a)
+            if ua is None and CORE_TEXT[0] <= a < CORE_TEXT[1]: ua = self.rm.exe_addr(a)   # a library entry inside a split unit
+            if ua is None:
+                # the executable's main-segment functions are also catalogue names of level copies
+                self.unmapped.append((where, "func_" + hexpart)); return "func_palonly_" + hexpart
+            return "func_" + (fmt % ua)
+        ctx = lv_name or lv_ctx                      # D_LNN_X: level NN's own data
+        pre = "D_L%s_" % lv_name if lv_name else "D_"
+        ua = self.rm.level_addr(ctx, a) if ctx else self.rm.exe_addr(a)
+        if ua is None: self.unmapped.append((where, pre + hexpart)); return pre.replace("D_", "D_palonly_") + hexpart
+        # one to one: a second PAL address landing on a US address already taken (two neighbours either
+        # side of a shift boundary) is not trusted; its uses then fail to build instead of reading other data
+        key = (ctx, bool(lv_name), ua)
+        if self.taken.setdefault(key, a) != a:
+            self.unmapped.append((where, "%s%s (clashes with PAL %08X)" % (pre, hexpart, self.taken[key]))); return pre.replace("D_", "D_palonly_") + hexpart
+        return pre + (fmt % ua)
+
+    def text(self, s, where, lv_ctx=None, literals=True):
+        def nm(q):
+            r = self.name(q.group(1), q.group(2), q.group(3), lv_ctx, where)
+            return r
+        s = NAME.sub(nm, s)
+        if literals:
+            def lit(q):
+                h = q.group(1); a = int(h, 16)
+                if not (0x00100000 <= a < 0x02000000): return q.group(0)
+                # only what is surely an address: a section's edge, a function's first instruction, data
+                # (a constant inside a function's range, such as a size of 0x200000, stays as it is)
+                ua = self.rm.section_edge(a)
+                if ua is None:
+                    core = CORE_TEXT[0] <= a < CORE_TEXT[1]   # the Sony libraries: entries inside split units, same code in both
+                    ua = self.rm.level_addr(lv_ctx, a, interior=core) if lv_ctx else self.rm.exe_addr(a, interior=core)
+                if ua is None: self.unmapped.append((where, "0x" + h)); return q.group(0)
+                if ua != a: self.log.append((where, "0x%s -> 0x%08X" % (h, ua)))
+                return "0x%0*X" % (len(h), ua)
+            s = "".join(l if "region:keep" in l else LIT.sub(lit, l) for l in s.splitlines(True))
+        return s
+
+
+def level_of(rel):
+    q = re.search(r"overlays[/\\]l(\d\d)_", rel)
+    return q.group(1) if q else None
+
+
+def write_repo(rm, rw, repo, out):
+    """the decomp, as the tools read it, in US addresses"""
+    for sub in ("include", "src", "nonmatching", "tools"):
+        top = os.path.join(repo, sub)
+        for root, _, files in os.walk(top):
+            for f in files:
+                p = os.path.join(root, f); rel = os.path.relpath(p, repo); d = os.path.join(out, rel)
+                os.makedirs(os.path.dirname(d), exist_ok=True)
+                if sub == "tools" or not f.endswith((".c", ".h", ".inc", ".s")):
+                    shutil.copyfile(p, d); continue
+                d_name = os.path.join(os.path.dirname(d), rw.text(f, rel, literals=False))
+                open(d_name, "w").write(rw.text(open(p, errors="replace").read(), rel, level_of(rel)))
+    os.makedirs(os.path.join(out, "config", "overlays"), exist_ok=True)
+    os.makedirs(os.path.join(out, "progress"), exist_ok=True)
+    # symbols
+    for fn in ("config/symbol_addrs.txt", "config/port_symbols.txt"):
+        p = os.path.join(repo, fn)
+        if not os.path.exists(p): continue
+        lines = []
+        for l in open(p, errors="replace"):
+            q = re.match(r"^(\s*)(\w+)(\s*=\s*)0x([0-9A-Fa-f]+)(.*)$", l.rstrip("\n"))
+            if q:
+                a = int(q.group(4), 16); ua = rm.exe_addr(a)
+                if ua is None:
+                    rw.unmapped.append((fn, q.group(2))); continue
+                l = "%s%s%s0x%08X%s" % (q.group(1), rw.text(q.group(2), fn, literals=False), q.group(3), ua, q.group(5))
+            else:
+                l = rw.text(l.rstrip("\n"), fn)
+            lines.append(l)
+        open(os.path.join(out, fn), "w").write("\n".join(lines) + "\n")
+    for fn in ("config/names.tsv", "config/symbol_names.txt", "config/file_cflags.txt", "config/linker_remnants.txt", "config/handwritten_asm.txt"):
+        p = os.path.join(repo, fn)
+        if os.path.exists(p): open(os.path.join(out, fn), "w").write(rw.text(open(p, errors="replace").read(), fn, literals=False))
+    # the executable's functions (progress/report.json): US addresses and sizes
+    rep = json.load(open(os.path.join(repo, "progress", "report.json")))
+    seen = set()
+    for u in rep.get("units", []):
+        fl = []
+        for f in u.get("functions", []):
+            md = f.get("metadata") or {}
+            if "virtual_address" not in md: fl.append(f); continue
+            a = int(md["virtual_address"])
+            if a in rm.exe:
+                ua, sz, _ = rm.exe[a]
+                f = dict(f); f["name"] = rw.text(f["name"], "report.json", literals=False); f["size"] = str(sz)
+                md = dict(md); md["virtual_address"] = str(ua); f["metadata"] = md
+                fl.append(f); seen.add(ua)
+            # (PAL-only: dropped)
+        u["functions"] = fl
+    extra = [{"name": "func_%08X" % ua, "size": str(sz), "metadata": {"virtual_address": str(ua)}}
+             for ua, sz in rm.us_only if ua not in seen and sz >= 8]
+    rep.setdefault("units", []).append({"name": "us_only", "functions": extra})
+    json.dump(rep, open(os.path.join(out, "progress", "report.json"), "w"))
+    # the US-only functions: assembly entries in a source file of their own
+    with open(os.path.join(out, "src", "game", "us_only.c"), "w") as f:
+        f.write('/* region.py: functions the US executable has and PAL does not (translated from the bytes) */\n#include "common.h"\n\n')
+        for e in extra: f.write('INCLUDE_ASM("asm/us_only", %s);\n' % e["name"])
+    # the level catalogue: each function's places in every level, in US addresses
+    rows = []
+    for l in open(os.path.join(repo, "config", "overlays", "functions.tsv")):
+        if l.startswith("#") or not l.strip(): rows.append(l.rstrip("\n")); continue
+        f = l.rstrip("\n").split("\t")
+        pl, size = [], None
+        for x in f[5].split(","):
+            if ":" not in x: continue
+            lv, a = x.split(":"); a = int(a, 16)
+            ua = rm.level_func(lv, a)
+            if ua is None: continue
+            pl.append("%s:%08X" % (lv, ua))
+            if size is None: size = rm.lv[lv][a][1]
+        if not pl: continue
+        name = rw.text(f[0], "functions.tsv", literals=False)
+        if name.startswith(("func_palonly", "D_palonly")):   # (named after a level copy the US build lacks: name it after one it has)
+            lv0, a0 = pl[0].split(":"); name = "func_L%s_%s" % (lv0, a0)
+        f[0], f[2], f[4], f[5] = name, str(size or f[2]), str(len({p.split(':')[0] for p in pl})), ",".join(pl)
+        rows.append("\t".join(f))
+    open(os.path.join(out, "config", "overlays", "functions.tsv"), "w").write("\n".join(rows) + "\n")
+    for fn in ("config/overlays/names.tsv",):
+        p = os.path.join(repo, fn)
+        if os.path.exists(p): open(os.path.join(out, fn), "w").write(rw.text(open(p, errors="replace").read(), fn, literals=False))
+    os.makedirs(os.path.join(out, "baserom"), exist_ok=True)
+
+
+# host sources that use the game's addresses (the rest: the platform, the GS, VU and movie code)
+HOST_GAME = ("src/rac1/", "src/sdk/", "overrides/", "src/main.c", "src/runtime.c", "include/")
+
+
+def write_host(rm, rw, src_host, out):
+    for root, dirs, files in os.walk(src_host):
+        dirs[:] = [d for d in dirs if d not in ("uya_only", "drafts")]
+        for f in files:
+            p = os.path.join(root, f); rel = os.path.relpath(p, src_host).replace(os.sep, "/")
+            if rel.startswith("region/"):
+                # host/region/<region>/: code only that region has, already in its addresses
+                if rel.startswith("region/%s/" % REGION):
+                    d = os.path.join(out, "src", "region", rel[len("region/%s/" % REGION):])
+                    os.makedirs(os.path.dirname(d), exist_ok=True); shutil.copyfile(p, d)
+                continue
+            d = os.path.join(out, os.path.dirname(rel), rw.text(f, "host/" + rel, literals=False))
+            os.makedirs(os.path.dirname(d), exist_ok=True)
+            if not f.endswith((".c", ".h")): shutil.copyfile(p, d); continue
+            s = open(p, errors="replace").read()
+            s = s.replace("SCES_509.16", REGIONS[REGION]["exe"])   # the executable's file name on the disc
+            open(d, "w").write(rw.text(s, "host/" + rel, literals=rel.startswith(HOST_GAME)) if rel.startswith(HOST_GAME) else s)
+
+
+REGION = None
+
+
+def main():
+    global REGION
+    region, repo, out = sys.argv[1], sys.argv[2], sys.argv[3]
+    REGION = region
+    cfg = REGIONS[region]
+    rm = RegionMap(cfg["map"])
+    rw = Rewriter(rm)
+    shutil.rmtree(os.path.join(out, "repo"), ignore_errors=True); shutil.rmtree(os.path.join(out, "host"), ignore_errors=True)
+    write_repo(rm, rw, repo, os.path.join(out, "repo"))
+    write_host(rm, rw, os.path.join(HERE, "host"), os.path.join(out, "host"))
+    with open(os.path.join(out, "region_report.txt"), "w") as f:
+        f.write("region %s: %d literals rewritten, %d names and literals not mapped\n\n== not mapped\n" % (region, len(rw.log), len(rw.unmapped)))
+        for w, x in sorted(set(rw.unmapped)): f.write("%s\t%s\n" % (w, x))
+        f.write("\n== literals rewritten\n")
+        for w, x in rw.log: f.write("%s\t%s\n" % (w, x))
+    hosts = [(w, x) for w, x in set(rw.unmapped) if w.startswith("host/")]
+    print("region %s: %d literals rewritten; not mapped: %d (host sources: %d) -> %s" % (
+        region, len(rw.log), len(set(rw.unmapped)), len(hosts), os.path.join(out, "region_report.txt")))
+
+
+if __name__ == "__main__":
+    main()

launcher/linux/project-solana

@@ -19,14 +19,16 @@ import argparse, configparser, hashlib, json, os, shutil, struct, subprocess, sy
 
 RELEASE_URL = "https://github.com/OpenRAC/project-solana/releases/download/"
 GAMES = {   # serial (BOOT2 in SYSTEM.CNF): what this version is
-    "SCES_509.16": {"title": "Ratchet & Clank (PAL)", "kit": "rac1", "md5": "d86429e2eca4e1da7d7f1cc656af2574", "exe": "rac1_wrapper"},
+    "SCES_509.16": {"title": "Ratchet & Clank (PAL)", "id": "rac1", "region": "pal", "kit": "rac1", "md5": "d86429e2eca4e1da7d7f1cc656af2574", "exe": "rac1_wrapper"},
+    "SCUS_971.99": {"title": "Ratchet & Clank (NTSC-U)", "id": "rac1us", "region": "us", "kit": "rac1", "md5": "0c081d4e0353b419d95e5f95eb0ebacb", "exe": "rac1_wrapper"},
 }
-OTHER = {"SCUS_971.99": "Ratchet & Clank (NTSC-U)", "SCPS_150.37": "Ratchet & Clank (NTSC-J)",
+BY_ID = {g["id"]: g for g in GAMES.values()}
+OTHER = {"SCPS_150.37": "Ratchet & Clank (NTSC-J)",
          "SCUS_972.68": "Ratchet & Clank: Going Commando (NTSC-U)", "SCES_516.07": "Ratchet & Clank 2 (PAL)",
          "SCUS_973.53": "Ratchet & Clank: Up Your Arsenal (NTSC-U)", "SCES_524.56": "Ratchet & Clank 3 (PAL)",
          "SCUS_974.65": "Ratchet: Deadlocked (NTSC-U)", "SCES_532.85": "Ratchet: Gladiator (PAL)"}
 
-VERSION = "1.5"
+VERSION = "1.6"
 HOME = os.path.join(os.environ.get("XDG_DATA_HOME") or os.path.expanduser("~/.local/share"), "project-solana")
 INI = os.path.join(HOME, "solana.ini")
 LAUNCHER_LOG = os.path.join(HOME, "launcher_log.txt")
@@ -65,6 +67,20 @@ def die(msg):
     sys.exit("project-solana: " + msg)
 
 
+def out_dir(g):
+    """the game's build output (and its saves): build/linux, build/linux_us"""
+    return os.path.join(HOME, "games", g["kit"], "build", "linux" if g["region"] == "pal" else "linux_" + g["region"])
+
+
+def chosen(c, want=None):
+    """the game to play: --game, else the one built last (else PAL)"""
+    if want:
+        g = BY_ID.get(want) or BY_ID.get("rac1" + want) or (BY_ID["rac1"] if want == "pal" else None)
+        if not g: die("no game %s: pal or us" % want)
+        return g
+    return BY_ID.get(c.get("play", "game", fallback="rac1"), BY_ID["rac1"])
+
+
 def settings():
     c = configparser.ConfigParser()
     c.read(INI)
@@ -343,7 +359,8 @@ def build(args):
         if os.path.isdir(gdir):   # everything but the build output (and its saves); a new kit version builds from scratch
             for e in os.listdir(gdir):
                 if e != "build": rmtree(os.path.join(gdir, e))
-            rmtree(os.path.join(gdir, "build", "linux", "gen")); rmtree(os.path.join(gdir, "build", "linux", "obj"))
+            for b in ("linux", "linux_us"):
+                rmtree(os.path.join(gdir, "build", b, "gen")); rmtree(os.path.join(gdir, "build", b, "obj"))
         unzip(zp, gdir, "build kit"); os.remove(zp)
         open(kv, "w").write(man["version"])   # (unpacked: an interrupted build resumes with what it compiled)
 
@@ -356,7 +373,7 @@ def build(args):
     env = dict(os.environ, PATH=os.path.join(tc_dir, "zig") + os.pathsep + os.environ.get("PATH", ""), PYTHONUTF8="1",
                ZIG_GLOBAL_CACHE_DIR=os.path.join(gdir, "zig-cache"), ZIG_LOCAL_CACHE_DIR=os.path.join(gdir, "zig-cache"))
     py = os.path.join(tc_dir, "python", "bin", "python3")
-    cmd = [py, "-u", "build.py", "--platform", "linux", "--levels", levels, "--iso", iso, "--repo", "decomp", "-j", str(jobs)]
+    cmd = [py, "-u", "build.py", "--platform", "linux", "--region", g["region"], "--levels", levels, "--iso", iso, "--repo", "decomp", "-j", str(jobs)]
     log = open(os.path.join(gdir, "build_log.txt"), "w")
     raise_open_files()
     p = subprocess.Popen(cmd, cwd=gdir, env=env, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, bufsize=1)
@@ -365,22 +382,25 @@ def build(args):
         if not progress_line(line) and ("error" in line.lower() or "failed" in line.lower()): print("\n  " + line.rstrip())
     p.wait(); log.close(); print()
     if p.returncode: die("the build stopped (see %s)" % os.path.join(gdir, "build_log.txt"))
-    for j in ("build/linux/gen", "build/linux/gen.new", "build/linux/obj", "build/levels", "build/disc", "zig-cache"): rmtree(os.path.join(gdir, j))
+    bd = os.path.relpath(out_dir(g), gdir)
-    c.set("disc", kit, iso); c.set("build", "built_" + kit, man["version"]); save(c)
+    for j in (bd + "/gen", bd + "/gen.new", bd + "/obj", bd + "/region", "build/levels", "build/levels_us", "build/disc", "zig-cache"): rmtree(os.path.join(gdir, j))
-    say("Built. Ready to play: project-solana play")
+    c.set("disc", g["id"], iso); c.set("build", "built_" + g["id"], man["version"]); c.set("play", "game", g["id"]); save(c)
+    other = [x for x in BY_ID.values() if x is not g and c.get("build", "built_" + x["id"], fallback="")]
+    say("Built. Ready to play: project-solana play" + ("  (the other version: project-solana play --game %s)" % other[0]["region"] if other else ""))
 
 
 # ---- play ----
 def play(args):
     c = settings()
-    kit = "rac1"
+    g = chosen(c, args.game)
-    iso = c.get("disc", kit, fallback="")
+    iso = c.get("disc", g["id"], fallback="")
-    exe = os.path.join(HOME, "games", kit, "build", "linux", GAMES["SCES_509.16"]["exe"])
+    exe = os.path.join(out_dir(g), g["exe"])
     if not os.path.exists(exe) or not iso: die("not built yet: run  project-solana build /path/to/your/disc.iso")
     if not os.path.exists(iso): die("the disc image has moved (%s): build again with its new path" % iso)
     pc = c["play"]
     for k, v in (("resolution", args.resolution), ("fullscreen", args.fullscreen), ("vsync", args.vsync)):
         if v is not None: pc[k] = str(v)
+    pc["game"] = g["id"]
     save(c)
     env = dict(os.environ)
     deck = os.environ.get("SteamDeck") == "1" or is_steamos()   # (Steam Deck, Steam Machine: full screen by default)
@@ -399,7 +419,7 @@ def play(args):
     gdir = os.path.dirname(exe)
     out = os.path.join(gdir, "game_output.txt")   # what SDL, the graphics driver and the game print (kept: one run back)
     if os.path.exists(out): os.replace(out, out + ".prev")
-    log("play: %s (UYA_SCALE=%s, fullscreen %s, vsync %s)" % (iso, env.get("UYA_SCALE", "1"), full, pc.get("vsync", "0")))
+    log("play: %s, %s (UYA_SCALE=%s, fullscreen %s, vsync %s)" % (g["title"], iso, env.get("UYA_SCALE", "1"), full, pc.get("vsync", "0")))
     with open(out, "w") as o:
         try:
             p = subprocess.Popen([exe, "-", iso], cwd=gdir, env=env, stdout=o, stderr=subprocess.STDOUT)
@@ -418,9 +438,9 @@ def status(args):
     say("Data folder: %s" % HOME)
     tv = os.path.join(HOME, "toolchain", "version.txt")
     say("Build tools: %s" % (open(tv).read().strip() if os.path.exists(tv) else "not downloaded"))
-    for kit in ("rac1",):
+    for g in BY_ID.values():
-        b = c.get("build", "built_" + kit, fallback="")
+        b = c.get("build", "built_" + g["id"], fallback="")
-        say("%s: %s, disc %s" % (kit, ("built from kit " + b) if b else "not built", c.get("disc", kit, fallback="(none)")))
+        say("%s: %s, disc %s" % (g["title"], ("built from kit " + b) if b else "not built", c.get("disc", g["id"], fallback="(none)")))
     say("32-bit SDL2 and OpenGL: %s" % ("found" if check_libs(quiet=True) else "missing"))
     if not check_libs(quiet=True): check_libs()
 
@@ -429,16 +449,20 @@ def desktop(args):
     me = os.path.abspath(sys.argv[0])
     d = os.path.join(os.environ.get("XDG_DATA_HOME") or os.path.expanduser("~/.local/share"), "applications")
     os.makedirs(d, exist_ok=True)
-    p = os.path.join(d, "project-solana-rac1.desktop")
+    c = settings()
-    with open(p, "w") as f:
+    built = [g for g in BY_ID.values() if c.get("build", "built_" + g["id"], fallback="")] or [BY_ID["rac1"]]
-        f.write("[Desktop Entry]\nType=Application\nName=Ratchet & Clank (Project Solana)\nComment=Built from your own disc\n"
+    for g in built:
-                "Exec=python3 \"%s\" play\nTerminal=false\nCategories=Game;\n" % me)   # (python3: works without the executable bit)
+        name = "Ratchet & Clank (Project Solana)" if len(built) == 1 else "%s (Project Solana)" % g["title"]
-    say("Added %s" % p)
+        p = os.path.join(d, "project-solana-%s.desktop" % g["id"])
-    say("Steam: Games > Add a Non-Steam Game to My Library, and pick \"Ratchet & Clank (Project Solana)\".")
+        with open(p, "w") as f:
+            f.write("[Desktop Entry]\nType=Application\nName=%s\nComment=Built from your own disc\n"
+                    "Exec=python3 \"%s\" play --game %s\nTerminal=false\nCategories=Game;\n" % (name, me, g["region"]))   # (python3: works without the executable bit)
+        say("Added %s" % p)
+        say("Steam: Games > Add a Non-Steam Game to My Library, and pick \"%s\"." % name)
 
 
 def show_log(args):
-    p = os.path.join(HOME, "games", "rac1", "build", "linux", "rac1_log.txt")
+    p = os.path.join(out_dir(chosen(settings(), args.game)), "rac1_log.txt")
     if not os.path.exists(p): die("no log yet: it is written when the game runs")
     sys.stdout.write(open(p, errors="replace").read())
 
@@ -454,8 +478,9 @@ def tail(path, n):
 def report(args):
     """one text file with what a bug report needs; nothing from the game, and no file of the user's"""
     c = settings()
-    gdir = os.path.join(HOME, "games", "rac1")
+    g = chosen(c, args.game)
-    bdir = os.path.join(gdir, "build", "linux")
+    gdir = os.path.join(HOME, "games", g["kit"])
+    bdir = out_dir(g)
     out = []
     w = out.append
     w("Project Solana bug report (Linux launcher %s), %s\n" % (VERSION, time.strftime("%Y-%m-%d %H:%M:%S %z")))
@@ -491,8 +516,9 @@ def report(args):
     w("build tools: %s\n" % (open(tv).read().strip() if os.path.exists(tv) else "not downloaded"))
     kv = os.path.join(gdir, "kit_version.txt")
     w("build kit: %s\n" % (open(kv).read().strip() if os.path.exists(kv) else "none"))
+    w("game: %s\n" % g["title"])
     w("game built: %s\n" % ("yes" if os.path.exists(os.path.join(bdir, "rac1_wrapper")) else "no"))
-    iso = c.get("disc", "rac1", fallback="")
+    iso = c.get("disc", g["id"], fallback="")
     w("disc image: %s\n" % ((("found, %.2f GB" % (os.path.getsize(iso) / 1e9)) if os.path.exists(iso) else "MISSING (moved?)") if iso else "none"))
     w("settings:\n")
     for s in c.sections():
@@ -520,6 +546,7 @@ def main():
     b.add_argument("--clean", action="store_true", help="fetch the build kit again and build from scratch")
     b.add_argument("--release-url", default="", help=argparse.SUPPRESS)
     p = sp.add_parser("play", help="play (options are remembered)")
+    p.add_argument("--game", default=None, help="pal or us, when both are built (default: the last one built or played)")
     g = p.add_mutually_exclusive_group()
     g.add_argument("--upscaled", dest="resolution", action="store_const", const="upscaled", help="render at about your screen's resolution (default)")
     g.add_argument("--native", dest="resolution", action="store_const", const="native", help="the PS2's own resolution")
@@ -530,8 +557,8 @@ def main():
     v.add_argument("--vsync", dest="vsync", action="store_const", const=1)
     v.add_argument("--no-vsync", dest="vsync", action="store_const", const=0)
     sp.add_parser("status", help="what is installed"); sp.add_parser("desktop", help="add it to the applications menu")
-    sp.add_parser("log", help="print the last game log")
+    for n, h in (("log", "print the last game log"), ("report", "write one file with everything a bug report needs")):
-    sp.add_parser("report", help="write one file with everything a bug report needs")
+        sp.add_parser(n, help=h).add_argument("--game", default=None, help="pal or us")
     a = ap.parse_args()
     if sys.platform != "linux": die("this is the Linux launcher: on Windows, use Project Solana.exe")
     log("project-solana %s: %s" % (VERSION, " ".join(sys.argv[1:])))

launcher/solana.c

@@ -36,6 +36,7 @@ typedef struct {
     const wchar_t *exe;          /* its wrapper in that folder (0: not supported yet) */
     const wchar_t *kit;          /* its GitHub release tag and folder under games\ (the Build button) */
     const wchar_t *const *levels;
+    const wchar_t *rgn;          /* build.py --region (0: pal); its output is build\windows_<rgn> */
 } game_t;
 
 static const wchar_t *const rac1_levels[] = {
@@ -45,7 +46,7 @@ static const wchar_t *const rac1_levels[] = {
 
 static const game_t games[] = {
     { "SCES_509.16", L"Ratchet & Clank", L"PAL", "d86429e2eca4e1da7d7f1cc656af2574", L"games\\rac1\\build\\windows", L"rac1_wrapper.exe", L"rac1", rac1_levels },
-    { "SCUS_971.99", L"Ratchet & Clank", L"NTSC-U", 0, 0, 0, 0, 0 },
+    { "SCUS_971.99", L"Ratchet & Clank", L"NTSC-U", "0c081d4e0353b419d95e5f95eb0ebacb", L"games\\rac1\\build\\windows_us", L"rac1_wrapper.exe", L"rac1", rac1_levels, L"us" },
     { "SCPS_150.37", L"Ratchet & Clank", L"NTSC-J", 0, 0, 0, 0, 0 },
     { "SCES_516.07", L"Ratchet & Clank 2", L"PAL", 0, 0, 0, 0, 0 },
     { "SCUS_972.68", L"Ratchet & Clank: Going Commando", L"NTSC-U", 0, 0, 0, 0, 0 },

launcher/solana.ini

@@ -0,0 +1,14 @@
+[settings]
+upscaled=1
+vsync=0
+no_gs_thread=0
+no_gl_thread=0
+fps_log=0
+capture_seconds=3
+start_level=-1
+minimize=1
+[library]
+count=2
+disc0=C:\Users\Nathan\Desktop\folder folder\Roms\ps2\Ratchet & Clank (Europe) (En,Fr,De,Es,It) (v2.00).iso
+disc1=C:\Users\Nathan\Desktop\folder folder\Roms\ps2\Ratchet & Clank.iso
+selected=1

launcher/solana_build.h

@@ -363,7 +363,7 @@ static void build_line(const char *l, double f0, double f1)
     else if (!strcmp(ph, "compile")) bld_status(f, L"%ls (%d of %d files)", what, a, b);
     else bld_status(f, L"%ls", what);
 }
-static int run_build(const wchar_t *gdir, const wchar_t *iso, const wchar_t *levels, double f0, double f1)
+static int run_build(const wchar_t *gdir, const wchar_t *iso, const wchar_t *levels, const wchar_t *rgn, double f0, double f1)
 {
     wchar_t py[MAX_PATH], zig[MAX_PATH], tc[MAX_PATH], cmd[4096], logp[MAX_PATH];
     path_join(tc, base_dir, L"toolchain");
@@ -375,7 +375,7 @@ static int run_build(const wchar_t *gdir, const wchar_t *iso, const wchar_t *lev
     int j = (int)si.dwNumberOfProcessors, jm = (int)(ms.ullTotalPhys / (1536ull << 20));
     if (j > jm) j = jm;
     if (j < 1) j = 1;
-    swprintf(cmd, 4096, L"\"%ls\" -u build.py --platform windows --levels %ls --iso \"%ls\" --repo decomp -j %d", py, levels, iso, j);
+    swprintf(cmd, 4096, L"\"%ls\" -u build.py --platform windows --region %ls --levels %ls --iso \"%ls\" --repo decomp -j %d", py, rgn, levels, iso, j);
     static wchar_t path[8192]; static int have_path;   /* (the PATH this program started with) */
     if (!have_path) { DWORD k = GetEnvironmentVariableW(L"PATH", path, 8192); if (k >= 8192) path[0] = 0; have_path = 1; }
     {   /* the toolchain first on PATH; Python in UTF-8 mode (the sources are UTF-8) */
@@ -496,7 +496,8 @@ static DWORD WINAPI build_thread(LPVOID unused)
             GetPrivateProfileStringW(g->kit, L"kit_version", L"", kv, 64, ini_path);
             MultiByteToWideChar(CP_UTF8, 0, version, -1, wv, 64);
             if (kv[0] && wcscmp(kv, wv)) {   /* (none recorded: a first build, or one from before this was kept) */
-                static const wchar_t *old[] = { L"build\\windows\\gen", L"build\\windows\\gen.new", L"build\\windows\\obj", L"zig-cache" };
+                static const wchar_t *old[] = { L"build\\windows\\gen", L"build\\windows\\gen.new", L"build\\windows\\obj",
+                                                L"build\\windows_us\\gen", L"build\\windows_us\\gen.new", L"build\\windows_us\\obj", L"zig-cache" };
                 for (unsigned i = 0; i < sizeof old / sizeof old[0]; i++) { path_join(p, gdir, old[i]); rm_tree(p); }
             }
             WritePrivateProfileStringW(g->kit, L"kit_version", wv, ini_path);
@@ -519,17 +520,20 @@ static DWORD WINAPI build_thread(LPVOID unused)
         GetPrivateProfileStringW(L"build", L"levels", L"", levels, 256, ini_path);
         if (!levels[0]) MultiByteToWideChar(CP_UTF8, 0, lv[0] ? lv : "0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18", -1, levels, 256);
         bld_status(0.32, L"Starting the build");
-        if (!run_build(gdir, d->path, levels, 0.32, 0.99)) goto out;
+        if (!run_build(gdir, d->path, levels, g->rgn ? g->rgn : L"pal", 0.32, 0.99)) goto out;
     }
     /* 5. keep what the game needs: the intermediate files (about 1.3 GB) and the bytes read from the disc go */
     bld_status(0.99, L"Cleaning up");
     {
-        static const wchar_t *junk[] = { L"build\\windows\\gen", L"build\\windows\\gen.new", L"build\\windows\\obj", L"build\\levels", L"build\\disc",
+        static const wchar_t *junk[] = { L"gen", L"gen.new", L"obj", L"region", L"rac1_game.pdb", L"rac1_wrapper.pdb", L"asm_stubs.c.lib" };   /* (in the build's own folder) */
-                                         L"build\\windows\\rac1_game.pdb", L"build\\windows\\rac1_wrapper.pdb", L"build\\windows\\asm_stubs.c.lib", L"zig-cache" };
+        static const wchar_t *junk2[] = { L"build\\levels", L"build\\levels_us", L"build\\disc", L"zig-cache" };
-        for (unsigned i = 0; i < sizeof junk / sizeof junk[0]; i++) { path_join(p, gdir, junk[i]); rm_tree(p); }
+        const wchar_t *bd = g->rgn && wcscmp(g->rgn, L"pal") ? L"build\\windows_" : L"build\\windows";
+        for (unsigned i = 0; i < sizeof junk / sizeof junk[0]; i++) {
+            swprintf(p, MAX_PATH, L"%ls\\%ls%ls\\%ls", gdir, bd, g->rgn && wcscmp(g->rgn, L"pal") ? g->rgn : L"", junk[i]); rm_tree(p); }
+        for (unsigned i = 0; i < sizeof junk2 / sizeof junk2[0]; i++) { path_join(p, gdir, junk2[i]); rm_tree(p); }
     }
     {   wchar_t wv[64]; MultiByteToWideChar(CP_UTF8, 0, version, -1, wv, 64);
-        WritePrivateProfileStringW(g->kit, L"built_version", wv, ini_path); }
+        WritePrivateProfileStringW(g->kit, g->rgn ? L"built_version_us" : L"built_version", wv, ini_path); }
     bld_status(1.0, L"Built");
 out:
     free(js);