added ntscu-u support

.gitignore

@@ -6,3 +6,4 @@ games/*/zig-cache/
 games/*/build_log.txt
 __pycache__/
 *.pyc
+release/*/

README.md

@@ -2,31 +2,46 @@
 
 Native PC versions of the PlayStation 2 Ratchet & Clank games, built on your own PC from your own disc.
 
-Project Solana is a launcher and a set of static recompilations. Each game's code is translated ahead of time from the game's executable into C, with the decompiled source from OpenRAC where it exists, and compiled into a native Windows program that reads everything else (models, textures, sound, movies) from your disc image at run time.
+Project Solana is a launcher and a set of static recompilations. Each game's code is translated ahead of time from the game's executable into C, with the decompiled source from OpenRAC where it exists, and compiled into a native Windows or Linux program that reads everything else (models, textures, sound, movies) from your disc image at run time.
 
 **Nothing from the games is distributed here.** The launcher downloads only tools and source code; the game is built on your machine from a disc image you provide.
 
-## Playing
+## Playing on Windows
 
 1. Download `Project Solana.exe` from the [latest launcher release](../../releases) and put it in a folder of its own (it keeps its files next to itself).
 2. Run it and add your disc image: **Add disc image**, **Scan a folder**, or drag an `.iso` onto the window.
 3. Select the game and press **Build**. The first build downloads about 95 MB (Python and the zig compiler, then the game's build kit) and builds the game from your disc: about 15 minutes on a recent 6 to 8 core CPU, an hour or more on an older 4 core one. It only happens once per game version.
 4. Press **Play**.
 
+## Playing on Linux and Steam Deck
+
+The game runs natively on Linux (no Wine or Proton). The Linux launcher is a single script, `launcher/linux/project-solana` (download it from the [latest launcher release](../../releases)). Start it the first time with `python3`; that also makes it executable, so `./project-solana` works afterwards:
+
+```
+python3 project-solana build ~/Games/ratchet.iso   # once: about 95 MB of downloads, then the same build as on Windows
+./project-solana play                        # --upscaled / --native, --fullscreen / --windowed, --vsync (remembered)
+./project-solana desktop                     # adds it to the applications menu; from there, Steam > Add a Non-Steam Game
+```
+
+With both the PAL and the NTSC-U version built, `play` starts the one built or played last; `play --game pal` or `play --game us` picks one (`log` and `report` take `--game` too).
+
+It needs `python3` and the **32-bit** SDL2 and OpenGL libraries. On SteamOS (Steam Deck, Steam Machine) nothing needs installing: the launcher uses the copy of SDL2 that comes with Steam. Elsewhere, `./project-solana status` says what to install for your distribution. Everything is kept in `~/.local/share/project-solana`. On a Steam Deck, run the build once in Desktop Mode (Konsole), then add the game to Steam with `desktop`; it starts full screen there (and on any SteamOS device), with the Deck's controls as the controller.
+
 ### Supported games
 
 | Game | Region | Disc executable | Status |
 |---|---|---|---|
 | Ratchet & Clank | PAL (v2.00) | `SCES_509.16`, MD5 `d86429e2eca4e1da7d7f1cc656af2574` | Playable start to finish |
-| Ratchet & Clank | NTSC-U, NTSC-J | `SCUS_971.99`, `SCPS_150.37` | Recognised, not supported yet |
+| Ratchet & Clank | NTSC-U | `SCUS_971.99`, MD5 `0c081d4e0353b419d95e5f95eb0ebacb` | New: boots, menus and all 19 levels load; less tested than PAL |
+| Ratchet & Clank | NTSC-J | `SCPS_150.37` | Recognised, not supported yet |
 | Going Commando, Up Your Arsenal, Deadlocked | all | | Recognised, not supported yet |
 
 ### Requirements
 
-- Windows 10 or 11, 64-bit
+- Windows 10 or 11, 64-bit, or Linux x86-64 (glibc 2.17 or newer, X11 or Wayland) with the 32-bit SDL2 and OpenGL libraries
 - A CPU with SSE4.2 (Intel since 2008, AMD since 2011) and 8 GB of RAM
 - A GPU with OpenGL 3.3 (tested: RTX 3060, Upscaled at 3x)
-- A controller: Xbox, DualShock 4 or DualSense, over USB or Bluetooth
+- A controller: Xbox, DualShock 4 or DualSense, over USB or Bluetooth (on Linux, anything SDL2 knows, including the Steam Deck's controls)
 - About 2.5 GB free while building, 600 MB after (the game plus the build tools, kept for rebuilds)
 
 ### Settings
@@ -36,6 +51,8 @@ Resolution (Native, or Upscaled to a whole multiple of the PS2's 448 lines that
 ### Reporting problems
 
 - **Open log** shows `rac1_log.txt` (or the build's `build_log.txt` when a build failed).
+- On Linux, `project-solana report` writes one file with everything a report needs (system details, the 32-bit library check, the launcher, build and game logs, and the previous run's game log); attach that file.
+- A crash writes the function it happened in and the call chain into the game log, and the previous run's log is kept as `rac1_log.prev.txt`.
 - **F9** in the game records the next few seconds of graphics input to `gsrec_NN.bin` (about 260 MB per second of capture); attach it to a report together with what looked wrong.
 
 ## Repository layout
@@ -43,7 +60,8 @@ Resolution (Native, or Upscaled to a whole multiple of the PS2's 448 lines that
 | Path | What |
 |---|---|
 | `launcher/` | `Project Solana.exe`: Win32 + GDI, no dependencies. `python launcher/build_launcher.py` builds it (needs zig: `pip install ziglang`) |
-| `games/rac1/` | Ratchet & Clank's build kit: `build.py`, the recompilers (`tools/`), the host engine (`host/`: GS in OpenGL and software, DMA, VU, SPU2 and 989snd, disc, pad, memory card, movies), the wrapper (`launcher/`) |
+| `launcher/linux/project-solana` | the Linux launcher (Python 3, no dependencies) |
+| `games/rac1/` | Ratchet & Clank's build kit: `build.py`, the recompilers (`tools/`), the host engine (`host/`: GS in OpenGL and software, DMA, VU, SPU2 and 989snd, disc, pad, memory card, movies; Win32 and Linux/SDL2 platform layers), the wrapper (`launcher/`) |
 | `games/rac1/dev/` | developer tools (GS capture replay, VU reference check); not part of the kit |
 | `release/make_release.py` | makes a game's release files |
 | `THIRD_PARTY_NOTICES.md` | pl_mpeg (MIT), jak-project's 989snd tables (ISC) |
@@ -56,22 +74,26 @@ The launcher fetches from the release named after the game (`releases/download/r
 |---|---|
 | `rac1.json` | version, file names, sizes and SHA-256s, the levels to build |
 | `toolchain-win64.zip` | Python 3.12 (python-build-standalone), rabbitizer, zig 0.16; shared by all games, kept in `toolchain\` and fetched again only when its version changes |
+| `toolchain-linux64.zip` | the same for Linux x86-64 (the Linux launcher's) |
 | `rac1-build.zip` | `games/rac1` plus the decomp's source as `decomp/` (no `baserom/`) |
 
-It checks both zips against their SHA-256s, unpacks them next to itself (`toolchain\`, `games\rac1\`), runs `build.py --platform windows --iso <your disc>`, and then deletes the intermediate files (about 1.3 GB) and the bytes it read from the disc, keeping `games\rac1\build\windows` (the game, its saves and logs).
+It checks both zips against their SHA-256s, unpacks them next to itself (`toolchain\`, `games\rac1\`), runs `build.py --platform windows --region pal|us --iso <your disc>`, and then deletes the intermediate files (about 1.3 GB) and the bytes it read from the disc, keeping `games\rac1\build\windows` (PAL) or `games\rac1\build\windows_us` (NTSC-U): the game, its saves and logs. Both versions come from the same kit.
+
+The NTSC-U build is made from the same sources: `tools/region.py` rewrites the decomp and the host engine from PAL addresses into NTSC-U ones with the address map `region/us.json` (functions paired by their code, data by the code that points at it; made by `dev/region_map.py`), and the code itself is translated from the NTSC-U executable and levels. `host/region/us/` holds the little the NTSC-U game needs that PAL's does not.
 
 `solana.ini` `[build]` `release_url=` (a URL or a local folder laid out like the release) and `levels=` override where it fetches from and which levels it builds, for testing.
 
 ## Making a release
 
 ```
-python release/make_release.py rac1 --decomp ../rac1-decomp --version 2026.10.13
+python release/make_release.py rac1 --decomp ../rac1-decomp --version 2026.10.14.2 \
-gh release create rac1 dist/rac1/rac1.json dist/rac1/rac1-build.zip dist/rac1/toolchain-win64.zip
+    --toolchain-zip dist/rac1/toolchain-win64.zip --toolchain-linux-zip dist/rac1/toolchain-linux64.zip   # (reuse, or omit to assemble them)
+gh release create rac1 dist/rac1/rac1.json dist/rac1/rac1-build.zip dist/rac1/toolchain-win64.zip dist/rac1/toolchain-linux64.zip
 ```
 
 To update an existing game release, upload the new files over the old ones (`gh release upload rac1 dist/rac1/* --clobber`). Players get them with **Rebuild the game**.
 
-The launcher itself goes in a release of its own (for example `launcher-v1`) with `Project Solana.exe`.
+The launchers go in a release of their own (for example `launcher-v1`): `Project Solana.exe` and `project-solana` (Linux).
 
 ## Licence
 

games/rac1/README.md

@@ -1,6 +1,6 @@
-# Ratchet & Clank (PAL v2.00): build kit
+# Ratchet & Clank (PAL v2.00, NTSC-U): build kit
 
-A static recompilation of Ratchet & Clank (`SCES_509.16`) for Windows (and Linux, for development).
+A static recompilation of Ratchet & Clank (PAL `SCES_509.16`, and NTSC-U `SCUS_971.99` with `--region us`) for Windows and Linux.
 
 `build.py` reads the game's executable and level programs from your disc image, generates C from them (the decomp's own C where it has it, everything else translated instruction by instruction by `tools/recomp.py`, the VU microcode by `tools/vu_recomp.py`), and compiles it with the host engine in `host/` into `build/windows/rac1_game.dll` and `rac1_wrapper.exe`.
 
@@ -8,6 +8,8 @@ A static recompilation of Ratchet & Clank (`SCES_509.16`) for Windows (and Linux
 python build.py --platform windows --iso "Ratchet & Clank (Europe).iso" --repo ../rac1-decomp --levels 0,1,...,18
 ```
 
+`--region us` builds the NTSC-U version into `build/<platform>_us` from the same sources: `tools/region.py` first writes the decomp and `host/` in NTSC-U addresses (`build/<platform>_us/region`, with `region_report.txt` listing what it changed and what had no counterpart), using the address map `region/us.json` (made by `dev/region_map.py` from both executables and both sets of levels). Code only the NTSC-U game has goes in `host/region/us/`, already in its addresses.
+
 Needs Python 3, `pip install rabbitizer ziglang`. The Project Solana launcher runs this for players with its own copies of both.
 
 | Path | What |
@@ -16,6 +18,7 @@ Needs Python 3, `pip install rabbitizer ziglang`. The Project Solana launcher ru
 | `tools/recomp.py` | MIPS (R5900) to C, the executable and each level |
 | `tools/vu_recomp.py` | VU0/VU1 microprograms to C |
 | `tools/level_dump.py` | a level's program from the disc image |
+| `tools/region.py`, `region/us.json` | the NTSC-U build: the sources rewritten in its addresses |
 | `host/src/gs/` | the Graphics Synthesizer: software and OpenGL 3.3 (upscaling: `UYA_SCALE`) |
 | `host/src/gfx/` | VIF, GIF and the VU runtime |
 | `host/src/sdk/`, `host/src/rac1/` | Sony library entry points, 989snd and the SPU2, levels, boot |

games/rac1/build.py

@@ -25,11 +25,19 @@ ap.add_argument("-j", type=int, default=os.cpu_count() or 2)
 ap.add_argument("-O", default="2")
 ap.add_argument("--iso", default=None)
 ap.add_argument("--levels", default="00")
+ap.add_argument("--region", default="pal", choices=("pal", "us"),
+                help="pal: SCES-50916 (the decomp's own); us: SCUS-97199, made from the same sources by tools/region.py")
 a = ap.parse_args()
+sys.path.insert(0, os.path.join(HERE, "tools"))
+import region as _region
+RC = _region.consts(a.region)
+EXE_NAME = RC["exe"]
 
 zig = [shutil.which("zig")] if shutil.which("zig") else [sys.executable, "-m", "ziglang"]
-target = {"linux": "x86-linux-musl", "windows": "x86-windows-gnu"}[a.platform]
+# Linux: 32-bit glibc 2.17 or newer (any distribution since 2013), linked dynamically so it can load
-out = os.path.join(HERE, "build", a.platform)
+# the system's SDL2 and OpenGL driver (32-bit: i386 libraries)
+target = {"linux": "x86-linux-gnu.2.17", "windows": "x86-windows-gnu"}[a.platform]
+out = os.path.join(HERE, "build", a.platform if a.region == "pal" else "%s_%s" % (a.platform, a.region))
 gen = os.path.join(out, "gen")
 stage = os.path.join(out, "gen.new")
 obj = os.path.join(out, "obj")
@@ -88,14 +96,22 @@ def iso_file(iso, name):
     return None
 
 
-env = dict(os.environ, UYA_PLATFORM=a.platform, RAC_PLATFORM=a.platform)
+env = dict(os.environ, UYA_PLATFORM=a.platform, RAC_PLATFORM=a.platform, UYA_REGION=a.region)
+# another region: the decomp and the host sources rewritten in its addresses (tools/region.py)
+HOST = os.path.join(HERE, "host")
+if a.region != "pal":
+    rdir = os.path.join(out, "region")
+    subprocess.run([sys.executable, os.path.join(HERE, "tools", "region.py"), a.region, a.repo, rdir], check=True)
+    a.repo = os.path.join(rdir, "repo")
+    HOST = os.path.join(rdir, "host")
+env["UYA_HOST"] = HOST
 # the static recompilation is built from the game itself: the executable comes
 # from the disc image (the decomp repo only lends its symbol and function lists)
 if find_iso():
-    _exe = iso_file(find_iso(), "SCES_509.16")
+    _exe = iso_file(find_iso(), EXE_NAME)
     if _exe:
         os.makedirs(os.path.join(HERE, "build", "disc"), exist_ok=True)
-        _p = os.path.join(HERE, "build", "disc", "SCES_509.16")
+        _p = os.path.join(HERE, "build", "disc", EXE_NAME)
         if not os.path.exists(_p) or open(_p, "rb").read() != _exe: open(_p, "wb").write(_exe)
         env["UYA_EXE"] = _p
         print("executable read from the disc image:", find_iso())
@@ -106,7 +122,7 @@ def progress(phase, i, n):
     print("[progress] %s %d %d" % (phase, i, n), flush=True)
 progress("prepare", 0, 3)
 subprocess.run([py, os.path.join(HERE, "tools", "gen_host.py"), a.repo, stage], check=True, env=env)
-dumps = os.path.join(HERE, "build", "levels")
+dumps = os.path.join(HERE, "build", "levels" if a.region == "pal" else "levels_" + a.region)
 # the level programs first: the executable's translation looks at the core
 # addresses their code calls (tools/recomp.py)
 for lv in levels:
@@ -144,7 +160,8 @@ with open(os.path.join(stage, "level_table.c"), "w") as f:
 sync(stage, gen)
 
 common = ["-target", target, "-O" + a.O, "-g", "-march=nehalem", "-mfpmath=sse", "-malign-double", "-fno-strict-aliasing", "-fwrapv",
-          "-fno-PIE", "-fno-stack-protector", "-I", os.path.join(gen, "include"), "-I", os.path.join(HERE, "host", "include")]
+          "-fno-PIE", "-fno-stack-protector", "-I", os.path.join(gen, "include"), "-I", os.path.join(HOST, "include")]
+if a.platform == "linux": common += ["-D_FILE_OFFSET_BITS=64"]   # (32-bit glibc: disc images are over 2 GB)
 # frontbin code: keep it permissive, it was written for gcc 2.95
 menu_flags = ["-w", "-Wno-error=incompatible-pointer-types", "-Wno-error=int-conversion",
               "-Wno-error=implicit-function-declaration", "-Wno-error=implicit-int",
@@ -165,9 +182,9 @@ jobs = []   # (source, extra flags, object name, forced header)
 for c in sorted(glob.glob(os.path.join(gen, "*.c"))):
     jobs.append((c, menu_flags if os.path.basename(c) not in ("tramp_table.c", "level_table.c") else [], None, None))
 jobs.append((os.path.join(gen, "abs_syms.s"), [], None, None))
-for c in sorted(glob.glob(os.path.join(HERE, "host", "overrides", "*.c"))):
+for c in sorted(glob.glob(os.path.join(HOST, "overrides", "*.c"))):
     jobs.append((c, menu_flags, None, None))
-for c in sorted(glob.glob(os.path.join(HERE, "host", "src", "**", "*.c"), recursive=True)):
+for c in sorted(glob.glob(os.path.join(HOST, "src", "**", "*.c"), recursive=True)):
     jobs.append((c, ["-Wall", "-Wno-unused"], None, None))
 for LV, *_ in level_info:
     lg = os.path.join(gen, "l" + LV)
@@ -178,7 +195,7 @@ for LV, *_ in level_info:
     jobs.append((os.path.join(lg, "abs_syms.s"), [], os.path.join("l" + LV, "abs_syms.s.o"), None))
     # the executable's main-segment overrides that the level has: the level's copies
     for nm in open(os.path.join(lg, "overrides.txt")).read().split():
-        c = os.path.join(HERE, "host", "overrides", nm + ".c")
+        c = os.path.join(HOST, "overrides", nm + ".c")
         jobs.append((c, menu_flags + ["-include", hdr], os.path.join("l" + LV, "override_" + nm + ".c.o"), hdr))
 
 
@@ -191,19 +208,26 @@ if not os.path.exists(_stamp) or open(_stamp).read() != _flags:
 flags_time = os.path.getmtime(_stamp)
 
 # headers every generated file sees: a change rebuilds them all
-headers_time = max([flags_time] + [os.path.getmtime(h) for h in glob.glob(os.path.join(gen, "include", "*.h")) + glob.glob(os.path.join(HERE, "host", "include", "*.h"))])
+headers_time = max([flags_time] + [os.path.getmtime(h) for h in glob.glob(os.path.join(gen, "include", "*.h")) + glob.glob(os.path.join(HOST, "include", "*.h"))])
 
 
 def cc(job):
     src, extra, oname, hdr = job
     o = os.path.join(obj, oname or (os.path.basename(src) + ".o"))
-    if os.path.exists(o) and src.endswith(".c") and os.path.join(HERE, "host") not in src:
+    if os.path.exists(o) and src.endswith(".c") and HOST not in src:
         t = os.path.getmtime(o)
         if t > os.path.getmtime(src) and t > headers_time and (not hdr or t > os.path.getmtime(hdr)):
             return o, None
     r = subprocess.run(zig + ["cc"] + common + extra + ["-c", src, "-o", o], capture_output=True, text=True)
     return o, (r.stderr if r.returncode else None)
 
+try:   # Linux: the link opens every object at once (~5,700); the default soft limit is often 1,024
+    import resource
+    _s, _h = resource.getrlimit(resource.RLIMIT_NOFILE)
+    if _s != resource.RLIM_INFINITY and (_h == resource.RLIM_INFINITY or _s < _h):
+        resource.setrlimit(resource.RLIMIT_NOFILE, (_h if _h != resource.RLIM_INFINITY else 1 << 20, _h))
+except (ImportError, ValueError, OSError):
+    pass
 progress("compile", 0, len(jobs))
 with ThreadPoolExecutor(a.j) as ex:
     futs = [ex.submit(cc, j) for j in jobs]
@@ -228,14 +252,17 @@ if errs:
     sys.exit("%d files failed to compile" % len(errs))
 if a.platform == "linux":
     exe = os.path.join(out, "rac1_wrapper")
-    link = zig + ["cc", "-target", target, "-o", exe, objs_rsp("link")] + ["-static", "-no-pie", "-Wl,--image-base=0x40000000"]
+    # at 0x40000000 like the Windows DLL (lower addresses are the PS2's); its symbols exported
+    # (-rdynamic) so the crash handler can name functions
+    link = zig + ["cc", "-target", target, "-o", exe, objs_rsp("link")] + ["-no-pie", "-rdynamic", "-Wl,--image-base=0x40000000",
+                                                                            "-ldl", "-lpthread", "-lm"]
     r = subprocess.run(link, capture_output=True, text=True)
     if r.returncode:
         print(r.stderr[-6000:]); sys.exit("link failed")
 else:
     # the menu and runtime as a DLL at 0x40000000, plus the small launcher that owns PS2 RAM
     dll = os.path.join(out, "rac1_game.dll")
-    link = zig + ["cc", "-shared", "-target", target, "-o", dll, objs_rsp("link")] + ["-Wl,--image-base=0x40000000", "-Wl,--no-dynamicbase", "-lshell32", "-lwinmm", "-lgdi32", "-luser32"]
+    link = zig + ["cc", "-shared", "-target", target, "-o", dll, objs_rsp("link")] + ["-Wl,--image-base=0x40000000", "-Wl,--no-dynamicbase", "-lshell32", "-lwinmm", "-lgdi32", "-luser32", "-ladvapi32"]
     r = subprocess.run(link, capture_output=True, text=True)
     if r.returncode:
         print(r.stderr[-6000:]); sys.exit("link failed (dll)")

games/rac1/dev/region_map.py

@@ -0,0 +1,267 @@
+#!/usr/bin/env python3
+"""region_map.py DECOMP_REPO PAL_EXE US_EXE PAL_LEVELS US_LEVELS OUT_JSON
+
+Developer tool (needs both executables): the PAL-to-US address map that lets the
+build make the US version (SCUS-97199) from the same sources as PAL (SCES-50916).
+The output is addresses only (no game bytes) and ships in the build kit as
+region/us.json; tools/region.py reads it.
+
+  functions  the executable's: OpenRAC's config/overlays/us_map.tsv (code alone),
+             PAL name -> US address and size, with how it was paired; and each
+             level's, from the same table and the catalogue's places
+             (config/overlays/functions.tsv), PAL place -> US place
+  data       the executable's data sections: PAL and US words aligned (an address
+             in either one counts as equal: the pointers moved), giving runs of
+             PAL addresses that share one shift. Sections whose size did not
+             change move as one. Checked against the data addresses the paired
+             functions form with lui/addiu (both builds load the same data).
+  levels     each level's data records (lit, bss, data, vtbl, camvtbl, sndvtbl)
+             aligned the same way (PAL_LEVELS and US_LEVELS: level_dump.py output)
+"""
+import json, os, re, struct, sys
+from collections import Counter, defaultdict
+
+REPO, PAL_EXE, US_EXE, PAL_LV, US_LV, OUT = sys.argv[1:7]
+pal, us = open(PAL_EXE, "rb").read(), open(US_EXE, "rb").read()
+
+
+def sections(e):
+    shoff, _, shnum, shstr = struct.unpack_from("<I", e, 32)[0], *struct.unpack_from("<HHH", e, 46)
+    S = [struct.unpack_from("<10I", e, shoff + i * 40) for i in range(shnum)]
+    names = e[S[shstr][4]:S[shstr][4] + S[shstr][5]]
+    out = {}
+    for s in S:
+        n = names[s[0]:names.index(b"\0", s[0])].decode()
+        if s[3]: out[n] = {"addr": s[3], "off": s[4], "size": s[5], "nobits": s[1] == 8}
+    return out
+
+
+SP, SU = sections(pal), sections(us)
+
+
+def seg_bytes(e, a, n):
+    phoff, phnum = struct.unpack_from("<I", e, 28)[0], struct.unpack_from("<H", e, 44)[0]
+    for i in range(phnum):
+        t, off, va, pa, fs = struct.unpack_from("<5I", e, phoff + 32 * i)
+        if va <= a and a + n <= va + fs: return e[off + a - va: off + a - va + n]
+
+
+# ---- functions -----------------------------------------------------------------------------------
+fsize = {}
+for u in json.load(open(os.path.join(REPO, "progress", "report.json")))["units"]:
+    for f in u.get("functions", []):
+        md = f.get("metadata") or {}
+        if "virtual_address" in md: fsize[f["name"]] = (int(md["virtual_address"]), int(f["size"]))
+places = {}   # catalogue name -> {level: [PAL addresses]}
+for l in open(os.path.join(REPO, "config", "overlays", "functions.tsv")):
+    if l.startswith("#") or not l.strip(): continue
+    f = l.rstrip("\n").split("\t")
+    d = defaultdict(list)
+    for x in f[5].split(","):
+        if ":" in x: d[x.split(":")[0]].append(int(x.split(":")[1], 16))
+    places[f[0]] = d
+
+exe_funcs, exe_us_only = {}, []
+lv_rows = defaultdict(list)
+for l in open(os.path.join(REPO, "config", "overlays", "us_map.tsv")):
+    if l.startswith("#") or not l.strip(): continue
+    prog, ua, sz, name, method, sim = l.rstrip("\n").split("\t")[:6]
+    ua, sz = int(ua, 16), int(sz)
+    if prog == "boot":
+        q = re.fullmatch(r"func_([0-9A-F]{8})", name)
+        if q: exe_funcs["%08X" % int(q.group(1), 16)] = [ua, sz, method]
+        else: exe_us_only.append([ua, sz])
+    else:
+        lv_rows[prog[6:]].append((ua, sz, name, method))
+levels = {}
+for lv, rows in sorted(lv_rows.items()):
+    by_name = defaultdict(list)
+    only = []
+    for ua, sz, name, method in sorted(rows):
+        if name in places and places[name].get(lv): by_name[name].append((ua, sz, method))
+        else: only.append([ua, sz])
+    m = {}
+    for name, us_pl in by_name.items():
+        for pa, (ua, sz, method) in zip(sorted(places[name][lv]), us_pl):   # several copies: paired in address order
+            m["%08X" % pa] = [ua, sz, method]
+    levels[lv] = {"funcs": m, "us_only": only}
+
+# ---- data ----------------------------------------------------------------------------------------
+def words(b): return list(struct.unpack("<%dI" % (len(b) // 4), b[:len(b) // 4 * 4]))
+def tok(w): return -1 if 0x00100000 <= w < 0x02000000 else w
+
+
+def align(a_words, b_words, a_base, b_base, W=8):
+    """runs [pal_lo, pal_hi, shift] (bytes) of a PAL range onto a US range"""
+    a, b = [tok(x) for x in a_words], [tok(x) for x in b_words]
+    d, runs, i = 0, [], 0
+    while i < len(a):
+        if 0 <= i + d < len(b) and a[i] == b[i + d]:
+            if runs and runs[-1][2] == d and runs[-1][1] == i: runs[-1][1] = i + 1
+            else: runs.append([i, i + 1, d])
+            i += 1; continue
+        best = None
+        for k in range(0, 1024):
+            for dd in (d - k, d + k):
+                if 0 <= i + dd and i + dd + W <= len(b) and a[i:i + W] == b[i + dd:i + dd + W] and \
+                        sum(1 for x in a[i:i + W] if x not in (0, -1)) >= 3:
+                    best = dd; break
+            if best is not None: break
+        if best is None: i += 1; continue
+        d = best
+    # runs of a few words that only match zeros or pointers are noise: drop the short ones
+    # between two runs of one shift
+    out = []
+    for s, e, dd in runs:
+        sh = (b_base + 4 * (s + dd)) - (a_base + 4 * s)
+        if out and out[-1][2] == sh: out[-1][1] = a_base + 4 * e
+        else: out.append([a_base + 4 * s, a_base + 4 * e, sh])
+    clean = []
+    for k, r in enumerate(out):
+        if 0 < k < len(out) - 1 and r[1] - r[0] <= 16 and out[k - 1][2] == out[k + 1][2] != r[2]:
+            continue
+        if clean and clean[-1][2] == r[2]: clean[-1][1] = r[1]
+        else: clean.append(r)
+    # close the gaps: a gap takes the shift of the run before it
+    for k in range(len(clean) - 1): clean[k][1] = clean[k + 1][0]
+    if clean: clean[0][0], clean[-1][1] = a_base, a_base + 4 * len(a_words)
+    return clean
+
+
+data = []
+for name in ("core.data", "core.rdata", "core.bss", "core.lit", ".lit", ".bss", ".data", "lvl.vtbl", "lvl.camvtbl", "lvl.sndvtbl"):
+    p, u = SP[name], SU[name]
+    if p["nobits"]:   # (no bytes to compare: it moves as one; checked against the code below)
+        data.append([p["addr"], p["addr"] + p["size"], u["addr"] - p["addr"], name])
+        continue
+    for lo, hi, sh in align(words(pal[p["off"]:p["off"] + p["size"]]), words(us[u["off"]:u["off"] + u["size"]]), p["addr"], u["addr"]):
+        data.append([lo, hi, sh, name])
+
+
+def data_shift(a):
+    for lo, hi, sh, _ in data:
+        if lo <= a < hi: return sh
+
+
+# the data addresses paired functions build (lui + addiu/ori/load/store): both builds load the
+# same data, so each is a PAL -> US pair. The executable's functions and every level's (level code
+# uses the executable's data a lot).
+def refs(b):
+    w = struct.unpack("<%dI" % (len(b) // 4), b); hi = {}; out = {}
+    for i, x in enumerate(w):
+        op, rs, rt, imm = x >> 26, (x >> 21) & 31, (x >> 16) & 31, x & 0xFFFF
+        if op == 0x0F: hi[rt] = imm << 16; continue
+        if rs in hi and (op == 0x09 or 0x20 <= op <= 0x3F or op in (0x1A, 0x1B, 0x1E, 0x1F)):
+            out[i] = (hi[rs] + (imm - 0x10000 if imm & 0x8000 else imm)) & 0xFFFFFFFF
+        if op in (0x09, 0x0D, 0x0F) or 0x20 <= op <= 0x27: hi.pop(rt, None)
+    return out
+
+
+anchor_votes = defaultdict(Counter)
+RESIDENT = 0x0015F000   # PAL: from here up, each level program has its own lit/bss/data/tables
+def collect(b1, b2, top):
+    if not b1 or not b2: return
+    r1, r2 = refs(b1), refs(b2)
+    for i, x in r1.items():
+        if i in r2 and SP["core.data"]["addr"] <= x < top:
+            anchor_votes[x][r2[i] - x] += 1
+
+
+for pa_s, (ua, sz, method) in exe_funcs.items():
+    if method == "fingerprint":
+        pa = int(pa_s, 16); collect(seg_bytes(pal, pa, sz), seg_bytes(us, ua, sz), SP[".text"]["addr"])
+lv_text = {}
+for lv in levels:
+    try:
+        tp = [r for r in json.load(open(os.path.join(PAL_LV, "level_" + lv, "manifest.json")))["records"] if r["name"] == "text"][0]
+        tu = [r for r in json.load(open(os.path.join(US_LV, "level_" + lv, "manifest.json")))["records"] if r["name"] == "text"][0]
+    except (OSError, IndexError):
+        continue
+    bp = open(os.path.join(PAL_LV, "level_" + lv, "text.bin"), "rb").read(); bu = open(os.path.join(US_LV, "level_" + lv, "text.bin"), "rb").read()
+    lv_text[lv] = (tp["address"], bp, tu["address"], bu)
+    for pa_s, (ua, sz, method) in levels[lv]["funcs"].items():
+        if method != "fingerprint": continue
+        pa = int(pa_s, 16)
+        collect(bp[pa - tp["address"]:pa - tp["address"] + sz], bu[ua - tu["address"]:ua - tu["address"] + sz], RESIDENT)
+
+# data sections move down (by 0x100 to 0x400): a "shift" outside that is a constant, not an address
+def align_shift(a):
+    for lo, hi, sh, _ in data:
+        if lo <= a < hi: return sh
+
+
+anchors = {}
+for x, c in anchor_votes.items():
+    sh, n = c.most_common(1)[0]
+    if not (-0x600 <= sh <= -0x40 and sh % 4 == 0): continue
+    # trusted when several references agree and none disagrees, or when the bytes agree
+    # (a lone or contested pair is often base + a constant the compiler folded differently)
+    if (n >= 2 and len(c) == 1) or sh == align_shift(x): anchors[x] = sh
+
+
+def build(sec_runs):
+    """the final runs for one section: anchors where the code has them; between two anchors of
+    one shift, that shift; between anchors of different shifts, the alignment's boundary if it
+    has one there, else halfway"""
+    lo, hi = sec_runs[0][0], sec_runs[-1][1]
+    pts = sorted((a, s) for a, s in anchors.items() if lo <= a < hi)
+    if not pts: return [r[:3] for r in sec_runs]
+    def align_at(a):
+        for r in sec_runs:
+            if r[0] <= a < r[1]: return r[2]
+    out = []
+    def put(a0, a1, sh):
+        if a0 >= a1: return
+        if out and out[-1][2] == sh and out[-1][1] == a0: out[-1][1] = a1
+        else: out.append([a0, a1, sh])
+    # before the first anchor: the alignment, as long as it agrees with that anchor, else the anchor's
+    put(lo, pts[0][0], pts[0][1])
+    for (a, s), (b, t) in zip(pts, pts[1:]):
+        if s == t: put(a, b, s); continue
+        # the boundary: the first address from a where the alignment says t
+        cut = None
+        for r in sec_runs:
+            if r[2] == t and a < r[0] <= b: cut = r[0]; break
+        if cut is None: cut = (a + (b - a) // 2) & ~3
+        put(a, cut, s); put(cut, b, t)
+    put(pts[-1][0], hi, pts[-1][1])
+    return out
+
+
+final = []
+for name in ("core.data", "core.rdata", "core.bss", "core.lit", ".lit", ".bss", ".data", "lvl.vtbl", "lvl.camvtbl", "lvl.sndvtbl"):
+    rs = [r for r in data if r[3] == name]
+    for r in build(rs): final.append(r + [name])
+data = final
+
+
+def data_shift(a):
+    for lo, hi, sh, _ in data:
+        if lo <= a < hi: return sh
+
+
+agree = sum(1 for x, s in anchors.items() if data_shift(x) == s)
+print("data map: %d runs; %d code anchors (%d agree with the map)" % (len(data), len(anchors), agree))
+for lo, hi, sh, n in data[:80]: print("   %-11s %08X-%08X %+X" % (n, lo, hi, sh))
+
+# ---- level data ----------------------------------------------------------------------------------
+level_data = {}
+for lv in sorted(levels):
+    pm = os.path.join(PAL_LV, "level_" + lv, "manifest.json"); um = os.path.join(US_LV, "level_" + lv, "manifest.json")
+    if not (os.path.exists(pm) and os.path.exists(um)): continue
+    P = {r["name"]: r for r in json.load(open(pm))["records"]}; U = {r["name"]: r for r in json.load(open(um))["records"]}
+    runs = []
+    for rec in ("lit", "bss", "data", "vtbl", "camvtbl", "sndvtbl"):
+        p, u = P[rec], U[rec]
+        if p["bytes"] == u["bytes"] or rec == "bss":
+            runs.append([p["address"], p["address"] + p["bytes"], u["address"] - p["address"], rec]); continue
+        pb = open(os.path.join(PAL_LV, "level_" + lv, rec + ".bin"), "rb").read()
+        ub = open(os.path.join(US_LV, "level_" + lv, rec + ".bin"), "rb").read()
+        for lo, hi, sh in align(words(pb), words(ub), p["address"], u["address"]): runs.append([lo, hi, sh, rec])
+    level_data[lv] = runs
+
+json.dump({"region": "us", "exe": "SCUS_971.99", "pal_exe": "SCES_509.16",
+           "sections": {n: [SP[n]["addr"], SP[n]["size"], SU[n]["addr"], SU[n]["size"]] for n in SP if n in SU},
+           "exe_funcs": exe_funcs, "exe_us_only": exe_us_only, "data": data,
+           "levels": levels, "level_data": level_data}, open(OUT, "w"), separators=(",", ":"))
+print("wrote %s: %d executable functions (%d US-only), %d levels" % (OUT, len(exe_funcs), len(exe_us_only), len(levels)))

games/rac1/host/include/uya_runtime.h

@@ -23,6 +23,7 @@ extern const int uya_tramp_count;
 /* platform layer */
 int  plat_map_fixed(uint32_t addr, uint32_t size, int exec);  /* 0 on success */
 int  plat_install_trap_handler(void);                         /* calls into unmapped PS2 code */
+void plat_log_system(void);                                   /* the system, for bug reports (once, at start) */
 void plat_flush_icache(void *p, size_t n);
 const char *plat_exe_dir(void);                                /* folder of the running program */
 void plat_at_exit(void);

games/rac1/host/region/us/mc_us.c

@@ -0,0 +1,11 @@
+/* NTSC-U (SCUS_971.99) only: library functions the PAL game does not have.
+ * Written in US addresses; tools/region.py copies this folder into the US
+ * host tree as it is (no rewriting) and the PAL build never sees it. */
+#include "uya_host.h"
+#include "uya_runtime.h"
+
+s32 rac1_mc_chdir(s32 port, s32 slot, const char *dir, char *prev);
+
+/* 0x001240C0: sceMcChdir(port, slot, newDir, curDir), RPC 0x0C (its sync
+ * callback 0x00124038 copies the previous directory into curDir) */
+s32 func_001240C0(s32 port, s32 slot, char *dir, char *cur) { return rac1_mc_chdir(port, slot, dir, cur); }

games/rac1/host/src/gs/gs.c

@@ -2095,7 +2095,7 @@ static void uya_gs_gif_(const u32 *d, u32 qwc)
 
 /* ---- init and display ---- */
 /* UYA_RENDERER=gl|soft (or renderer= in rac1_wrapper.cfg): the GPU draws
- * (default where there is GL: Windows) or the software rasterizer; GL falls
+ * (default) or the software rasterizer; GL falls
  * back to software if it cannot start */
 const char *uya_renderer_cfg;
 void uya_gs_init(void)
@@ -2107,11 +2107,7 @@ void uya_gs_init(void)
     {
         const char *r = getenv("UYA_RENDERER");
         if (!r) r = uya_renderer_cfg;
-#ifdef _WIN32
+        if (!r) r = "gl";   /* (with no window, as headless on Linux, GL does not start: software) */
-        if (!r) r = "gl";
-#else
-        if (!r) r = "soft";
-#endif
         if (!strcmp(r, "gl")) {
             if (plat_gl_init() && gsgl_init(vram)) { plat_gl_active = 1; uya_log("GS: drawing with OpenGL"); }
             else uya_log("GS: OpenGL not available, drawing in software");

games/rac1/host/src/main.c

@@ -63,6 +63,7 @@ int uya_main(int argc, char **argv)
     uya_log_open();
     uya_log("Project Solana: starting");
     plat_install_trap_handler();   /* first: a crash from here on is logged */
+    plat_log_system();
     {   /* renderer=gl|soft in the cfg (UYA_RENDERER overrides it) */
         extern const char *uya_renderer_cfg;
         const char *r = cfg_value("renderer");

games/rac1/host/src/platform_linux.c

@@ -1,5 +1,6 @@
 /* Linux (i386) platform layer: fixed mappings and the int3 trap for calls
- * into PS2 code the host does not implement. */
+ * into PS2 code the host does not implement; the window, OpenGL, gamepads and
+ * sound through SDL2 (platform_linux_sdl.h), or headless for testing. */
 #ifdef __linux__
 #define _GNU_SOURCE
 #include "uya_runtime.h"
@@ -81,10 +82,41 @@ static void on_trap(int sig, siginfo_t *si, void *ctxv)
     g[REG_ESP] = esp + 4;
 }
 
+#include <dlfcn.h>
+static void *dladdr_base(void)   /* the game executable's own load address */
+{
+    static void *b; Dl_info di;
+    if (!b && dladdr((void *)&dladdr_base, &di)) b = di.dli_fbase;
+    return b;
+}
+/* "name+0xoff" for an address in the game (its symbols are exported: -rdynamic) */
+static const char *sym_name(uint32_t a, char *buf, int cap)
+{
+    Dl_info di;
+    if (dladdr((void *)(uintptr_t)a, &di) && di.dli_sname) snprintf(buf, cap, "%s+0x%X", di.dli_sname, a - (uint32_t)(uintptr_t)di.dli_saddr);
+    else snprintf(buf, cap, "?");
+    return buf;
+}
 static void on_fault(int sig, siginfo_t *si, void *ctxv)
 {
+    static volatile int in_crash;
+    if (__atomic_exchange_n(&in_crash, 1, __ATOMIC_SEQ_CST)) _exit(1);   /* (a crash while reporting one) */
     ucontext_t *uc = ctxv;
-    uya_log("crash: signal %d, fault address %p, eip=0x%08X", sig, si->si_addr, (uint32_t)uc->uc_mcontext.gregs[REG_EIP]);
+    greg_t *g = uc->uc_mcontext.gregs;
+    char nm[256];
+    uint32_t eip = (uint32_t)g[REG_EIP];
+    uya_log("crash: signal %d, fault address %p, eip=0x%08X", sig, si->si_addr, eip);
+    uya_log("crash: in %s", sym_name(eip, nm, sizeof nm));
+    {   /* the return addresses on the stack, named: the call chain */
+        const uint32_t *sp = (const uint32_t *)(uintptr_t)(uint32_t)g[REG_ESP];
+        Dl_info di;
+        for (int i = 0, shown = 0; i < 256 && shown < 12; i++) {
+            uint32_t v = sp[i];
+            if (v >= 0x40000000u && v < 0x80000000u && dladdr((void *)(uintptr_t)v, &di) && di.dli_sname && di.dli_fbase == dladdr_base()) {
+                uya_log("crash: called from %s", sym_name(v, nm, sizeof nm)); shown++;
+            }
+        }
+    }
     uya_report();
     _exit(1);
 }
@@ -148,15 +180,37 @@ int plat_install_trap_handler(void)
     sigaction(SIGBUS, &sa, 0);
     sigaction(SIGILL, &sa, 0);
     sigaction(SIGFPE, &sa, 0);
+    sigaction(SIGABRT, &sa, 0);   /* (a library giving up: assert, abort) */
     return 0;
 }
+
+#include <sys/utsname.h>
+void plat_log_system(void)
+{
+    struct utsname u;
+    char os[160] = "unknown", line[256], cpu[160] = "?";
+    FILE *f = fopen("/etc/os-release", "r");
+    while (f && fgets(line, sizeof line, f))
+        if (!strncmp(line, "PRETTY_NAME=", 12)) { char *v = line + 12; if (*v == '"') v++; v[strcspn(v, "\"\n")] = 0; snprintf(os, sizeof os, "%s", v); }
+    if (f) fclose(f);
+    f = fopen("/proc/cpuinfo", "r");
+    while (f && fgets(line, sizeof line, f))
+        if (!strncmp(line, "model name", 10)) { char *v = strchr(line, ':'); if (v) { v += 2; v[strcspn(v, "\n")] = 0; snprintf(cpu, sizeof cpu, "%s", v); break; } }
+    if (f) fclose(f);
+    if (uname(&u) == 0) uya_log("system: Linux %s (%s), %s", u.release, u.machine, os);
+    uya_log("system: %s, %ld cores, %ld MB RAM", cpu, sysconf(_SC_NPROCESSORS_ONLN), (long)((long long)sysconf(_SC_PHYS_PAGES) * sysconf(_SC_PAGESIZE) >> 20));
+    const char *st = getenv("XDG_SESSION_TYPE"), *de = getenv("XDG_CURRENT_DESKTOP");
+    uya_log("system: session %s%s%s, DISPLAY %s, WAYLAND_DISPLAY %s%s", st ? st : "?", de ? ", desktop " : "", de ? de : "",
+            getenv("DISPLAY") ? getenv("DISPLAY") : "-", getenv("WAYLAND_DISPLAY") ? getenv("WAYLAND_DISPLAY") : "-",
+            getenv("SteamDeck") && !strcmp(getenv("SteamDeck"), "1") ? ", Steam Deck" : getenv("SteamAppId") ? ", started by Steam" : "");
+}
 #endif
 
 #ifdef __linux__
-/* No window on Linux (yet): with UYA_DUMP=dir every 15th presented frame is
+/* Headless (no display, no SDL2, or UYA_HEADLESS / UYA_DUMP set): with UYA_DUMP=dir every
- * written there as a PPM, so output can be checked. */
+ * 15th presented frame is written there as a PPM, so output can be checked. */
 #include <stdlib.h>
-int plat_present(const uint32_t *px, int w, int h)
+static int hl_present(const uint32_t *px, int w, int h)
 {
     static unsigned long n;
     const char *dir = getenv("UYA_DUMP");
@@ -189,8 +243,7 @@ static const struct { const char *name; unsigned bit; } pad_names[] = {
     {"lup", 0x10000}, {"ldown", 0x20000}, {"lleft", 0x40000}, {"lright", 0x80000},
     {"rup", 0x100000}, {"rdown", 0x200000}, {"rleft", 0x400000}, {"rright", 0x800000},
 };
-int plat_pad_sticks(uint8_t st[4]) { (void)st; return 0; }
+static unsigned hl_buttons(void)
-unsigned plat_pad_buttons(void)
 {
     const char *s = getenv("UYA_KEYS");
     unsigned now = uya_frame_count(), bits = 0;
@@ -211,18 +264,9 @@ unsigned plat_pad_buttons(void)
     return bits;
 }
 /* UYA_SKIPMOVIES=n skips the first n movies (plat_movie_started counts them) */
-static int movies_started;
-void plat_movie_started(void) { movies_started++; }
-int plat_key_skip(void)
-{
-    const char *e = getenv("UYA_SKIPMOVIES");
-    if (e && movies_started <= atoi(e)) return 1;
-    return (plat_pad_buttons() & 0x4008) != 0;
-}
-void plat_poll(void) {}
 
 /* Sound on Linux: with UYA_DUMP set, each sound is saved there as a WAV file. */
-void plat_audio_play(const int16_t *lr, size_t frames, int rate, const char *tag)
+static void hl_audio_play(const int16_t *lr, size_t frames, int rate, const char *tag)
 {
     const char *dir = getenv("UYA_DUMP");
     if (!dir) return;
@@ -241,7 +285,6 @@ void plat_audio_play(const int16_t *lr, size_t frames, int rate, const char *tag
     fwrite(lr, 4, frames, f);
     fclose(f);
 }
-void plat_audio_stop(void) {}
 #endif
 
 #ifdef __linux__
@@ -250,7 +293,7 @@ void plat_audio_stop(void) {}
 static uint64_t stream_written, stream_t0;
 uint64_t uya_fixed_clock(void);
 static uint64_t stream_now(void) { uint64_t v = uya_fixed_clock(); return v ? v : plat_time_us(); }   /* UYA_FIXEDTIME: the game's clock */
-void plat_stream_write(const int16_t *lr, int frames)
+static void hl_stream_write(const int16_t *lr, int frames)
 {
     static FILE *f;
     static int tried;
@@ -263,18 +306,46 @@ void plat_stream_write(const int16_t *lr, int frames)
     if (f) fwrite(lr, 4, (size_t)frames, f);
     stream_written += (uint64_t)frames;
 }
-int plat_stream_queued(void)
+static int hl_stream_queued(void)
 {
     if (!stream_t0) return 0;
     uint64_t played = (stream_now() - stream_t0) * 48000 / 1000000;
     return played >= stream_written ? 0 : (int)(stream_written - played);
 }
-/* OpenGL: this build is 32-bit static (musl) and cannot load a GL driver, so
+#include "platform_linux_sdl.h"
- * the GS draws in software (the GL renderer is tested with tools/gs_replay.c) */
+
-int plat_gl_active;
+/* ---- what the game calls: SDL when there is a display, else headless (testing) ---- */
-int plat_gl_init(void) { return 0; }
+int plat_present(const uint32_t *px, int w, int h) { return sdl_on() ? sdl_present(px, w, h) : hl_present(px, w, h); }
-void plat_gl_current(int on) { (void)on; }
+int plat_pad_sticks(uint8_t st[4]) { return sdl_on() ? sdl_sticks(st) : 0; }
-void plat_gl_swap(void) {}
+unsigned plat_pad_buttons(void) { return hl_buttons() | (sdl_on() ? sdl_buttons() : 0); }   /* (UYA_KEYS: scripted presses) */
-void plat_gl_size(int *w, int *h) { *w = 640; *h = 480; }
+/* movies: a fresh press of Cross or Start skips (buttons already held when the movie starts do not);
-void *plat_gl_proc(const char *name) { (void)name; return 0; }
+ * UYA_SKIPMOVIES=n skips the first n movies (testing) */
+static int movies_started;
+static unsigned skip_held;
+void plat_movie_started(void) { movies_started++; pump(); skip_held = plat_pad_buttons() & 0x4008; }
+int plat_key_skip(void)
+{
+    const char *e = getenv("UYA_SKIPMOVIES");
+    if (e && movies_started <= atoi(e)) return 1;
+    pump();
+    unsigned b = plat_pad_buttons() & 0x4008, fresh = b & ~skip_held;
+    skip_held = b;
+    return fresh != 0;
+}
+void plat_poll(void)
+{
+    if (!win) return;
+    pump();
+    if (win_closed) { uya_log("window closed"); exit(0); }
+}
+void plat_audio_play(const int16_t *lr, size_t frames, int rate, const char *tag) { if (sdl_on()) sdl_audio_play(lr, frames, rate); else hl_audio_play(lr, frames, rate, tag); }
+void plat_audio_stop(void) { if (sdl_on()) sdl_audio_stop(); }
+void plat_stream_write(const int16_t *lr, int frames) { if (sdl_on()) sdl_stream_write(lr, frames); else hl_stream_write(lr, frames); }
+int plat_stream_queued(void) { return sdl_on() ? sdl_stream_queued() : hl_stream_queued(); }
+/* OpenGL through SDL (none headless: the GS draws in software) */
+int plat_gl_init(void) { return sdl_gl_init(); }
+void plat_gl_current(int on) { sdl_gl_current(on); }
+void plat_gl_swap(void) { sdl_gl_swap(); }
+void plat_gl_size(int *w, int *h) { sdl_gl_size(w, h); }
+void *plat_gl_proc(const char *name) { return sdl_gl_proc(name); }
 #endif

games/rac1/host/src/platform_linux_sdl.h

@@ -0,0 +1,319 @@
+/* platform_linux_sdl.h: the window, OpenGL, gamepads and sound on Linux, through SDL2
+ * (included once by platform_linux.c).
+ *
+ * libSDL2-2.0.so.0 is loaded at run time (dlopen), so the game builds without SDL's
+ * headers and still runs headless (testing) where there is no SDL or no display. Only
+ * the few SDL2 declarations used are written out here (SDL2's ABI is stable).
+ *
+ *   window   one resizable window, 640x480 to start; F11 or UYA_FULLSCREEN=1 for full screen
+ *   OpenGL   a 3.3 core context for the GS renderer (gs_gl.c); vsync with UYA_VSYNC=1
+ *   gamepads SDL's game controller API: Xbox, PlayStation, Switch and Steam Deck controls
+ *            by position (south = Cross, east = Circle, west = Square, north = Triangle)
+ *   sound    two queued audio devices: one-shot sounds (plat_audio_play), and the 48 kHz
+ *            effects stream (plat_stream_write) */
+#include <dlfcn.h>
+
+/* ---- SDL2 declarations (SDL.h, 2.0.x) ---- */
+typedef struct SDL_Window SDL_Window;
+typedef struct SDL_GameController SDL_GameController;
+typedef void *SDL_GLContext;
+typedef uint32_t SDL_AudioDeviceID;
+typedef struct { int freq; uint16_t format; uint8_t channels, silence; uint16_t samples, padding; uint32_t size; void *callback, *userdata; } SDL_AudioSpec;
+typedef struct { int x, y, w, h; } SDL_Rect;
+typedef struct { uint32_t flags; void *format; int w, h, pitch; void *pixels; } SDL_Surface_head;   /* (the start of SDL_Surface) */
+#define SDL_INIT_AUDIO 0x10u
+#define SDL_INIT_VIDEO 0x20u
+#define SDL_INIT_GAMECONTROLLER 0x2000u
+#define SDL_WINDOWPOS_CENTERED 0x2FFF0000
+#define SDL_WINDOW_FULLSCREEN_DESKTOP 0x1001u
+#define SDL_WINDOW_OPENGL 0x2u
+#define SDL_WINDOW_SHOWN 0x4u
+#define SDL_WINDOW_RESIZABLE 0x20u
+#define SDL_WINDOW_ALLOW_HIGHDPI 0x2000u
+#define SDL_QUIT_EV 0x100u
+#define SDL_WINDOWEVENT_EV 0x200u
+#define SDL_KEYDOWN_EV 0x300u
+#define SDL_WINDOWEVENT_CLOSE 14
+#define SDL_SCANCODE_F9 66
+#define SDL_SCANCODE_F11 68
+#define SDL_GL_DOUBLEBUFFER 5
+#define SDL_GL_CONTEXT_MAJOR_VERSION 17
+#define SDL_GL_CONTEXT_MINOR_VERSION 18
+#define SDL_GL_CONTEXT_PROFILE_MASK 21
+#define SDL_GL_CONTEXT_PROFILE_CORE 1
+#define AUDIO_S16LSB 0x8010
+#define SDL_PIXELFORMAT_RGB888 0x16161804u   /* (XRGB8888: the frames' BGRX bytes) */
+
+static struct {
+    int (*Init)(uint32_t);
+    const char *(*GetError)(void);
+    int (*SetHint)(const char *, const char *);
+    SDL_Window *(*CreateWindow)(const char *, int, int, int, int, uint32_t);
+    void (*SetWindowTitle)(SDL_Window *, const char *);
+    int (*SetWindowFullscreen)(SDL_Window *, uint32_t);
+    uint32_t (*GetWindowFlags)(SDL_Window *);
+    void (*GetWindowSize)(SDL_Window *, int *, int *);
+    int (*PollEvent)(void *);
+    void (*PumpEvents)(void);
+    int (*ShowCursor)(int);
+    int (*GL_SetAttribute)(int, int);
+    SDL_GLContext (*GL_CreateContext)(SDL_Window *);
+    int (*GL_MakeCurrent)(SDL_Window *, SDL_GLContext);
+    void (*GL_SwapWindow)(SDL_Window *);
+    void *(*GL_GetProcAddress)(const char *);
+    int (*GL_SetSwapInterval)(int);
+    void (*GL_GetDrawableSize)(SDL_Window *, int *, int *);
+    void *(*GetWindowSurface)(SDL_Window *);
+    int (*UpdateWindowSurface)(SDL_Window *);
+    void *(*CreateRGBSurfaceWithFormatFrom)(void *, int, int, int, int, uint32_t);
+    void (*FreeSurface)(void *);
+    int (*FillRect)(void *, const SDL_Rect *, uint32_t);
+    int (*UpperBlitScaled)(void *, const SDL_Rect *, void *, SDL_Rect *);
+    int (*NumJoysticks)(void);
+    int (*IsGameController)(int);
+    SDL_GameController *(*GameControllerOpen)(int);
+    void (*GameControllerClose)(SDL_GameController *);
+    int (*GameControllerGetAttached)(SDL_GameController *);
+    const char *(*GameControllerName)(SDL_GameController *);
+    uint8_t (*GameControllerGetButton)(SDL_GameController *, int);
+    int16_t (*GameControllerGetAxis)(SDL_GameController *, int);
+    SDL_AudioDeviceID (*OpenAudioDevice)(const char *, int, const SDL_AudioSpec *, SDL_AudioSpec *, int);
+    void (*CloseAudioDevice)(SDL_AudioDeviceID);
+    void (*PauseAudioDevice)(SDL_AudioDeviceID, int);
+    int (*QueueAudio)(SDL_AudioDeviceID, const void *, uint32_t);
+    uint32_t (*GetQueuedAudioSize)(SDL_AudioDeviceID);
+    void (*ClearQueuedAudio)(SDL_AudioDeviceID);
+    void (*GetVersion)(uint8_t v[3]);
+    const char *(*GetCurrentVideoDriver)(void);
+    const char *(*GetCurrentAudioDriver)(void);
+} sdl;
+
+/* 1: SDL is there and a display is (the game draws in a window); 0: headless */
+static int sdl_state = -1;
+static int sdl_on(void)
+{
+    if (sdl_state >= 0) return sdl_state;
+    sdl_state = 0;
+    if (getenv("UYA_HEADLESS") || getenv("UYA_DUMP")) return 0;   /* (testing: frames and sound to files) */
+    if (!getenv("DISPLAY") && !getenv("WAYLAND_DISPLAY")) { uya_log("no display: running without a window"); return 0; }
+    void *h = dlopen("libSDL2-2.0.so.0", RTLD_NOW | RTLD_GLOBAL);
+    if (!h) h = dlopen("libSDL2.so", RTLD_NOW | RTLD_GLOBAL);
+    if (!h) { uya_log("SDL2 not found (%s): install the 32-bit SDL2 library (libSDL2-2.0.so.0, i386)", dlerror()); return 0; }
+    static const char *const names[] = {
+        "SDL_Init", "SDL_GetError", "SDL_SetHint", "SDL_CreateWindow", "SDL_SetWindowTitle", "SDL_SetWindowFullscreen",
+        "SDL_GetWindowFlags", "SDL_GetWindowSize", "SDL_PollEvent", "SDL_PumpEvents", "SDL_ShowCursor",
+        "SDL_GL_SetAttribute", "SDL_GL_CreateContext", "SDL_GL_MakeCurrent", "SDL_GL_SwapWindow", "SDL_GL_GetProcAddress",
+        "SDL_GL_SetSwapInterval", "SDL_GL_GetDrawableSize", "SDL_GetWindowSurface", "SDL_UpdateWindowSurface",
+        "SDL_CreateRGBSurfaceWithFormatFrom", "SDL_FreeSurface", "SDL_FillRect", "SDL_UpperBlitScaled",
+        "SDL_NumJoysticks", "SDL_IsGameController", "SDL_GameControllerOpen", "SDL_GameControllerClose",
+        "SDL_GameControllerGetAttached", "SDL_GameControllerName", "SDL_GameControllerGetButton", "SDL_GameControllerGetAxis",
+        "SDL_OpenAudioDevice", "SDL_CloseAudioDevice", "SDL_PauseAudioDevice", "SDL_QueueAudio", "SDL_GetQueuedAudioSize",
+        "SDL_ClearQueuedAudio", "SDL_GetVersion", "SDL_GetCurrentVideoDriver", "SDL_GetCurrentAudioDriver",
+    };
+    void **fp = (void **)&sdl;
+    for (unsigned i = 0; i < sizeof names / sizeof names[0]; i++)
+        if (!(fp[i] = dlsym(h, names[i]))) { uya_log("SDL2: %s missing (SDL 2.0.10 or newer needed)", names[i]); return 0; }
+    sdl.SetHint("SDL_JOYSTICK_ALLOW_BACKGROUND_EVENTS", "1");
+    sdl.SetHint("SDL_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR", "0");
+    if (sdl.Init(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER | SDL_INIT_AUDIO) != 0 && sdl.Init(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER) != 0) {
+        uya_log("SDL2: cannot start (%s)", sdl.GetError());
+        return 0;
+    }
+    {
+        uint8_t v[3] = {0, 0, 0};
+        const char *vd = sdl.GetCurrentVideoDriver(), *ad = sdl.GetCurrentAudioDriver();
+        sdl.GetVersion(v);
+        uya_log("SDL %d.%d.%d: video %s, audio %s", v[0], v[1], v[2], vd ? vd : "none", ad ? ad : "none");
+    }
+    sdl_state = 1;
+    return 1;
+}
+
+/* ---- the window ---- */
+static SDL_Window *win;
+static SDL_GLContext glc;
+int plat_gl_active;
+static void win_create(uint32_t extra)
+{
+    if (win || !sdl_on()) return;
+    uint32_t fl = SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE | SDL_WINDOW_ALLOW_HIGHDPI | extra;
+    const char *fs = getenv("UYA_FULLSCREEN");
+    if (fs && atoi(fs)) fl |= SDL_WINDOW_FULLSCREEN_DESKTOP;
+    win = sdl.CreateWindow("Project Solana", SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED, 640, 480, fl);
+    if (!win) uya_log("SDL2: no window (%s)", sdl.GetError());
+    else if (fl & SDL_WINDOW_FULLSCREEN_DESKTOP) sdl.ShowCursor(0);
+}
+
+static void toggle_fullscreen(void)
+{
+    if (!win) return;
+    int fs = (sdl.GetWindowFlags(win) & SDL_WINDOW_FULLSCREEN_DESKTOP) == SDL_WINDOW_FULLSCREEN_DESKTOP;
+    sdl.SetWindowFullscreen(win, fs ? 0 : SDL_WINDOW_FULLSCREEN_DESKTOP);
+    sdl.ShowCursor(fs);
+}
+
+static volatile int win_closed;
+static void pump(void)
+{
+    if (!win) return;
+    union { uint32_t type; uint8_t b[64]; struct { uint32_t type, ts, wid; uint8_t state, repeat, p2, p3; int32_t scancode, sym; } key;
+            struct { uint32_t type, ts, wid; uint8_t event; } w; uint8_t pad[128]; } ev;
+    while (sdl.PollEvent(&ev)) {
+        if (ev.type == SDL_QUIT_EV) win_closed = 1;
+        else if (ev.type == SDL_WINDOWEVENT_EV && ev.w.event == SDL_WINDOWEVENT_CLOSE) win_closed = 1;
+        else if (ev.type == SDL_KEYDOWN_EV && !ev.key.repeat) {
+            if (ev.key.scancode == SDL_SCANCODE_F9) { void uya_gs_rec_hotkey(void); uya_gs_rec_hotkey(); }   /* testing: record the GS input */
+            if (ev.key.scancode == SDL_SCANCODE_F11) toggle_fullscreen();
+        }
+    }
+}
+
+/* ---- OpenGL ---- */
+void *sdl_gl_proc(const char *name) { return sdl.GL_GetProcAddress ? sdl.GL_GetProcAddress(name) : 0; }
+static int sdl_gl_init(void)
+{
+    if (!sdl_on()) return 0;
+    sdl.GL_SetAttribute(SDL_GL_CONTEXT_MAJOR_VERSION, 3);
+    sdl.GL_SetAttribute(SDL_GL_CONTEXT_MINOR_VERSION, 3);
+    sdl.GL_SetAttribute(SDL_GL_CONTEXT_PROFILE_MASK, SDL_GL_CONTEXT_PROFILE_CORE);
+    sdl.GL_SetAttribute(SDL_GL_DOUBLEBUFFER, 1);
+    win_create(SDL_WINDOW_OPENGL);
+    if (!win) return 0;
+    glc = sdl.GL_CreateContext(win);
+    if (!glc) { uya_log("GL: no OpenGL 3.3 core context (%s)", sdl.GetError()); return 0; }
+    {   /* UYA_VSYNC=1: swaps wait for the monitor (the game paces itself anyway) */
+        const char *e = getenv("UYA_VSYNC");
+        sdl.GL_SetSwapInterval(e ? atoi(e) : 0);
+    }
+    return 1;
+}
+static void sdl_gl_current(int on) { if (glc) sdl.GL_MakeCurrent(win, on ? glc : 0); }
+static void sdl_gl_swap(void) { if (glc) sdl.GL_SwapWindow(win); }
+static void sdl_gl_size(int *w, int *h)
+{
+    *w = 640; *h = 480;
+    if (win && glc) sdl.GL_GetDrawableSize(win, w, h);   /* pixels (HiDPI) */
+    else if (win) sdl.GetWindowSize(win, w, h);
+}
+
+/* the frame rate in the title, as on Windows */
+static void title_fps(void)
+{
+    extern unsigned long long nsends;
+    static uint64_t t0; static unsigned long long s0;
+    uint64_t t = plat_time_us();
+    if (!t0) { t0 = t; s0 = nsends; return; }
+    if (t - t0 < 1000000) return;
+    char title[64];
+    snprintf(title, sizeof title, "Project Solana - %.0f FPS", (double)(nsends - s0) / ((double)(t - t0) / 1e6));
+    sdl.SetWindowTitle(win, title);
+    t0 = t; s0 = nsends;
+}
+
+/* the software renderer's frames (when there is no OpenGL): scaled into the window, 4:3 */
+static int sdl_present(const uint32_t *px, int w, int h)
+{
+    win_create(0);
+    pump();
+    if (win_closed) return 0;
+    if (!win) return 1;
+    title_fps();
+    if (plat_gl_active || !px) return 1;   /* (GL: the GS thread draws the window) */
+    void *dst = sdl.GetWindowSurface(win);
+    void *src = sdl.CreateRGBSurfaceWithFormatFrom((void *)px, w, h, 32, w * 4, SDL_PIXELFORMAT_RGB888);
+    if (dst && src) {
+        const SDL_Surface_head *d = dst;
+        int vw = d->w, vh = d->w * 3 / 4;
+        if (vh > d->h) { vh = d->h; vw = d->h * 4 / 3; }
+        SDL_Rect r = { (d->w - vw) / 2, (d->h - vh) / 2, vw, vh };
+        sdl.FillRect(dst, 0, 0);
+        sdl.UpperBlitScaled(src, 0, dst, &r);
+        sdl.UpdateWindowSurface(win);
+    }
+    if (src) sdl.FreeSurface(src);
+    return 1;
+}
+
+/* ---- gamepads ---- */
+static SDL_GameController *pad;
+static uint64_t pad_next;
+static SDL_GameController *pad_get(void)
+{
+    if (pad && !sdl.GameControllerGetAttached(pad)) { uya_log("gamepad: disconnected"); sdl.GameControllerClose(pad); pad = 0; }
+    if (pad) return pad;
+    uint64_t t = plat_time_us();
+    if (t < pad_next) return 0;
+    pad_next = t + 1000000;   /* looked for every second */
+    int n = sdl.NumJoysticks();
+    for (int i = 0; i < n && !pad; i++)
+        if (sdl.IsGameController(i) && (pad = sdl.GameControllerOpen(i))) uya_log("gamepad: %s", sdl.GameControllerName(pad));
+    return pad;
+}
+static unsigned sdl_buttons(void)
+{
+    if (!sdl_on()) return 0;
+    if (!win) sdl.PumpEvents();
+    SDL_GameController *g = pad_get();
+    if (!g) return 0;
+    /* SDL_GameControllerButton: A B X Y Back Guide Start LS RS LB RB Up Down Left Right */
+    static const unsigned map[15] = { 0x4000, 0x2000, 0x8000, 0x1000, 0x0001, 0, 0x0008, 0x0002, 0x0004, 0x0400, 0x0800, 0x0010, 0x0040, 0x0080, 0x0020 };
+    unsigned b = 0;
+    for (int i = 0; i < 15; i++) if (map[i] && sdl.GameControllerGetButton(g, i)) b |= map[i];
+    if (sdl.GameControllerGetAxis(g, 4) > 8000) b |= 0x0100;   /* L2 */
+    if (sdl.GameControllerGetAxis(g, 5) > 8000) b |= 0x0200;   /* R2 */
+    return b;
+}
+static uint8_t sdl_axis(int16_t v)   /* -32768..32767 -> 0..255, 0x80 centre, with a dead zone; down is + (as the PS2) */
+{
+    int x = v / 256;
+    if (x > -24 && x < 24) return 0x7F;
+    x += 0x80;
+    return (uint8_t)(x < 0 ? 0 : x > 255 ? 255 : x);
+}
+static int sdl_sticks(uint8_t st[4])
+{
+    if (!sdl_on()) return 0;
+    SDL_GameController *g = pad_get();
+    if (!g) return 0;
+    for (int i = 0; i < 4; i++) st[i] = sdl_axis(sdl.GameControllerGetAxis(g, i));
+    return 1;
+}
+
+/* ---- sound ---- */
+static SDL_AudioDeviceID one_dev, stream_dev;
+static int stream_failed;
+static SDL_AudioDeviceID audio_open(int rate)
+{
+    SDL_AudioSpec want, have;
+    memset(&want, 0, sizeof want);
+    want.freq = rate; want.format = AUDIO_S16LSB; want.channels = 2; want.samples = 1024;
+    SDL_AudioDeviceID d = sdl.OpenAudioDevice(0, 0, &want, &have, 0);   /* (SDL converts to what the device takes) */
+    if (!d) { uya_log("sound: no output device (%s)", sdl.GetError()); return 0; }
+    sdl.PauseAudioDevice(d, 0);
+    return d;
+}
+static void sdl_audio_stop(void)
+{
+    if (!one_dev) return;
+    sdl.ClearQueuedAudio(one_dev);
+    sdl.CloseAudioDevice(one_dev);
+    one_dev = 0;
+}
+static void sdl_audio_play(const int16_t *lr, size_t frames, int rate)
+{
+    sdl_audio_stop();
+    if (!(one_dev = audio_open(rate))) return;
+    sdl.QueueAudio(one_dev, lr, (uint32_t)(frames * 4));
+}
+#define STREAM_MAX_FRAMES (32 * 1024)   /* as the Windows ring: beyond it, writes are dropped */
+static int sdl_stream_queued(void)
+{
+    if (!stream_dev) return stream_failed ? 1 << 20 : 0;
+    return (int)(sdl.GetQueuedAudioSize(stream_dev) / 4);
+}
+static void sdl_stream_write(const int16_t *lr, int frames)
+{
+    if (!stream_dev && !stream_failed && !(stream_dev = audio_open(48000))) stream_failed = 1;
+    if (!stream_dev || sdl_stream_queued() + frames > STREAM_MAX_FRAMES) return;
+    sdl.QueueAudio(stream_dev, lr, (uint32_t)frames * 4);
+}

games/rac1/host/src/platform_win32.c

@@ -180,11 +180,37 @@ static LONG CALLBACK on_exception(EXCEPTION_POINTERS *ep)
     return EXCEPTION_CONTINUE_SEARCH;
 }
 
+#include <signal.h>
+/* abort() (a library giving up) ends the process without an exception: raise one, so the
+ * crash handler reports it with the call chain */
+static void on_abort(int sig)
+{
+    (void)sig;
+    uya_log("crash: abort() was called");
+    RaiseException(0xE0000ABE, EXCEPTION_NONCONTINUABLE, 0, 0);
+}
 int plat_install_trap_handler(void)
 {
+    signal(SIGABRT, on_abort);
     return AddVectoredExceptionHandler(1, on_exception) ? 0 : -1;
 }
 
+void plat_log_system(void)
+{
+    typedef struct { ULONG size, major, minor, build, platform; WCHAR sp[128]; } osv_t;
+    HMODULE nt = GetModuleHandleA("ntdll.dll");
+    LONG (WINAPI *rtl_ver)(osv_t *) = nt ? (void *)GetProcAddress(nt, "RtlGetVersion") : 0;
+    const char *(*wine_ver)(void) = nt ? (void *)GetProcAddress(nt, "wine_get_version") : 0;
+    osv_t v; memset(&v, 0, sizeof v); v.size = sizeof v;
+    if (rtl_ver) rtl_ver(&v);
+    SYSTEM_INFO si; GetNativeSystemInfo(&si);
+    MEMORYSTATUSEX ms; memset(&ms, 0, sizeof ms); ms.dwLength = sizeof ms; GlobalMemoryStatusEx(&ms);
+    char cpu[128] = "?"; DWORD cl = sizeof cpu;
+    RegGetValueA(HKEY_LOCAL_MACHINE, "HARDWARE\\DESCRIPTION\\System\\CentralProcessor\\0", "ProcessorNameString", RRF_RT_REG_SZ, 0, cpu, &cl);
+    uya_log("system: Windows %lu.%lu build %lu%s%s, %s, %lu cores, %llu MB RAM", v.major, v.minor, v.build,
+            wine_ver ? ", Wine " : "", wine_ver ? wine_ver() : "", cpu, si.dwNumberOfProcessors, (unsigned long long)(ms.ullTotalPhys >> 20));
+}
+
 uint64_t plat_time_us(void)
 {
     static LARGE_INTEGER f;

games/rac1/host/src/rac1/core_map.c

@@ -107,7 +107,7 @@ s32 func_00118E70(s32 reg) { (void)reg; return 0x00070000; }         /* sceSifGe
  *    game sends the data with sceSifSetDma (above). Without it every stash
  *    slot was refused (size 0) and the pause menu unpacked the weapon data
  *    from a scrubbed buffer (a crash reading 0xDEE6BEEF). ---------- */
-#define STASH_BASE 0x00160000u
+#define STASH_BASE 0x00160000u                       /* an IOP address (region:keep) */
 #define STASH_SIZE 0x00094000u
 void func_0011AE20(s32 mode) { (void)mode; }                         /* sceSifInitRpc */
 s32 func_0011B2F8(void *cd, u32 server, s32 mode)                    /* sceSifBindRpc */

games/rac1/host/src/rac1/rc_runtime.c

@@ -144,7 +144,7 @@ const char *rc_stack_scan(void)
     t[0] = 0;
     for (int i = 0; sp && i < 1024 && k < 12; i += 2) {
         u32 v = sp[i];
-        if (v >= 0x00112380u && v < 0x00300000u && !(v & 3) && (v & 0xFFFF) && v != 0x00160000u && v != 0x00170000u) {
+        if (v >= 0x00112380u && v < 0x00300000u && !(v & 3) && (v & 0xFFFF) && v != 0x00160000u && v != 0x00170000u) { /* region:keep */
             n += snprintf(t + n, sizeof t - n, " %08X@%d", v, i * 4); k++;
         }
     }

games/rac1/host/src/rac1/snd989.c

@@ -410,7 +410,7 @@ static u32 bank_add(const u8 *src, u32 size)
         b->nsounds = (s16)rd16(b->sblk + 0x16);
         b->first_sound = rd32(b->sblk + 0x1C);
         b->first_grain = rd32(b->sblk + 0x20);
-        b->handle = 0x00120000u + (u32)i * 0x1000u + ((++bank_serial) & 0xFFF);
+        b->handle = 0x00120000u + (u32)i * 0x1000u + ((++bank_serial) & 0xFFF);   /* region:keep */
         if (rd32(b->sblk + 4) != 1) uya_log("989snd: SBlk version %u (only 1 is expected)", rd32(b->sblk + 4));
         uya_log("989snd: bank %08X loaded: %d sounds, %u KB of samples", b->handle, b->nsounds, b->smp_size / 1024);
         if (snd_log > 0 && getenv("UYA_DUMP")) {               /* for checking: the bank as loaded */

games/rac1/host/src/runtime.c

@@ -109,6 +109,11 @@ void uya_log_open(void)
 {
     char p[1024];
     snprintf(p, sizeof p, "%s/rac1_log.txt", plat_exe_dir());
+    {   /* the previous run's log is kept as rac1_log.prev.txt (a retry after a crash keeps the crash) */
+        char q[1100];
+        snprintf(q, sizeof q, "%s/rac1_log.prev.txt", plat_exe_dir());
+        remove(q); rename(p, q);
+    }
     logfile = fopen(p, "w");
     if (logfile) fprintf(stderr, "[uya] log file: %s\n", p);
 }

games/rac1/host/src/sdk/memcard.c

@@ -96,11 +96,15 @@ static int glob_match(const char *p, const char *s)
 /* Card path ("/DIR/file" or "DIR/file") to a host path. Wildcards in the
  * directory part pick the first directory that matches. Returns 0 if a
  * wildcard directory had no match. */
+static char mc_cwd[2][512] = {"/", "/"};   /* sceMcChdir's current directory (the NTSC-U game sets it) */
 static int host_path(int port, const char *name, char *out, size_t n, char *last, size_t nlast)
 {
     char tmp[512], cur[1024];
     snprintf(cur, sizeof cur, "%s", slot_dir(port));
-    snprintf(tmp, sizeof tmp, "%s", name ? name : "");
+    if (name && name[0] != '/' && strcmp(mc_cwd[port & 1], "/"))
+        snprintf(tmp, sizeof tmp, "%s/%s", mc_cwd[port & 1], name);   /* relative to the current directory */
+    else
+        snprintf(tmp, sizeof tmp, "%s", name ? name : "");
     char *parts[16]; int np = 0;
     for (char *t = strtok(tmp, "/"); t && np < 16; t = strtok(0, "/")) parts[np++] = t;
     for (int i = 0; i < np - 1; i++) {
@@ -177,6 +181,32 @@ s32 uyafn_12E400(s32 port, s32 slot, s32 *type, s32 *free_kb, s32 *format)
     return 0;
 }
 
+/* sceMcChdir(port, slot, new, cur*) (RPC 0x0C): the card's current directory
+ * for relative names; cur, if given, gets the previous one. PAL's game never
+ * calls it; NTSC-U's sets "/" before listing (host/region/us maps it). */
+s32 rac1_mc_chdir(s32 port, s32 slot, const char *dir, char *prev)
+{
+    (void)slot;
+    if (mclog()) uya_log("memory card: chdir %s", dir ? dir : "(null)");
+    if (!card_present(port)) { finish(0xC, -10); return 0; }
+    if (prev) snprintf(prev, 1024, "%s", mc_cwd[port & 1]);
+    char want[512];
+    if (!dir || !*dir) { finish(0xC, 0); return 0; }
+    if (dir[0] == '/') snprintf(want, sizeof want, "%s", dir);
+    else if (!strcmp(dir, "..")) {
+        snprintf(want, sizeof want, "%s", mc_cwd[port & 1]);
+        char *sl = strrchr(want, '/'); if (sl && sl != want) *sl = 0; else snprintf(want, sizeof want, "/");
+    } else snprintf(want, sizeof want, "%s%s%s", mc_cwd[port & 1], strcmp(mc_cwd[port & 1], "/") ? "/" : "", dir);
+    size_t l = strlen(want);
+    while (l > 1 && want[l - 1] == '/') want[--l] = 0;
+    char hp[1024];
+    snprintf(hp, sizeof hp, "%s%s", slot_dir(port), want);
+    if (!is_dir(hp)) { finish(0xC, -4); return 0; }   /* no such directory */
+    snprintf(mc_cwd[port & 1], sizeof mc_cwd[0], "%s", want);
+    finish(0xC, 0);
+    return 0;
+}
+
 /* sceMcGetDir table entry (sceMcTblGetDir), 0x40 bytes */
 typedef struct {
     u8 create[8], modify[8];

games/rac1/tools/gen_host.py

@@ -48,9 +48,15 @@ while _a:
 LV = "%02d" % LEVEL if LEVEL is not None else None
 SFX = "_l%s" % LV if LEVEL is not None else ""
 HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+# the region (build.py --region): its host sources (tools/region.py writes them) and constants
+REGION = os.environ.get("UYA_REGION", "pal")
+HOSTDIR = os.environ.get("UYA_HOST") or os.path.join(HERE, "host")
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import region as _region
+RC = _region.consts(REGION)
 PLAT = os.environ.get("RAC_PLATFORM", os.environ.get("UYA_PLATFORM", "linux"))  # linux | windows (COFF i386 adds a leading _)
 os.makedirs(OUT, exist_ok=True)
-overrides = {os.path.basename(p)[:-2] for p in glob.glob(os.path.join(HERE, "host", "overrides", "func_*.c"))}
+overrides = {os.path.basename(p)[:-2] for p in glob.glob(os.path.join(HOSTDIR, "overrides", "func_*.c"))}
 ASM_MACRO = re.compile(r"^\s*(INCLUDE_ASM|ASM_FUNC|LINKER_REMNANT|INCLUDE_RODATA|TEXT_PADDING)\s*\(([^()]*)\)\s*;?\s*(?:/\*.*?\*/|//.*)?\s*$", re.M)
 
 
@@ -82,7 +88,7 @@ def strip_comments_keep_len(s):
 
 host_defined = set()
 STATIC = os.environ.get("UYA_STATIC", "1") != "0"   # RAC1_STATICRECOMP: translate the whole game from its bytes
-for p in glob.glob(os.path.join(HERE, "host", "src", "**", "*.c"), recursive=True):
+for p in glob.glob(os.path.join(HOSTDIR, "src", "**", "*.c"), recursive=True):
     host_defined |= set(re.findall(r"^[\w \t\*]*\b(func_(?:L\d\d_)?[0-9A-Fa-f]{6,8})\s*\([^;{]*\)\s*\{", strip_comments_keep_len(open(p).read()), re.M))
 
 
@@ -171,8 +177,8 @@ def rewrite_types(s):
 DECL = re.compile(r"^[ \t]*(?:extern[ \t]+)?(?P<t>[A-Za-z_][\w \t\*]*?)[ \t\*]*\b(?P<n>(?:func|D)_(?:L\d\d_)?[0-9A-Fa-f]{6,8}\w*)[ \t]*(?P<s>\((?:[^;{}]*)\)|\[[^;{}]*\])?[ \t]*;", re.M)
 
 
-MAIN_TEXT = (0x001E9080, 0x0023E740)   # the executable's game program (.text of the main segment)
+MAIN_TEXT = RC["main_text"]   # the executable's game program (.text of the main segment); PAL 0x001E9080-0x0023E740
-RESIDENT = 0x0015F000                   # below: core segment and core_lit, the same in every program
+RESIDENT = RC["resident"]     # below: core segment and core_lit, the same in every program; PAL 0x0015F000
 PLACES = {}                             # level mode: catalogue name -> its addresses in this level
 CAT_KIND = {}
 CAT_SIZE = {}
@@ -333,7 +339,7 @@ if LEVEL is None: os.makedirs(INC, exist_ok=True)
 for h in (glob.glob(os.path.join(REPO, "include", "*.h")) if LEVEL is None else []):
     bn = os.path.basename(h)
     if bn == "common.h":
-        t = open(os.path.join(HERE, "host", "include", "rac1_common.h")).read()
+        t = open(os.path.join(HOSTDIR, "include", "rac1_common.h")).read()
     elif bn == "include_asm.h":
         t = "/* host: assembly inclusion has no meaning on the PC */\n"
     else:
@@ -344,7 +350,7 @@ for h in (glob.glob(os.path.join(REPO, "include", "*.h")) if LEVEL is None else
 
 stubbed, defined, asm_entries, report = [], set(), [], {}
 gen_texts = {}
-drafts = {os.path.basename(p)[:-2]: p for p in glob.glob(os.path.join(HERE, "host", "drafts", "func_*.c"))}
+drafts = {os.path.basename(p)[:-2]: p for p in glob.glob(os.path.join(HOSTDIR, "drafts", "func_*.c"))}
 used_drafts = set()
 files = sorted(os.path.relpath(p, REPO) for p in glob.glob(os.path.join(REPO, "src", "game", "*.c")))
 files += sorted(os.path.relpath(p, REPO) for p in glob.glob(os.path.join(REPO, "src", "game", "movie", "*.c")))
@@ -359,6 +365,9 @@ dropped = []      # level mode: functions of other levels' programs
 # the EE ABI but not for x86 (each entry: file, old text, new text)
 sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
 from source_patches import PATCHES
+if REGION != "pal":   # (written against the PAL sources: in the region's names, as tools/region.py rewrote them)
+    _rw = _region.Rewriter(_region.RegionMap(_region.REGIONS[REGION]["map"]))
+    PATCHES = [(pf, _rw.text(old, "patch:" + pf), _rw.text(new, "patch:" + pf)) for pf, old, new in PATCHES]
 patches_applied = 0
 # names.h: readable names are macros for the address names; expand them in
 # the C (outside string literals) so every later pass sees func_/D_ names.
@@ -453,7 +462,8 @@ for rel in files:
     # UYA_FORCE_ASM="" compiles all the C; or a list: file substrings, re:regex,
     # !file (keep that file's C), e.g. "help_0020CDF0,help_00232560" (the old default).
     _fa = os.environ.get("UYA_FORCE_ASM", "re:.")
-    if LEVEL is not None and _fa:
+    # (another region: the executable's C too, so all of it comes from the region's own bytes)
+    if (LEVEL is not None or REGION != "pal") and _fa:
         _ex = [t[1:] for t in _fa.split(",") if t.startswith("!")]
         if any(x and x in rel for x in _ex): _fa = ""            # "!file-substring": that file keeps its C
         for tok in _fa.split(","):
@@ -483,7 +493,9 @@ for rel in files:
             seg = "/* host: top-level asm removed */"
         else:
             name, bpos = extra
-            if LEVEL is not None and base_func(name) == name and (name.startswith("func_L") or is_main_func(name)) and name not in PLACES:
+            # (another region: func_palonly_X, a PAL function with no counterpart found in this region,
+            # is not in any of its programs either; its C would be defined once per level)
+            if LEVEL is not None and (name.startswith("func_palonly_") or base_func(name) == name and (name.startswith("func_L") or is_main_func(name)) and name not in PLACES):
                 dropped.append(name)
                 out.append("/* host: %s is not in level %s's program */" % (name, LV)); pos = e
                 continue
@@ -602,7 +614,7 @@ for fn, t in gen_texts.items():
         rets[q.group(1)] = ret_type(t[q.start():q.start(1) + len(q.group(1))], q.group(1))
 # host functions (host/src, host/overrides) too: a decomp call that leaves out
 # arguments the EE would still pass in registers gets them filled in
-for hp in glob.glob(os.path.join(HERE, "host", "src", "**", "*.c"), recursive=True) + glob.glob(os.path.join(HERE, "host", "overrides", "*.c")):
+for hp in glob.glob(os.path.join(HOSTDIR, "src", "**", "*.c"), recursive=True) + glob.glob(os.path.join(HOSTDIR, "overrides", "*.c")):
     ht = open(hp, errors="replace").read()
     hm = strip_comments_keep_len(ht)
     for q in HDR.finditer(hm):
@@ -808,7 +820,7 @@ for n, p in defs.items():
     a = ps2_addr(n)
     if a is not None and re.fullmatch(r"func_(?:L\d\d_)?[0-9A-Fa-f]{8}", n) and not any(x[0] == "..." for x in p):
         fdefs[a] = p
-for op_ in glob.glob(os.path.join(HERE, "host", "overrides", "func_*.c")):
+for op_ in glob.glob(os.path.join(HOSTDIR, "overrides", "func_*.c")):
     ot = open(op_, errors="replace").read()
     for q in HDR.finditer(strip_comments_keep_len(ot)):
         n = q.group(1); a = ps2_addr(n)
@@ -1224,7 +1236,7 @@ for fn_, t_ in gen_texts.items():
 # value in an int (`int mode = GetEffectTex(1);` then passed on as the texture),
 # gets 64 bits: on the EE the register keeps all of them whatever the C says.
 _r64 = sorted(n for n, s in sigs.items() if s.get("src") == "decl" and s["ret"] == "l" and re.fullmatch(r"func_(?:L\d\d_)?[0-9A-Fa-f]{8}", n)
-              and (s.get("addr") or 0) >= 0x0015F000)     # game code (the core's library calls are the host's)
+              and (s.get("addr") or 0) >= RESIDENT)     # game code (the core's library calls are the host's)
 if _r64:
     _names = "|".join(re.escape(n) for n in _r64)
     _INT = r"(?:int|s32|u32|unsigned int|unsigned|signed int|signed)"

games/rac1/tools/recomp.py

@@ -43,6 +43,11 @@ rabbitizer.config.pseudos_enablePseudos = False
 
 GEN, REPO = sys.argv[1], sys.argv[2]
 HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+REGION = os.environ.get("UYA_REGION", "pal")
+HOSTDIR = os.environ.get("UYA_HOST") or os.path.join(HERE, "host")
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import region as _region
+RC = _region.consts(REGION)
 LEVEL = DUMP = EXEGEN = None
 _a = sys.argv[3:]
 while _a:
@@ -53,14 +58,15 @@ while _a:
     else: sys.exit("recomp.py: unknown option " + _o)
 LV = "%02d" % LEVEL if LEVEL is not None else None
 P = "rc_" if LEVEL is None else "rc%s_" % LV        # prefix of the generated symbols
-RESIDENT = 0x0015F000
+RESIDENT = RC["resident"]   # PAL 0x0015F000
 
 # ---- inputs -----------------------------------------------------------------
 # testing: UYA_RC_TRACE="func_A,func_B" (build time) logs every call of those translated functions
 TRACE_FUNCS = set(x for x in os.environ.get("UYA_RC_TRACE", "").split(",") if x)
-EXE = os.environ.get("UYA_EXE") or os.path.join(REPO, "baserom", "SCES_509.16")   # build.py: the copy read from the disc image
+EXE = os.environ.get("UYA_EXE") or os.path.join(REPO, "baserom", RC["exe"])   # build.py: the copy read from the disc image
 img = open(EXE, "rb").read()
-TEXT_LO, TEXT_HI = 0x00100080, 0x00100080 + 0x13E6B0
+TEXT_LO = 0x00100080   # the executable's one loadable segment (PAL size 0x13E6B0)
+TEXT_HI = TEXT_LO + struct.unpack_from("<I", img, struct.unpack_from("<I", img, 28)[0] + 16)[0]
 LTEXT_LO = LTEXT_HI = 0
 if LEVEL is not None:
     _ld = os.path.join(DUMP, "level_%s" % LV)
@@ -86,16 +92,15 @@ for u in report["units"]:
             fsize[f["name"]] = (int(md["virtual_address"]), int(f["size"]))
 sigs = json.load(open(os.path.join(GEN, "sigs.json")))
 entries = json.load(open(os.path.join(GEN, "asm_entries.json")))
-overrides = {os.path.basename(p)[:-2] for p in os.listdir(os.path.join(HERE, "host", "overrides")) if p.endswith(".c")} if os.path.isdir(os.path.join(HERE, "host", "overrides")) else set()
+overrides = {os.path.basename(p)[:-2] for p in os.listdir(os.path.join(HOSTDIR, "overrides")) if p.endswith(".c")} if os.path.isdir(os.path.join(HOSTDIR, "overrides")) else set()
 
 todo = [n for n in entries if n in fsize and n not in overrides and re.fullmatch(r"func_[0-9A-F]{8}", n)
         and not (n in sigs and sigs[n].get("src") == "def")
         and TEXT_LO <= fsize[n][0] < TEXT_HI]
 # Sony libraries that are pure computation are translated too (libvu0: the
 # sceVu0 matrix and vector routines, core_text 0x1252A0-0x125630)
-EXTRA_RANGES = [(0x001252A0, 0x00125630),
+EXTRA_RANGES = RC["extra_ranges"]   # PAL 0x001252A0-0x00125630, and libgcc 0x0011DF10-0x001207B8
-                # libgcc (64-bit integer and soft-float helpers; level code calls some the executable's C does not)
+                # (libgcc: 64-bit integer and soft-float helpers; level code calls some the executable's C does not)
-                (0x0011DF10, 0x001207B8)]
 todo += [n for n, (a, sz) in fsize.items() if any(lo <= a < hi for lo, hi in EXTRA_RANGES)
          and re.fullmatch(r"func_[0-9A-F]{8}", n) and not (n in sigs and sigs[n].get("src") == "def") and n not in overrides]
 base_of = {}
@@ -259,7 +264,7 @@ else:
     _is_start_x = {a for a, _ in _starts_x}
     _abs = os.path.join(GEN, "abs_syms.s")
     _cand = {int(x, 16) for x in re.findall(r"\.set\s+_?D_([0-9A-F]{8})\w*,", open(_abs).read())} if os.path.exists(_abs) else set()
-    _cand = {v for v in _cand if 0x001E9080 <= v < 0x0023E740}
+    _cand = {v for v in _cand if RC["main_text"][0] <= v < RC["main_text"][1]}
     # and functions the C (the executable's and the levels') calls by an address
     # inside an assembly-only function: func_001252C0 is 0x20 into libvu0's
     # func_001252A0 (the levels' hero code calls it)
@@ -277,7 +282,7 @@ else:
             _w = struct.unpack_from("<I", _d, _i)[0]
             if _w >> 26 == 3:
                 _t = (_w & 0x3FFFFFF) << 2
-                if TEXT_LO <= _t < 0x0015F000: _cand.add(_t)
+                if TEXT_LO <= _t < RESIDENT: _cand.add(_t)
     for _v in sorted(_cand):
         if not (TEXT_LO <= _v < TEXT_HI) or _v % 4 or _v in _is_start_x: continue
         _own = [(a, e) for a, e in _starts_x if a < _v < e]

games/rac1/tools/region.py

@@ -0,0 +1,298 @@
+#!/usr/bin/env python3
+"""region.py: build another region's version of the game from the PAL sources.
+
+  python tools/region.py us DECOMP_REPO OUT_DIR
+
+The decomp, the host sources and every tool here work in the PAL executable's
+addresses (SCES-50916), and they name functions and data by address
+(func_0012F4A8, D_0013A548). region/us.json (made by tools/region_map.py from both
+executables; addresses only) maps every PAL address to the US one (SCUS-97199).
+This writes, under OUT_DIR:
+
+  repo/   the decomp as the tools read it, in US addresses: its catalogues
+          (progress/report.json, config/*, config/overlays/functions.tsv), headers and
+          sources with every func_/D_ name and address literal rewritten
+  host/   the host sources (host/src, host/include, host/overrides) the same way
+  region_report.txt   every literal rewritten, and what could not be mapped
+
+A PAL function the US build does not have is renamed func_palonly_X, so nothing is
+ever placed at its PAL address; a US function the PAL build does not have is added to
+the catalogue (and translated from the bytes like the rest). The build then runs the
+usual tools on these trees (build.py --region us)."""
+import bisect, json, os, re, shutil, sys
+
+HERE = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
+REGIONS = {"us": {"map": os.path.join(HERE, "region", "us.json"), "exe": "SCUS_971.99", "pal_exe": "SCES_509.16",
+                  "video": "ntsc"}}
+RESIDENT_PAL = 0x0015F000
+CORE_TEXT = (0x00112380, 0x0012F580)   # PAL core.text: the Sony libraries
+
+
+class RegionMap:
+    def __init__(self, path):
+        m = json.load(open(path))
+        self.m = m
+        self.exe = {int(k, 16): v for k, v in m["exe_funcs"].items()}
+        self.exe_starts = sorted(self.exe)
+        self.data = sorted(m["data"])
+        self.lv = {lv: {int(k, 16): v for k, v in d["funcs"].items()} for lv, d in m["levels"].items()}
+        self.lv_starts = {lv: sorted(d) for lv, d in self.lv.items()}
+        self.lv_data = {lv: sorted(r) for lv, r in m["level_data"].items()}
+        self.sections = m["sections"]
+        self.us_only = m["exe_us_only"]
+        self.lv_us_only = {lv: d["us_only"] for lv, d in m["levels"].items()}
+
+    @staticmethod
+    def _func(table, starts, a, interior=True):
+        if a in table: return table[a][0]
+        if not interior: return None
+        k = bisect.bisect_right(starts, a) - 1
+        if k >= 0:
+            s = starts[k]; ua, sz, method = table[s]
+            if s < a < s + sz and method == "fingerprint": return ua + (a - s)   # same instructions: same offsets
+        return None
+
+    @staticmethod
+    def _runs(runs, a):
+        for r in runs:
+            if r[0] <= a < r[1]: return a + r[2]
+        return None
+
+    def section_edge(self, a):
+        """a PAL section's first address or its end: the US section's"""
+        if a == 0x0023E740:                      # the end of PAL's game program as the tools give it (consts)
+            t = self.sections[".text"]; return (t[2] + t[3] + 15) & ~15
+        for n, (pa, ps, ua, us_) in self.sections.items():
+            if not pa: continue
+            if a == pa: return ua
+            if a == pa + ps: return ua + us_
+            if a == (pa + ps + 15) & ~15: return (ua + us_ + 15) & ~15     # (an end rounded up to 16)
+        return None
+
+    def exe_addr(self, a, interior=True):
+        """the US address of a PAL executable address, or None"""
+        if 0x00100000 <= a < 0x00112380: return a                # .reginfo, .vutext: the same in both
+        f = self._func(self.exe, self.exe_starts, a, interior)
+        if f is not None: return f
+        return self._runs(self.data, a)
+
+    def level_addr(self, lv, a, interior=True):
+        if a < RESIDENT_PAL: return self.exe_addr(a, interior)  # the core: shared by every program
+        f = self._func(self.lv.get(lv, {}), self.lv_starts.get(lv, []), a, interior)
+        if f is not None: return f
+        return self._runs(self.lv_data.get(lv, []), a)
+
+    def exe_func(self, a): return self.exe[a][0] if a in self.exe else None
+    def level_func(self, lv, a): return self.lv[lv][a][0] if lv in self.lv and a in self.lv[lv] else None
+
+
+def consts(region):
+    """what the tools need to know about the executable, for this region"""
+    pal = {"exe": "SCES_509.16", "main_text": (0x001E9080, 0x0023E740), "resident": 0x0015F000,
+           "core_text": CORE_TEXT, "extra_ranges": [(0x001252A0, 0x00125630), (0x0011DF10, 0x001207B8)]}
+    if region in (None, "", "pal"): return pal
+    rm = RegionMap(REGIONS[region]["map"])
+    t = rm.sections[".text"]; lit = rm.sections[".lit"]; ct = rm.sections["core.text"]
+    return {"exe": REGIONS[region]["exe"],
+            "main_text": (t[2], (t[2] + t[3] + 15) & ~15),
+            "resident": lit[2],
+            "core_text": (ct[2], ct[2] + ct[3]),
+            "extra_ranges": [(rm.exe_addr(lo), rm.exe_addr(hi) if rm.exe_addr(hi) else rm.exe_addr(hi - 4) + 4) for lo, hi in pal["extra_ranges"]]}
+
+
+# ---- rewriting ---------------------------------------------------------------------------------------
+NAME = re.compile(r"\b(func|D)_(?:L(\d\d)_)?([0-9A-Fa-f]{6,8})(?![0-9A-Fa-f])")
+LIT = re.compile(r"\b0x(00[1-9][0-9A-Fa-f]{5}|0[1-9][0-9A-Fa-f]{5}|[1-9][0-9A-Fa-f]{5})(?=[uUlL]*\b)")
+# a line marked region:keep holds values that only look like addresses (IOP addresses, handles, lui values)
+
+
+class Rewriter:
+    def __init__(self, rm):
+        self.rm = rm
+        self.log, self.unmapped = [], []
+        self.taken = {}
+
+    def name(self, kind, lv_name, hexpart, lv_ctx, where):
+        a = int(hexpart, 16)
+        up = hexpart.isupper() or not any(c.isalpha() for c in hexpart)
+        fmt = "%08X" if len(hexpart) == 8 else ("%06X" if up else "%06x")
+        if kind == "func":
+            if lv_name is not None:   # a level catalogue name: its address in that level
+                ua = self.rm.level_func(lv_name, a)
+                if ua is None: self.unmapped.append((where, "func_L%s_%s" % (lv_name, hexpart))); return "func_palonly_L%s_%s" % (lv_name, hexpart)
+                return "func_L%s_%s" % (lv_name, fmt % ua)
+            ua = self.rm.exe_func(a)
+            if ua is None and CORE_TEXT[0] <= a < CORE_TEXT[1]: ua = self.rm.exe_addr(a)   # a library entry inside a split unit
+            if ua is None:
+                # the executable's main-segment functions are also catalogue names of level copies
+                self.unmapped.append((where, "func_" + hexpart)); return "func_palonly_" + hexpart
+            return "func_" + (fmt % ua)
+        ctx = lv_name or lv_ctx                      # D_LNN_X: level NN's own data
+        pre = "D_L%s_" % lv_name if lv_name else "D_"
+        ua = self.rm.level_addr(ctx, a) if ctx else self.rm.exe_addr(a)
+        if ua is None: self.unmapped.append((where, pre + hexpart)); return pre.replace("D_", "D_palonly_") + hexpart
+        # one to one: a second PAL address landing on a US address already taken (two neighbours either
+        # side of a shift boundary) is not trusted; its uses then fail to build instead of reading other data
+        key = (ctx, bool(lv_name), ua)
+        if self.taken.setdefault(key, a) != a:
+            self.unmapped.append((where, "%s%s (clashes with PAL %08X)" % (pre, hexpart, self.taken[key]))); return pre.replace("D_", "D_palonly_") + hexpart
+        return pre + (fmt % ua)
+
+    def text(self, s, where, lv_ctx=None, literals=True):
+        def nm(q):
+            r = self.name(q.group(1), q.group(2), q.group(3), lv_ctx, where)
+            return r
+        s = NAME.sub(nm, s)
+        if literals:
+            def lit(q):
+                h = q.group(1); a = int(h, 16)
+                if not (0x00100000 <= a < 0x02000000): return q.group(0)
+                # only what is surely an address: a section's edge, a function's first instruction, data
+                # (a constant inside a function's range, such as a size of 0x200000, stays as it is)
+                ua = self.rm.section_edge(a)
+                if ua is None:
+                    core = CORE_TEXT[0] <= a < CORE_TEXT[1]   # the Sony libraries: entries inside split units, same code in both
+                    ua = self.rm.level_addr(lv_ctx, a, interior=core) if lv_ctx else self.rm.exe_addr(a, interior=core)
+                if ua is None: self.unmapped.append((where, "0x" + h)); return q.group(0)
+                if ua != a: self.log.append((where, "0x%s -> 0x%08X" % (h, ua)))
+                return "0x%0*X" % (len(h), ua)
+            s = "".join(l if "region:keep" in l else LIT.sub(lit, l) for l in s.splitlines(True))
+        return s
+
+
+def level_of(rel):
+    q = re.search(r"overlays[/\\]l(\d\d)_", rel)
+    return q.group(1) if q else None
+
+
+def write_repo(rm, rw, repo, out):
+    """the decomp, as the tools read it, in US addresses"""
+    for sub in ("include", "src", "nonmatching", "tools"):
+        top = os.path.join(repo, sub)
+        for root, _, files in os.walk(top):
+            for f in files:
+                p = os.path.join(root, f); rel = os.path.relpath(p, repo); d = os.path.join(out, rel)
+                os.makedirs(os.path.dirname(d), exist_ok=True)
+                if sub == "tools" or not f.endswith((".c", ".h", ".inc", ".s")):
+                    shutil.copyfile(p, d); continue
+                d_name = os.path.join(os.path.dirname(d), rw.text(f, rel, literals=False))
+                open(d_name, "w").write(rw.text(open(p, errors="replace").read(), rel, level_of(rel)))
+    os.makedirs(os.path.join(out, "config", "overlays"), exist_ok=True)
+    os.makedirs(os.path.join(out, "progress"), exist_ok=True)
+    # symbols
+    for fn in ("config/symbol_addrs.txt", "config/port_symbols.txt"):
+        p = os.path.join(repo, fn)
+        if not os.path.exists(p): continue
+        lines = []
+        for l in open(p, errors="replace"):
+            q = re.match(r"^(\s*)(\w+)(\s*=\s*)0x([0-9A-Fa-f]+)(.*)$", l.rstrip("\n"))
+            if q:
+                a = int(q.group(4), 16); ua = rm.exe_addr(a)
+                if ua is None:
+                    rw.unmapped.append((fn, q.group(2))); continue
+                l = "%s%s%s0x%08X%s" % (q.group(1), rw.text(q.group(2), fn, literals=False), q.group(3), ua, q.group(5))
+            else:
+                l = rw.text(l.rstrip("\n"), fn)
+            lines.append(l)
+        open(os.path.join(out, fn), "w").write("\n".join(lines) + "\n")
+    for fn in ("config/names.tsv", "config/symbol_names.txt", "config/file_cflags.txt", "config/linker_remnants.txt", "config/handwritten_asm.txt"):
+        p = os.path.join(repo, fn)
+        if os.path.exists(p): open(os.path.join(out, fn), "w").write(rw.text(open(p, errors="replace").read(), fn, literals=False))
+    # the executable's functions (progress/report.json): US addresses and sizes
+    rep = json.load(open(os.path.join(repo, "progress", "report.json")))
+    seen = set()
+    for u in rep.get("units", []):
+        fl = []
+        for f in u.get("functions", []):
+            md = f.get("metadata") or {}
+            if "virtual_address" not in md: fl.append(f); continue
+            a = int(md["virtual_address"])
+            if a in rm.exe:
+                ua, sz, _ = rm.exe[a]
+                f = dict(f); f["name"] = rw.text(f["name"], "report.json", literals=False); f["size"] = str(sz)
+                md = dict(md); md["virtual_address"] = str(ua); f["metadata"] = md
+                fl.append(f); seen.add(ua)
+            # (PAL-only: dropped)
+        u["functions"] = fl
+    extra = [{"name": "func_%08X" % ua, "size": str(sz), "metadata": {"virtual_address": str(ua)}}
+             for ua, sz in rm.us_only if ua not in seen and sz >= 8]
+    rep.setdefault("units", []).append({"name": "us_only", "functions": extra})
+    json.dump(rep, open(os.path.join(out, "progress", "report.json"), "w"))
+    # the US-only functions: assembly entries in a source file of their own
+    with open(os.path.join(out, "src", "game", "us_only.c"), "w") as f:
+        f.write('/* region.py: functions the US executable has and PAL does not (translated from the bytes) */\n#include "common.h"\n\n')
+        for e in extra: f.write('INCLUDE_ASM("asm/us_only", %s);\n' % e["name"])
+    # the level catalogue: each function's places in every level, in US addresses
+    rows = []
+    for l in open(os.path.join(repo, "config", "overlays", "functions.tsv")):
+        if l.startswith("#") or not l.strip(): rows.append(l.rstrip("\n")); continue
+        f = l.rstrip("\n").split("\t")
+        pl, size = [], None
+        for x in f[5].split(","):
+            if ":" not in x: continue
+            lv, a = x.split(":"); a = int(a, 16)
+            ua = rm.level_func(lv, a)
+            if ua is None: continue
+            pl.append("%s:%08X" % (lv, ua))
+            if size is None: size = rm.lv[lv][a][1]
+        if not pl: continue
+        name = rw.text(f[0], "functions.tsv", literals=False)
+        if name.startswith(("func_palonly", "D_palonly")):   # (named after a level copy the US build lacks: name it after one it has)
+            lv0, a0 = pl[0].split(":"); name = "func_L%s_%s" % (lv0, a0)
+        f[0], f[2], f[4], f[5] = name, str(size or f[2]), str(len({p.split(':')[0] for p in pl})), ",".join(pl)
+        rows.append("\t".join(f))
+    open(os.path.join(out, "config", "overlays", "functions.tsv"), "w").write("\n".join(rows) + "\n")
+    for fn in ("config/overlays/names.tsv",):
+        p = os.path.join(repo, fn)
+        if os.path.exists(p): open(os.path.join(out, fn), "w").write(rw.text(open(p, errors="replace").read(), fn, literals=False))
+    os.makedirs(os.path.join(out, "baserom"), exist_ok=True)
+
+
+# host sources that use the game's addresses (the rest: the platform, the GS, VU and movie code)
+HOST_GAME = ("src/rac1/", "src/sdk/", "overrides/", "src/main.c", "src/runtime.c", "include/")
+
+
+def write_host(rm, rw, src_host, out):
+    for root, dirs, files in os.walk(src_host):
+        dirs[:] = [d for d in dirs if d not in ("uya_only", "drafts")]
+        for f in files:
+            p = os.path.join(root, f); rel = os.path.relpath(p, src_host).replace(os.sep, "/")
+            if rel.startswith("region/"):
+                # host/region/<region>/: code only that region has, already in its addresses
+                if rel.startswith("region/%s/" % REGION):
+                    d = os.path.join(out, "src", "region", rel[len("region/%s/" % REGION):])
+                    os.makedirs(os.path.dirname(d), exist_ok=True); shutil.copyfile(p, d)
+                continue
+            d = os.path.join(out, os.path.dirname(rel), rw.text(f, "host/" + rel, literals=False))
+            os.makedirs(os.path.dirname(d), exist_ok=True)
+            if not f.endswith((".c", ".h")): shutil.copyfile(p, d); continue
+            s = open(p, errors="replace").read()
+            s = s.replace("SCES_509.16", REGIONS[REGION]["exe"])   # the executable's file name on the disc
+            open(d, "w").write(rw.text(s, "host/" + rel, literals=rel.startswith(HOST_GAME)) if rel.startswith(HOST_GAME) else s)
+
+
+REGION = None
+
+
+def main():
+    global REGION
+    region, repo, out = sys.argv[1], sys.argv[2], sys.argv[3]
+    REGION = region
+    cfg = REGIONS[region]
+    rm = RegionMap(cfg["map"])
+    rw = Rewriter(rm)
+    shutil.rmtree(os.path.join(out, "repo"), ignore_errors=True); shutil.rmtree(os.path.join(out, "host"), ignore_errors=True)
+    write_repo(rm, rw, repo, os.path.join(out, "repo"))
+    write_host(rm, rw, os.path.join(HERE, "host"), os.path.join(out, "host"))
+    with open(os.path.join(out, "region_report.txt"), "w") as f:
+        f.write("region %s: %d literals rewritten, %d names and literals not mapped\n\n== not mapped\n" % (region, len(rw.log), len(rw.unmapped)))
+        for w, x in sorted(set(rw.unmapped)): f.write("%s\t%s\n" % (w, x))
+        f.write("\n== literals rewritten\n")
+        for w, x in rw.log: f.write("%s\t%s\n" % (w, x))
+    hosts = [(w, x) for w, x in set(rw.unmapped) if w.startswith("host/")]
+    print("region %s: %d literals rewritten; not mapped: %d (host sources: %d) -> %s" % (
+        region, len(rw.log), len(set(rw.unmapped)), len(hosts), os.path.join(out, "region_report.txt")))
+
+
+if __name__ == "__main__":
+    main()

launcher/linux/project-solana

@@ -0,0 +1,583 @@
+#!/usr/bin/env python3
+"""Project Solana for Linux: builds the game from your own disc and plays it.
+
+  project-solana build DISC.iso     identify the disc, fetch the build files, build the game (once)
+  project-solana play [options]     play it  (--upscaled / --native, --fullscreen / --windowed, --vsync)
+  project-solana status             what is installed, and whether the 32-bit libraries it needs are there
+  project-solana desktop            add Project Solana to your applications menu (and so to Steam)
+  project-solana log                print the last game log (rac1_log.txt)
+  project-solana report             write one file with everything needed for a bug report
+
+Like the Windows launcher: nothing from the game is downloaded. The build files (the
+recompiler, the runtime, the decomp's source) come from the project's GitHub release, the
+game itself from your disc image. Everything lives in ~/.local/share/project-solana
+(or $XDG_DATA_HOME/project-solana); `play` remembers your last options.
+
+Needs: python3 (3.6 or newer), an x86-64 CPU with SSE4.2, and the 32-bit (i386) SDL2 and
+OpenGL libraries, which Steam installs (SteamOS and any PC with Steam have them)."""
+import argparse, configparser, hashlib, json, os, shutil, struct, subprocess, sys, time, urllib.request, zipfile
+
+RELEASE_URL = "https://github.com/OpenRAC/project-solana/releases/download/"
+GAMES = {   # serial (BOOT2 in SYSTEM.CNF): what this version is
+    "SCES_509.16": {"title": "Ratchet & Clank (PAL)", "id": "rac1", "region": "pal", "kit": "rac1", "md5": "d86429e2eca4e1da7d7f1cc656af2574", "exe": "rac1_wrapper"},
+    "SCUS_971.99": {"title": "Ratchet & Clank (NTSC-U)", "id": "rac1us", "region": "us", "kit": "rac1", "md5": "0c081d4e0353b419d95e5f95eb0ebacb", "exe": "rac1_wrapper"},
+}
+BY_ID = {g["id"]: g for g in GAMES.values()}
+OTHER = {"SCPS_150.37": "Ratchet & Clank (NTSC-J)",
+         "SCUS_972.68": "Ratchet & Clank: Going Commando (NTSC-U)", "SCES_516.07": "Ratchet & Clank 2 (PAL)",
+         "SCUS_973.53": "Ratchet & Clank: Up Your Arsenal (NTSC-U)", "SCES_524.56": "Ratchet & Clank 3 (PAL)",
+         "SCUS_974.65": "Ratchet: Deadlocked (NTSC-U)", "SCES_532.85": "Ratchet: Gladiator (PAL)"}
+
+VERSION = "1.6"
+HOME = os.path.join(os.environ.get("XDG_DATA_HOME") or os.path.expanduser("~/.local/share"), "project-solana")
+INI = os.path.join(HOME, "solana.ini")
+LAUNCHER_LOG = os.path.join(HOME, "launcher_log.txt")
+
+
+def log(text):
+    """everything the launcher says also goes to launcher_log.txt (started from Steam or a menu,
+    there is no terminal to read it in)"""
+    try:
+        os.makedirs(HOME, exist_ok=True)
+        if os.path.exists(LAUNCHER_LOG) and os.path.getsize(LAUNCHER_LOG) > (1 << 20):
+            os.replace(LAUNCHER_LOG, LAUNCHER_LOG + ".old")
+        with open(LAUNCHER_LOG, "a") as f: f.write(time.strftime("%Y-%m-%d %H:%M:%S ") + text + "\n")
+    except OSError:
+        pass
+
+
+def notify(text):
+    """a desktop notification, when there is no terminal to print to"""
+    if sys.stdout.isatty(): return
+    for cmd in (["notify-send", "-a", "Project Solana", "Project Solana", text], ["zenity", "--error", "--title=Project Solana", "--text=" + text],
+                ["kdialog", "--title", "Project Solana", "--error", text]):
+        if shutil.which(cmd[0]):
+            try: subprocess.Popen(cmd, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL); return
+            except OSError: pass
+
+
+def say(*a):
+    t = " ".join(str(x) for x in a)
+    print(t, flush=True); log(t)
+
+
+def die(msg):
+    log("error: " + msg)
+    notify(msg + "\n\n(project-solana report writes a file to send with a bug report)")
+    sys.exit("project-solana: " + msg)
+
+
+def out_dir(g):
+    """the game's build output (and its saves): build/linux, build/linux_us"""
+    return os.path.join(HOME, "games", g["kit"], "build", "linux" if g["region"] == "pal" else "linux_" + g["region"])
+
+
+def chosen(c, want=None):
+    """the game to play: --game, else the one built last (else PAL)"""
+    if want:
+        g = BY_ID.get(want) or BY_ID.get("rac1" + want) or (BY_ID["rac1"] if want == "pal" else None)
+        if not g: die("no game %s: pal or us" % want)
+        return g
+    return BY_ID.get(c.get("play", "game", fallback="rac1"), BY_ID["rac1"])
+
+
+def settings():
+    c = configparser.ConfigParser()
+    c.read(INI)
+    for s in ("disc", "play", "build"):
+        if not c.has_section(s): c.add_section(s)
+    return c
+
+
+def save(c):
+    os.makedirs(HOME, exist_ok=True)
+    with open(INI, "w") as f: c.write(f)
+
+
+# ---- the disc: ISO 9660, SYSTEM.CNF's BOOT2, and that executable's MD5 (as the Windows launcher) ----
+def iso_root_find(f, root, name):
+    o = 0
+    while o < len(root):
+        ln = root[o]
+        if not ln: o = (o // 2048 + 1) * 2048; continue
+        fl = root[o + 32]
+        fn = root[o + 33:o + 33 + fl].decode("latin-1")
+        if fn.split(";")[0].upper() == name.upper():
+            return struct.unpack_from("<I", root, o + 2)[0], struct.unpack_from("<I", root, o + 10)[0]
+        o += ln
+    return None
+
+
+def identify(path):
+    """(serial, md5 of its executable) or raises"""
+    with open(path, "rb") as f:
+        f.seek(16 * 2048); pvd = f.read(2048)
+        if pvd[1:6] != b"CD001": raise ValueError("not a disc image (no ISO 9660 volume)")
+        rlba, rsize = struct.unpack_from("<I", pvd, 158)[0], min(struct.unpack_from("<I", pvd, 166)[0], 1 << 20)
+        f.seek(rlba * 2048); root = f.read(rsize)
+        cnf = iso_root_find(f, root, "SYSTEM.CNF")
+        if not cnf: raise ValueError("not a PlayStation 2 disc (no SYSTEM.CNF)")
+        f.seek(cnf[0] * 2048); text = f.read(cnf[1]).decode("latin-1")
+        boot = [l for l in text.splitlines() if l.strip().upper().startswith("BOOT2")]
+        if not boot or "\\" not in boot[0]: raise ValueError("SYSTEM.CNF names no BOOT2 executable")
+        serial = boot[0].split("\\")[-1].split(";")[0].strip()
+        exe = iso_root_find(f, root, serial)
+        md5 = ""
+        if exe and exe[1] < (64 << 20):
+            f.seek(exe[0] * 2048); md5 = hashlib.md5(f.read(exe[1])).hexdigest()
+        return serial, md5
+
+
+# ---- downloads ----
+def fetch(src, dest=None, what="", size=0):
+    """src: a URL, or a file in a local folder ([build] release_url=/path for offline installs)"""
+    if not src.startswith(("http://", "https://")):
+        if dest: shutil.copyfile(src, dest); return None
+        return open(src, "rb").read()
+    req = urllib.request.Request(src, headers={"User-Agent": "ProjectSolana-Linux/1.0"})
+    with urllib.request.urlopen(req, timeout=60) as r:
+        total = int(r.headers.get("Content-Length") or size or 0)
+        out = open(dest + ".part", "wb") if dest else None
+        buf, got, last = [], 0, 0
+        while True:
+            b = r.read(1 << 16)
+            if not b: break
+            got += len(b)
+            if out: out.write(b)
+            else: buf.append(b)
+            if what and time.time() - last > 0.5:
+                last = time.time()
+                print("\r  %s: %.0f of %.0f MB " % (what, got / 1048576, total / 1048576) if total else "\r  %s: %.0f MB " % (what, got / 1048576), end="", flush=True)
+        if what: print("\r  %s: done%s" % (what, " " * 30), flush=True)
+        if out: out.close(); os.replace(dest + ".part", dest); return None
+        return b"".join(buf)
+
+
+def sha256(p):
+    h = hashlib.sha256()
+    with open(p, "rb") as f:
+        for b in iter(lambda: f.read(1 << 20), b""): h.update(b)
+    return h.hexdigest()
+
+
+def unzip(zp, dest, what):
+    with zipfile.ZipFile(zp) as z:
+        n = len(z.infolist())
+        for i, info in enumerate(z.infolist()):
+            z.extract(info, dest)
+            mode = (info.external_attr >> 16) & 0o777
+            if mode and not info.is_dir(): os.chmod(os.path.join(dest, info.filename), mode)
+            if i % 200 == 0: print("\r  unpacking %s: %d of %d files" % (what, i + 1, n), end="", flush=True)
+        print("\r  unpacking %s: done%s" % (what, " " * 20), flush=True)
+
+
+def rmtree(p):
+    if os.path.isdir(p): shutil.rmtree(p, ignore_errors=True)
+    elif os.path.exists(p): os.remove(p)
+
+
+# ---- the 32-bit libraries the game loads ----
+# Distributions keep 32-bit libraries in different places (Debian: /usr/lib/i386-linux-gnu, Arch and
+# SteamOS: /usr/lib32, Fedora: /usr/lib, where Arch keeps its 64-bit ones), so a file is only counted
+# when it really is 32-bit x86. SteamOS has no 32-bit SDL2 of its own (and its system is read-only),
+# but Steam carries one in its runtime: that copy is used when the system has none.
+SYS_DIRS = ["/usr/lib32", "/usr/lib/i386-linux-gnu", "/lib/i386-linux-gnu", "/usr/lib/i686-linux-gnu", "/lib32", "/usr/lib", "/lib",
+            "/usr/local/lib32", "/usr/local/lib"]
+STEAM_ROOTS = ["~/.local/share/Steam", "~/.steam/steam", "~/.steam/root", "~/.var/app/com.valvesoftware.Steam/.local/share/Steam",
+               "~/.var/app/com.valvesoftware.Steam/data/Steam", "~/snap/steam/common/.local/share/Steam"]
+STEAM_SUBDIRS = ["ubuntu12_32/steam-runtime/usr/lib/i386-linux-gnu", "ubuntu12_32/steam-runtime/lib/i386-linux-gnu",
+                 "ubuntu12_32/steam-runtime/pinned_libs_32", "ubuntu12_32/steam-runtime/i386/usr/lib/i386-linux-gnu",
+                 "ubuntu12_32/steam-runtime/i386/lib/i386-linux-gnu",
+                 # the runtime Steam installs as a tool ("Steam Linux Runtime 1.0 (scout)")
+                 "steamapps/common/SteamLinuxRuntime/steam-runtime/usr/lib/i386-linux-gnu",
+                 "steamapps/common/SteamLinuxRuntime/steam-runtime/lib/i386-linux-gnu",
+                 "steamapps/common/SteamLinuxRuntime/steam-runtime/pinned_libs_32"]
+SDL = "libSDL2-2.0.so.0"
+
+
+def elf32(path):
+    """a 32-bit x86 ELF file (symlinks followed)"""
+    try:
+        with open(path, "rb") as f: h = f.read(20)
+        return h[:4] == b"\x7fELF" and h[4] == 1 and struct.unpack_from("<H", h, 18)[0] == 3
+    except OSError:
+        return False
+
+
+def ldconfig_paths(name):
+    exe = shutil.which("ldconfig") or next((p for p in ("/sbin/ldconfig", "/usr/sbin/ldconfig") if os.path.exists(p)), None)
+    if not exe: return []
+    try: out = subprocess.run([exe, "-p"], capture_output=True, text=True, timeout=10).stdout
+    except (OSError, subprocess.SubprocessError): return []
+    return [l.split("=>", 1)[1].strip() for l in out.splitlines() if l.strip().startswith(name + " ") and "=>" in l]
+
+
+def lib_candidates(name):
+    seen, out = set(), []
+    for p in [os.path.join(d, name) for d in SYS_DIRS] + ldconfig_paths(name):
+        r = os.path.realpath(p)
+        if os.path.exists(p) and r not in seen: seen.add(r); out.append(p)
+    return out
+
+
+def find32(name):
+    return next((p for p in lib_candidates(name) if elf32(p)), None)
+
+
+def steam_sdl():
+    """Steam's own 32-bit SDL2 (its runtime), if Steam is installed and has run once"""
+    for root in STEAM_ROOTS:
+        for sub in STEAM_SUBDIRS:
+            p = os.path.join(os.path.expanduser(root), sub, SDL)
+            if elf32(p): return p
+    for root in STEAM_ROOTS:   # (another runtime layout: look for it)
+        tops = [os.path.join(os.path.expanduser(root), "ubuntu12_32")]
+        common = os.path.join(os.path.expanduser(root), "steamapps", "common")
+        if os.path.isdir(common): tops += [os.path.join(common, d) for d in sorted(os.listdir(common)) if d.startswith("SteamLinuxRuntime")]
+        for top in tops:
+            for d, dirs, files in os.walk(top):
+                if SDL in files and elf32(os.path.join(d, SDL)): return os.path.join(d, SDL)
+                if d.count(os.sep) - top.count(os.sep) > 5: dirs[:] = []
+    return None
+
+
+def find_sdl():
+    """(path, from Steam) of a 32-bit SDL2, or (None, False)"""
+    p = find32(SDL)
+    if p: return p, False
+    p = steam_sdl()
+    return (p, True) if p else (None, False)
+
+
+def check_libs(quiet=False):
+    missing = []
+    if not find32("ld-linux.so.2"): missing.append("the 32-bit C library (ld-linux.so.2)")
+    if not find_sdl()[0]: missing.append(SDL)
+    if not find32("libGL.so.1"): missing.append("libGL.so.1")
+    if missing and not quiet:
+        say("\nThe game is a 32-bit program and needs these 32-bit libraries: %s" % ", ".join(missing))
+        if missing == [SDL]:
+            say("If Steam is installed, start it once: the game can use the copy of SDL2 that comes with Steam.")
+        say("To install them:")
+        say("  Debian, Ubuntu, Mint, Pop!_OS:  sudo dpkg --add-architecture i386 && sudo apt update && sudo apt install libc6:i386 libsdl2-2.0-0:i386 libgl1:i386 libgl1-mesa-dri:i386")
+        say("  Fedora, Nobara, Bazzite:        sudo dnf install glibc.i686 SDL2.i686 mesa-libGL.i686 mesa-dri-drivers.i686")
+        say("  Arch, Manjaro, EndeavourOS:      enable [multilib], then sudo pacman -S lib32-glibc lib32-sdl2 lib32-mesa")
+        say("  openSUSE:                        sudo zypper install glibc-32bit libSDL2-2_0-0-32bit Mesa-libGL1-32bit")
+        say("  (NVIDIA's own driver: also its 32-bit OpenGL library, e.g. lib32-nvidia-utils on Arch)")
+    return not missing
+
+
+def is_steamos():
+    try: return any(l.strip() in ("ID=steamos", 'ID="steamos"') for l in open("/etc/os-release"))
+    except OSError: return False
+
+
+def sdl_env(env):
+    """when the SDL2 is Steam's: a folder holding only a link to it goes first on the library path
+    (not Steam's whole runtime: its other libraries would replace the system's)"""
+    p, steam = find_sdl()
+    if not steam: return
+    d = os.path.join(HOME, "lib32")
+    os.makedirs(d, exist_ok=True)
+    link = os.path.join(d, SDL)
+    if os.path.islink(link) or os.path.exists(link): os.remove(link)
+    os.symlink(p, link)
+    env["LD_LIBRARY_PATH"] = d + (os.pathsep + env["LD_LIBRARY_PATH"] if env.get("LD_LIBRARY_PATH") else "")
+    log("using Steam's SDL2: %s" % p)
+
+
+# ---- build ----
+OPEN_FILES_NEEDED = 16384   # the link opens every object file at once (about 5,700 for all levels)
+
+
+def raise_open_files():
+    """Linux allows 1,024 open files per program by default (the soft limit); a program may raise
+    its own up to the hard limit (usually 524,288) without root. The build inherits it."""
+    try:
+        import resource
+        soft, hard = resource.getrlimit(resource.RLIMIT_NOFILE)
+        want = hard if hard == resource.RLIM_INFINITY else min(hard, max(soft, 1 << 20))
+        if soft != resource.RLIM_INFINITY and soft < want:
+            resource.setrlimit(resource.RLIMIT_NOFILE, (want, hard))
+        soft2 = resource.getrlimit(resource.RLIMIT_NOFILE)[0]
+        log("open files limit: %s (was %s, hard %s)" % (soft2, soft, hard))
+        if soft2 != resource.RLIM_INFINITY and soft2 < OPEN_FILES_NEEDED:
+            say("Warning: this system lets a program open only %d files at once; linking the game needs about %d." % (soft2, OPEN_FILES_NEEDED))
+            say("If the build stops at linking, raise the limit (for example DefaultLimitNOFILE=524288 in /etc/systemd/system.conf, then log in again).")
+    except (ImportError, ValueError, OSError) as e:
+        log("open files limit: could not raise it (%s)" % e)
+
+
+def progress_line(line):
+    p = line.split()
+    if len(p) >= 4 and p[0] == "[progress]":
+        ph, i, n = p[1], int(p[2]), max(int(p[3]), 1)
+        label = {"prepare": "preparing", "levels": "translating the levels", "compile": "compiling", "link": "linking", "done": "done"}.get(ph, ph)
+        print("\r  %s (%d of %d)%s" % (label, i + (ph == "levels"), n, " " * 10) if ph in ("levels", "compile") else "\r  %s%s" % (label, " " * 30), end="", flush=True)
+        return True
+    return False
+
+
+def build(args):
+    iso = os.path.abspath(os.path.expanduser(args.disc))
+    if not os.path.isfile(iso): die("no such file: %s" % iso)
+    say("Disc: %s" % iso)
+    try: serial, md5 = identify(iso)
+    except (OSError, ValueError) as e: die("%s: %s" % (iso, e))
+    g = GAMES.get(serial)
+    if not g: die("%s (%s) is not supported yet." % (OTHER.get(serial, "this game"), serial) if serial in OTHER else "not a game Project Solana knows (%s)" % serial)
+    if g["md5"] and md5 != g["md5"]:
+        die("this is %s, but another version of it (executable MD5 %s): the build is made for %s" % (g["title"], md5, g["md5"]))
+    say("Game: %s" % g["title"])
+    check_libs()
+    c = settings()
+    rel = (args.release_url or c.get("build", "release_url", fallback="") or RELEASE_URL).rstrip("/") + "/"
+    kit = g["kit"]
+    src = (lambda f: rel + kit + "/" + f) if rel.startswith("http") else (lambda f: os.path.join(rel, f))
+
+    say("Checking for the latest build files")
+    man = json.loads(fetch(src(kit + ".json")).decode())
+    tc_dir = os.path.join(HOME, "toolchain")
+    os.makedirs(HOME, exist_ok=True)
+    have = open(os.path.join(tc_dir, "version.txt")).read().strip() if os.path.exists(os.path.join(tc_dir, "version.txt")) else ""
+    if have != man["toolchain_version"]:
+        say("Getting the build tools (Python and the zig compiler, %d MB, kept for later builds)" % (int(man["toolchain_linux_size"]) >> 20))
+        zp = os.path.join(HOME, man["toolchain_linux_file"])
+        fetch(src(man["toolchain_linux_file"]), zp, "build tools", int(man["toolchain_linux_size"]))
+        if sha256(zp) != man["toolchain_linux_sha256"]: os.remove(zp); die("the build tools did not download correctly (checksum): try again")
+        rmtree(tc_dir); unzip(zp, tc_dir, "build tools"); os.remove(zp)
+        open(os.path.join(tc_dir, "version.txt"), "w").write(man["toolchain_version"])
+
+    gdir = os.path.join(HOME, "games", kit)
+    kv = os.path.join(gdir, "kit_version.txt")
+    old = open(kv).read().strip() if os.path.exists(kv) else ""
+    if old != man["version"] or args.clean:
+        say("Getting the build kit %s (%d MB)" % (man["version"], int(man["kit_size"]) >> 20))
+        zp = os.path.join(HOME, man["kit_file"])
+        fetch(src(man["kit_file"]), zp, "build kit", int(man["kit_size"]))
+        if sha256(zp) != man["kit_sha256"]: os.remove(zp); die("the build kit did not download correctly (checksum): try again")
+        if os.path.isdir(gdir):   # everything but the build output (and its saves); a new kit version builds from scratch
+            for e in os.listdir(gdir):
+                if e != "build": rmtree(os.path.join(gdir, e))
+            for b in ("linux", "linux_us"):
+                rmtree(os.path.join(gdir, "build", b, "gen")); rmtree(os.path.join(gdir, "build", b, "obj"))
+        unzip(zp, gdir, "build kit"); os.remove(zp)
+        open(kv, "w").write(man["version"])   # (unpacked: an interrupted build resumes with what it compiled)
+
+    levels = c.get("build", "levels", fallback="") or man.get("levels") or ",".join(str(i) for i in range(19))
+    cpus = os.cpu_count() or 2
+    try: ram_gb = os.sysconf("SC_PAGE_SIZE") * os.sysconf("SC_PHYS_PAGES") / (1 << 30)
+    except (ValueError, OSError): ram_gb = 8
+    jobs = args.jobs or max(1, min(cpus, int(ram_gb / 1.5)))
+    say("Building the game from your disc (%d jobs): about 15 minutes on a recent 6 to 8 core CPU, longer on older ones" % jobs)
+    env = dict(os.environ, PATH=os.path.join(tc_dir, "zig") + os.pathsep + os.environ.get("PATH", ""), PYTHONUTF8="1",
+               ZIG_GLOBAL_CACHE_DIR=os.path.join(gdir, "zig-cache"), ZIG_LOCAL_CACHE_DIR=os.path.join(gdir, "zig-cache"))
+    py = os.path.join(tc_dir, "python", "bin", "python3")
+    cmd = [py, "-u", "build.py", "--platform", "linux", "--region", g["region"], "--levels", levels, "--iso", iso, "--repo", "decomp", "-j", str(jobs)]
+    log = open(os.path.join(gdir, "build_log.txt"), "w")
+    raise_open_files()
+    p = subprocess.Popen(cmd, cwd=gdir, env=env, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, bufsize=1)
+    for line in p.stdout:
+        log.write(line)
+        if not progress_line(line) and ("error" in line.lower() or "failed" in line.lower()): print("\n  " + line.rstrip())
+    p.wait(); log.close(); print()
+    if p.returncode: die("the build stopped (see %s)" % os.path.join(gdir, "build_log.txt"))
+    bd = os.path.relpath(out_dir(g), gdir)
+    for j in (bd + "/gen", bd + "/gen.new", bd + "/obj", bd + "/region", "build/levels", "build/levels_us", "build/disc", "zig-cache"): rmtree(os.path.join(gdir, j))
+    c.set("disc", g["id"], iso); c.set("build", "built_" + g["id"], man["version"]); c.set("play", "game", g["id"]); save(c)
+    other = [x for x in BY_ID.values() if x is not g and c.get("build", "built_" + x["id"], fallback="")]
+    say("Built. Ready to play: project-solana play" + ("  (the other version: project-solana play --game %s)" % other[0]["region"] if other else ""))
+
+
+# ---- play ----
+def play(args):
+    c = settings()
+    g = chosen(c, args.game)
+    iso = c.get("disc", g["id"], fallback="")
+    exe = os.path.join(out_dir(g), g["exe"])
+    if not os.path.exists(exe) or not iso: die("not built yet: run  project-solana build /path/to/your/disc.iso")
+    if not os.path.exists(iso): die("the disc image has moved (%s): build again with its new path" % iso)
+    pc = c["play"]
+    for k, v in (("resolution", args.resolution), ("fullscreen", args.fullscreen), ("vsync", args.vsync)):
+        if v is not None: pc[k] = str(v)
+    pc["game"] = g["id"]
+    save(c)
+    env = dict(os.environ)
+    deck = os.environ.get("SteamDeck") == "1" or is_steamos()   # (Steam Deck, Steam Machine: full screen by default)
+    if pc.get("resolution", "upscaled") == "upscaled":
+        try:   # (as the Windows launcher: render at about the screen's height)
+            out = subprocess.run(["xrandr", "--current"], capture_output=True, text=True, timeout=5).stdout
+            h = max(int(l.split("current ")[1].split(" x ")[1].split(",")[0]) for l in out.splitlines() if "current " in l)
+        except Exception:
+            h = 800 if deck else 1080
+        env["UYA_SCALE"] = str(min(4, max(2, -(-h // 448))))
+    full = pc.get("fullscreen", "1" if deck else "0") in ("1", "True", "true")
+    if full: env["UYA_FULLSCREEN"] = "1"
+    if pc.get("vsync", "0") in ("1", "True", "true"): env["UYA_VSYNC"] = "1"
+    if not check_libs(quiet=True): check_libs(); die("install the 32-bit libraries above, then play again")
+    sdl_env(env)
+    gdir = os.path.dirname(exe)
+    out = os.path.join(gdir, "game_output.txt")   # what SDL, the graphics driver and the game print (kept: one run back)
+    if os.path.exists(out): os.replace(out, out + ".prev")
+    log("play: %s, %s (UYA_SCALE=%s, fullscreen %s, vsync %s)" % (g["title"], iso, env.get("UYA_SCALE", "1"), full, pc.get("vsync", "0")))
+    with open(out, "w") as o:
+        try:
+            p = subprocess.Popen([exe, "-", iso], cwd=gdir, env=env, stdout=o, stderr=subprocess.STDOUT)
+        except OSError as e:
+            die("the game could not start (%s): run  project-solana status" % e)
+        try: rc = p.wait()
+        except KeyboardInterrupt: p.terminate(); rc = p.wait()
+    import signal
+    if rc == 0: log("play: the game closed normally"); return
+    why = ("it crashed (%s)" % signal.Signals(-rc).name) if rc < 0 and -rc in [s.value for s in signal.Signals] else "it stopped with an error (code %d)" % rc
+    die("the game closed: %s. project-solana report writes a file to send with a bug report" % why)
+
+
+def status(args):
+    c = settings()
+    say("Data folder: %s" % HOME)
+    tv = os.path.join(HOME, "toolchain", "version.txt")
+    say("Build tools: %s" % (open(tv).read().strip() if os.path.exists(tv) else "not downloaded"))
+    for g in BY_ID.values():
+        b = c.get("build", "built_" + g["id"], fallback="")
+        say("%s: %s, disc %s" % (g["title"], ("built from kit " + b) if b else "not built", c.get("disc", g["id"], fallback="(none)")))
+    say("32-bit SDL2 and OpenGL: %s" % ("found" if check_libs(quiet=True) else "missing"))
+    if not check_libs(quiet=True): check_libs()
+
+
+def desktop(args):
+    me = os.path.abspath(sys.argv[0])
+    d = os.path.join(os.environ.get("XDG_DATA_HOME") or os.path.expanduser("~/.local/share"), "applications")
+    os.makedirs(d, exist_ok=True)
+    c = settings()
+    built = [g for g in BY_ID.values() if c.get("build", "built_" + g["id"], fallback="")] or [BY_ID["rac1"]]
+    for g in built:
+        name = "Ratchet & Clank (Project Solana)" if len(built) == 1 else "%s (Project Solana)" % g["title"]
+        p = os.path.join(d, "project-solana-%s.desktop" % g["id"])
+        with open(p, "w") as f:
+            f.write("[Desktop Entry]\nType=Application\nName=%s\nComment=Built from your own disc\n"
+                    "Exec=python3 \"%s\" play --game %s\nTerminal=false\nCategories=Game;\n" % (name, me, g["region"]))   # (python3: works without the executable bit)
+        say("Added %s" % p)
+        say("Steam: Games > Add a Non-Steam Game to My Library, and pick \"%s\"." % name)
+
+
+def show_log(args):
+    p = os.path.join(out_dir(chosen(settings(), args.game)), "rac1_log.txt")
+    if not os.path.exists(p): die("no log yet: it is written when the game runs")
+    sys.stdout.write(open(p, errors="replace").read())
+
+
+def tail(path, n):
+    try:
+        with open(path, errors="replace") as f: lines = f.readlines()
+    except OSError:
+        return "(none)\n"
+    return ("(... %d earlier lines left out)\n" % (len(lines) - n) if len(lines) > n else "") + "".join(lines[-n:])
+
+
+def report(args):
+    """one text file with what a bug report needs; nothing from the game, and no file of the user's"""
+    c = settings()
+    g = chosen(c, args.game)
+    gdir = os.path.join(HOME, "games", g["kit"])
+    bdir = out_dir(g)
+    out = []
+    w = out.append
+    w("Project Solana bug report (Linux launcher %s), %s\n" % (VERSION, time.strftime("%Y-%m-%d %H:%M:%S %z")))
+    u = os.uname()
+    pretty = "?"
+    try:
+        for l in open("/etc/os-release"):
+            if l.startswith("PRETTY_NAME="): pretty = l.split("=", 1)[1].strip().strip('"')
+    except OSError: pass
+    w("\n== system\nLinux %s %s, %s\n" % (u.release, u.machine, pretty))
+    try:
+        cpu = next((l.split(":", 1)[1].strip() for l in open("/proc/cpuinfo") if l.startswith("model name")), "?")
+        mem = next((l.split(":", 1)[1].strip() for l in open("/proc/meminfo") if l.startswith("MemTotal")), "?")
+    except OSError: cpu = mem = "?"
+    w("CPU %s, %d cores, RAM %s\n" % (cpu, os.cpu_count() or 0, mem))
+    for k in ("XDG_SESSION_TYPE", "XDG_CURRENT_DESKTOP", "DISPLAY", "WAYLAND_DISPLAY", "SteamDeck", "SteamAppId"):
+        w("%s=%s\n" % (k, os.environ.get(k, "")))
+    for cmd in (["lspci"], ["glxinfo", "-B"]):
+        if shutil.which(cmd[0]):
+            try:
+                r = subprocess.run(cmd, capture_output=True, text=True, timeout=10).stdout
+                if cmd[0] == "lspci": r = "".join(l + "\n" for l in r.splitlines() if "VGA" in l or "3D" in l or "Display" in l)
+                w("-- %s\n%s" % (" ".join(cmd), r))
+            except Exception: pass
+    w("\n== 32-bit libraries\n")
+    for n in ("ld-linux.so.2", SDL, "libGL.so.1"):
+        found = lib_candidates(n)
+        w("%s: %s\n" % (n, ", ".join("%s (%s)" % (p, "32-bit" if elf32(p) else "not 32-bit") for p in found) or "none"))
+    sp, steam = find_sdl()
+    w("SDL2 the game uses: %s\n" % ((sp + (" (Steam's runtime)" if steam else "")) if sp else "MISSING"))
+    w("\n== Project Solana\ndata folder: %s\n" % HOME)
+    tv = os.path.join(HOME, "toolchain", "version.txt")
+    w("build tools: %s\n" % (open(tv).read().strip() if os.path.exists(tv) else "not downloaded"))
+    kv = os.path.join(gdir, "kit_version.txt")
+    w("build kit: %s\n" % (open(kv).read().strip() if os.path.exists(kv) else "none"))
+    w("game: %s\n" % g["title"])
+    w("game built: %s\n" % ("yes" if os.path.exists(os.path.join(bdir, "rac1_wrapper")) else "no"))
+    iso = c.get("disc", g["id"], fallback="")
+    w("disc image: %s\n" % ((("found, %.2f GB" % (os.path.getsize(iso) / 1e9)) if os.path.exists(iso) else "MISSING (moved?)") if iso else "none"))
+    w("settings:\n")
+    for s in c.sections():
+        for k, v in c.items(s):
+            if s == "disc" or k == "release_url": continue
+            w("  [%s] %s = %s\n" % (s, k, v))
+    for title, path, n in (("launcher log", LAUNCHER_LOG, 60), ("build log (end)", os.path.join(gdir, "build_log.txt"), 80),
+                           ("game log", os.path.join(bdir, "rac1_log.txt"), 400), ("game log, the run before", os.path.join(bdir, "rac1_log.prev.txt"), 150),
+                           ("game output (SDL, graphics driver)", os.path.join(bdir, "game_output.txt"), 120),
+                           ("game output, the run before", os.path.join(bdir, "game_output.txt.prev"), 60)):
+        w("\n== %s\n%s" % (title, tail(path, n)))
+    text = "".join(out).replace(os.path.expanduser("~"), "~")   # (no user name in the report)
+    dest_dir = next((d for d in (os.path.expanduser("~/Desktop"), os.path.expanduser("~")) if os.path.isdir(d)), HOME)
+    dest = os.path.join(dest_dir, "project-solana-report-%s.txt" % time.strftime("%Y%m%d-%H%M%S"))
+    open(dest, "w").write(text)
+    say("Wrote %s: attach it to your bug report (it has your system details and the logs, nothing else)." % dest)
+
+
+def main():
+    ap = argparse.ArgumentParser(prog="project-solana", description="Build Ratchet & Clank from your own disc, then play it.")
+    ap.add_argument("--version", action="version", version="project-solana %s (Linux launcher)" % VERSION)
+    sp = ap.add_subparsers(dest="cmd")
+    b = sp.add_parser("build", help="build the game from your disc image")
+    b.add_argument("disc"); b.add_argument("-j", "--jobs", type=int, default=0)
+    b.add_argument("--clean", action="store_true", help="fetch the build kit again and build from scratch")
+    b.add_argument("--release-url", default="", help=argparse.SUPPRESS)
+    p = sp.add_parser("play", help="play (options are remembered)")
+    p.add_argument("--game", default=None, help="pal or us, when both are built (default: the last one built or played)")
+    g = p.add_mutually_exclusive_group()
+    g.add_argument("--upscaled", dest="resolution", action="store_const", const="upscaled", help="render at about your screen's resolution (default)")
+    g.add_argument("--native", dest="resolution", action="store_const", const="native", help="the PS2's own resolution")
+    f = p.add_mutually_exclusive_group()
+    f.add_argument("--fullscreen", dest="fullscreen", action="store_const", const=1, help="full screen (F11 switches; default on Steam Deck)")
+    f.add_argument("--windowed", dest="fullscreen", action="store_const", const=0)
+    v = p.add_mutually_exclusive_group()
+    v.add_argument("--vsync", dest="vsync", action="store_const", const=1)
+    v.add_argument("--no-vsync", dest="vsync", action="store_const", const=0)
+    sp.add_parser("status", help="what is installed"); sp.add_parser("desktop", help="add it to the applications menu")
+    for n, h in (("log", "print the last game log"), ("report", "write one file with everything a bug report needs")):
+        sp.add_parser(n, help=h).add_argument("--game", default=None, help="pal or us")
+    a = ap.parse_args()
+    if sys.platform != "linux": die("this is the Linux launcher: on Windows, use Project Solana.exe")
+    log("project-solana %s: %s" % (VERSION, " ".join(sys.argv[1:])))
+    try:   # downloads lose the executable bit: set it, so ./project-solana works next time
+        me = os.path.abspath(sys.argv[0]); m = os.stat(me).st_mode
+        if not m & 0o100: os.chmod(me, m | 0o111)
+    except OSError:
+        pass
+    try:
+        {"build": build, "play": play, "status": status, "desktop": desktop, "log": show_log, "report": report}.get(a.cmd, lambda _: ap.print_help())(a)
+    except SystemExit:
+        raise
+    except KeyboardInterrupt:
+        sys.exit(130)
+    except Exception as e:   # (an unexpected error: in the launcher log too, with where)
+        import traceback
+        log("unexpected error:\n" + traceback.format_exc())
+        die("unexpected error: %s (details in %s)" % (e, LAUNCHER_LOG))
+
+
+if __name__ == "__main__":
+    main()

launcher/solana.c

@@ -36,6 +36,7 @@ typedef struct {
     const wchar_t *exe;          /* its wrapper in that folder (0: not supported yet) */
     const wchar_t *kit;          /* its GitHub release tag and folder under games\ (the Build button) */
     const wchar_t *const *levels;
+    const wchar_t *rgn;          /* build.py --region (0: pal); its output is build\windows_<rgn> */
 } game_t;
 
 static const wchar_t *const rac1_levels[] = {
@@ -45,7 +46,7 @@ static const wchar_t *const rac1_levels[] = {
 
 static const game_t games[] = {
     { "SCES_509.16", L"Ratchet & Clank", L"PAL", "d86429e2eca4e1da7d7f1cc656af2574", L"games\\rac1\\build\\windows", L"rac1_wrapper.exe", L"rac1", rac1_levels },
-    { "SCUS_971.99", L"Ratchet & Clank", L"NTSC-U", 0, 0, 0, 0, 0 },
+    { "SCUS_971.99", L"Ratchet & Clank", L"NTSC-U", "0c081d4e0353b419d95e5f95eb0ebacb", L"games\\rac1\\build\\windows_us", L"rac1_wrapper.exe", L"rac1", rac1_levels, L"us" },
     { "SCPS_150.37", L"Ratchet & Clank", L"NTSC-J", 0, 0, 0, 0, 0 },
     { "SCES_516.07", L"Ratchet & Clank 2", L"PAL", 0, 0, 0, 0, 0 },
     { "SCUS_972.68", L"Ratchet & Clank: Going Commando", L"NTSC-U", 0, 0, 0, 0, 0 },

launcher/solana.ini

@@ -0,0 +1,14 @@
+[settings]
+upscaled=1
+vsync=0
+no_gs_thread=0
+no_gl_thread=0
+fps_log=0
+capture_seconds=3
+start_level=-1
+minimize=1
+[library]
+count=2
+disc0=C:\Users\Nathan\Desktop\folder folder\Roms\ps2\Ratchet & Clank (Europe) (En,Fr,De,Es,It) (v2.00).iso
+disc1=C:\Users\Nathan\Desktop\folder folder\Roms\ps2\Ratchet & Clank.iso
+selected=1

launcher/solana_build.h

@@ -363,7 +363,7 @@ static void build_line(const char *l, double f0, double f1)
     else if (!strcmp(ph, "compile")) bld_status(f, L"%ls (%d of %d files)", what, a, b);
     else bld_status(f, L"%ls", what);
 }
-static int run_build(const wchar_t *gdir, const wchar_t *iso, const wchar_t *levels, double f0, double f1)
+static int run_build(const wchar_t *gdir, const wchar_t *iso, const wchar_t *levels, const wchar_t *rgn, double f0, double f1)
 {
     wchar_t py[MAX_PATH], zig[MAX_PATH], tc[MAX_PATH], cmd[4096], logp[MAX_PATH];
     path_join(tc, base_dir, L"toolchain");
@@ -375,7 +375,7 @@ static int run_build(const wchar_t *gdir, const wchar_t *iso, const wchar_t *lev
     int j = (int)si.dwNumberOfProcessors, jm = (int)(ms.ullTotalPhys / (1536ull << 20));
     if (j > jm) j = jm;
     if (j < 1) j = 1;
-    swprintf(cmd, 4096, L"\"%ls\" -u build.py --platform windows --levels %ls --iso \"%ls\" --repo decomp -j %d", py, levels, iso, j);
+    swprintf(cmd, 4096, L"\"%ls\" -u build.py --platform windows --region %ls --levels %ls --iso \"%ls\" --repo decomp -j %d", py, rgn, levels, iso, j);
     static wchar_t path[8192]; static int have_path;   /* (the PATH this program started with) */
     if (!have_path) { DWORD k = GetEnvironmentVariableW(L"PATH", path, 8192); if (k >= 8192) path[0] = 0; have_path = 1; }
     {   /* the toolchain first on PATH; Python in UTF-8 mode (the sources are UTF-8) */
@@ -496,7 +496,8 @@ static DWORD WINAPI build_thread(LPVOID unused)
             GetPrivateProfileStringW(g->kit, L"kit_version", L"", kv, 64, ini_path);
             MultiByteToWideChar(CP_UTF8, 0, version, -1, wv, 64);
             if (kv[0] && wcscmp(kv, wv)) {   /* (none recorded: a first build, or one from before this was kept) */
-                static const wchar_t *old[] = { L"build\\windows\\gen", L"build\\windows\\gen.new", L"build\\windows\\obj", L"zig-cache" };
+                static const wchar_t *old[] = { L"build\\windows\\gen", L"build\\windows\\gen.new", L"build\\windows\\obj",
+                                                L"build\\windows_us\\gen", L"build\\windows_us\\gen.new", L"build\\windows_us\\obj", L"zig-cache" };
                 for (unsigned i = 0; i < sizeof old / sizeof old[0]; i++) { path_join(p, gdir, old[i]); rm_tree(p); }
             }
             WritePrivateProfileStringW(g->kit, L"kit_version", wv, ini_path);
@@ -519,17 +520,20 @@ static DWORD WINAPI build_thread(LPVOID unused)
         GetPrivateProfileStringW(L"build", L"levels", L"", levels, 256, ini_path);
         if (!levels[0]) MultiByteToWideChar(CP_UTF8, 0, lv[0] ? lv : "0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18", -1, levels, 256);
         bld_status(0.32, L"Starting the build");
-        if (!run_build(gdir, d->path, levels, 0.32, 0.99)) goto out;
+        if (!run_build(gdir, d->path, levels, g->rgn ? g->rgn : L"pal", 0.32, 0.99)) goto out;
     }
     /* 5. keep what the game needs: the intermediate files (about 1.3 GB) and the bytes read from the disc go */
     bld_status(0.99, L"Cleaning up");
     {
-        static const wchar_t *junk[] = { L"build\\windows\\gen", L"build\\windows\\gen.new", L"build\\windows\\obj", L"build\\levels", L"build\\disc",
+        static const wchar_t *junk[] = { L"gen", L"gen.new", L"obj", L"region", L"rac1_game.pdb", L"rac1_wrapper.pdb", L"asm_stubs.c.lib" };   /* (in the build's own folder) */
-                                         L"build\\windows\\rac1_game.pdb", L"build\\windows\\rac1_wrapper.pdb", L"build\\windows\\asm_stubs.c.lib", L"zig-cache" };
+        static const wchar_t *junk2[] = { L"build\\levels", L"build\\levels_us", L"build\\disc", L"zig-cache" };
-        for (unsigned i = 0; i < sizeof junk / sizeof junk[0]; i++) { path_join(p, gdir, junk[i]); rm_tree(p); }
+        const wchar_t *bd = g->rgn && wcscmp(g->rgn, L"pal") ? L"build\\windows_" : L"build\\windows";
+        for (unsigned i = 0; i < sizeof junk / sizeof junk[0]; i++) {
+            swprintf(p, MAX_PATH, L"%ls\\%ls%ls\\%ls", gdir, bd, g->rgn && wcscmp(g->rgn, L"pal") ? g->rgn : L"", junk[i]); rm_tree(p); }
+        for (unsigned i = 0; i < sizeof junk2 / sizeof junk2[0]; i++) { path_join(p, gdir, junk2[i]); rm_tree(p); }
     }
     {   wchar_t wv[64]; MultiByteToWideChar(CP_UTF8, 0, version, -1, wv, 64);
-        WritePrivateProfileStringW(g->kit, L"built_version", wv, ini_path); }
+        WritePrivateProfileStringW(g->kit, g->rgn ? L"built_version_us" : L"built_version", wv, ini_path); }
     bld_status(1.0, L"Built");
 out:
     free(js);

release/make_release.py

@@ -2,7 +2,7 @@
 """make_release.py: makes the files of a game's GitHub release (the tag is the game, e.g. "rac1").
 
 usage: python release/make_release.py rac1 --decomp PATH_TO_RAC1_DECOMP [--version V]
-                                      [--toolchain-zip toolchain-win64.zip]
+                                      [--toolchain-zip toolchain-win64.zip] [--toolchain-linux-zip toolchain-linux64.zip]
 
 Writes dist/rac1/:
   rac1.json             what the launcher's Build button fetches (version, files, SHA-256s, levels)
@@ -10,8 +10,9 @@ Writes dist/rac1/:
                         source as decomp/ (everything but baserom/ and .git: no game files)
   toolchain-win64.zip   Python, rabbitizer and zig for Windows x64 (release/toolchain.json); reused
                         from --toolchain-zip when given, else fetched (PyPI and GitHub) and assembled
+  toolchain-linux64.zip the same for Linux x86-64 (--toolchain-linux-zip to reuse one)
 
-Upload the three files to the release named after the game (gh release create rac1 dist/rac1/*).
+Upload the four files to the release named after the game (gh release create rac1 dist/rac1/*).
 Players never download anything from the game itself: the build reads their own disc image."""
 import argparse, datetime, glob, hashlib, io, json, os, shutil, subprocess, sys, tarfile, tempfile, zipfile
 
@@ -21,7 +22,8 @@ ap = argparse.ArgumentParser()
 ap.add_argument("game")
 ap.add_argument("--decomp", required=True, help="the game's decomp repo (only its source is packed; baserom/ is left out)")
 ap.add_argument("--version", default=datetime.date.today().strftime("%Y.%m.%d"))
-ap.add_argument("--toolchain-zip", default=None)
+ap.add_argument("--toolchain-zip", default=None, help="reuse this toolchain-win64.zip")
+ap.add_argument("--toolchain-linux-zip", default=None, help="reuse this toolchain-linux64.zip")
 ap.add_argument("--levels", default=",".join(str(i) for i in range(19)))
 a = ap.parse_args()
 
@@ -35,7 +37,7 @@ SKIP_DIRS = {"__pycache__", ".git", "build", "dev", "decomp", "zig-cache"}
 DECOMP_SKIP = {"baserom", ".git", ".github"}
 
 
-def add_tree(z, top, arc, skip):
+def add_tree(z, top, arc, skip, modes=False):
     for root, dirs, files in os.walk(top):
         dirs[:] = sorted(d for d in dirs if d not in skip)
         for f in sorted(files):
@@ -45,7 +47,7 @@ def add_tree(z, top, arc, skip):
             # newer, so a retry of the same kit reuses them (a new kit version starts clean)
             zi = zipfile.ZipInfo(os.path.join(arc, os.path.relpath(p, top)).replace(os.sep, "/"), date_time=(2020, 1, 1, 0, 0, 0))
             zi.compress_type = zipfile.ZIP_DEFLATED
-            zi.external_attr = 0o644 << 16
+            zi.external_attr = ((os.stat(p).st_mode & 0o777) if modes else 0o644) << 16   # (Linux: executables stay executable)
             with open(p, "rb") as fh: z.writestr(zi, fh.read(), compresslevel=9)
 
 
@@ -67,44 +69,70 @@ with zipfile.ZipFile(kit, "w", zipfile.ZIP_DEFLATED, compresslevel=9) as z:
         if n.startswith("decomp/baserom/") or n.endswith(("SCES_509.16", ".iso")): sys.exit("refusing to pack %s" % n)
 print("kit:", kit, os.path.getsize(kit) // 1024, "KB")
 
-# the toolchain
+# the toolchains: Python, rabbitizer and zig, for building on Windows (x64) and on Linux (x86-64)
 tcj = json.load(open(os.path.join(HERE, "toolchain.json")))
-tcz = os.path.join(out, "toolchain-win64.zip")
+
-if a.toolchain_zip:
+
-    if os.path.abspath(a.toolchain_zip) != os.path.abspath(tcz): shutil.copyfile(a.toolchain_zip, tcz)
+def make_toolchain(plat, out_zip):
-else:
     import urllib.request
     with tempfile.TemporaryDirectory() as t:
-        print("fetching", tcj["python"])
+        print("fetching", tcj["python"][plat])
-        data = urllib.request.urlopen(tcj["python"]).read()
+        data = urllib.request.urlopen(tcj["python"][plat]).read()
         tarfile.open(fileobj=io.BytesIO(data)).extractall(t)
         py = os.path.join(t, "python")
         for pat in ("*.pdb", "**/*.pdb"):
             for p in glob.glob(os.path.join(py, pat), recursive=True): os.remove(p)
-        for d in ("tcl", "Lib/test", "Lib/idlelib", "Lib/tkinter", "Lib/turtledemo", "Lib/ensurepip", "include", "libs", "Scripts"):
+        site = os.path.join(py, "Lib", "site-packages") if plat == "windows" else glob.glob(os.path.join(py, "lib", "python3.*", "site-packages"))[0]
+        stdlib = os.path.join(py, "Lib") if plat == "windows" else os.path.dirname(site)
+        for d in ("test", "idlelib", "tkinter", "turtledemo", "ensurepip", "lib2to3", "config-3.12-x86_64-linux-gnu"):
+            shutil.rmtree(os.path.join(stdlib, d), ignore_errors=True)
+        for d in ("tcl", "include", "libs", "Scripts", "share"):
             shutil.rmtree(os.path.join(py, d), ignore_errors=True)
+        for p in glob.glob(os.path.join(py, "lib", "libtcl*")) + glob.glob(os.path.join(py, "lib", "libtk*")) + glob.glob(os.path.join(py, "lib", "tcl*")) + glob.glob(os.path.join(py, "lib", "tk*")):
+            shutil.rmtree(p, ignore_errors=True) if os.path.isdir(p) else os.remove(p)
         w = os.path.join(t, "wheels")
-        subprocess.run([sys.executable, "-m", "pip", "download", "--no-deps", "--platform", "win_amd64", "--only-binary=:all:", "-d", w,
+        tags = ["--platform", "win_amd64"] if plat == "windows" else ["--platform", "manylinux_2_17_x86_64", "--platform", "manylinux2014_x86_64", "--platform", "manylinux_2_12_x86_64"]
+        subprocess.run([sys.executable, "-m", "pip", "download", "--no-deps", *tags, "--python-version", "3.12", "--only-binary=:all:", "-d", w,
                         tcj["ziglang_wheel"], tcj["rabbitizer_wheel"]], check=True)
-        for whl in glob.glob(os.path.join(w, "rabbitizer-*.whl")): zipfile.ZipFile(whl).extractall(os.path.join(py, "Lib", "site-packages"))
+        for whl in glob.glob(os.path.join(w, "rabbitizer-*.whl")): zipfile.ZipFile(whl).extractall(site)
         for whl in glob.glob(os.path.join(w, "ziglang-*.whl")): zipfile.ZipFile(whl).extractall(os.path.join(t, "zw"))
         zg = os.path.join(t, "zig")
         shutil.move(os.path.join(t, "zw", "ziglang"), zg)
         for f in ("__init__.py", "__main__.py"):
             if os.path.exists(os.path.join(zg, f)): os.remove(os.path.join(zg, f))
-        # the C library sources and headers of other systems are not needed for x86-windows-gnu
+        if plat == "linux":
+            os.chmod(os.path.join(zg, "zig"), 0o755)
+            # symlinks would be stored as second copies, and the binaries carry debug symbols (hundreds of MB)
+            for root, dirs, files in os.walk(py):
+                for f in files:
+                    q = os.path.join(root, f)
+                    if os.path.islink(q) and f.startswith("libpython"): os.remove(q)   # (libpython3.12.so: for linking against it)
+            for q in glob.glob(os.path.join(py, "lib", "libpython3*.so*")) + [os.path.join(zg, "zig")]:
+                subprocess.run(["strip", "--strip-unneeded", q], check=True)
+        # the C library sources and headers of other systems: x86-windows-gnu needs mingw, x86-linux-gnu glibc
+        keep_libc = ("mingw", "include") if plat == "windows" else ("glibc", "include", "include-linux") 
         lc = os.path.join(zg, "lib", "libc")
         for d in os.listdir(lc):
-            if d in ("glibc", "musl", "darwin", "wasi", "freebsd", "netbsd", "openbsd"): shutil.rmtree(os.path.join(lc, d))
+            if d in ("glibc", "musl", "darwin", "wasi", "freebsd", "netbsd", "openbsd", "mingw") and d not in keep_libc: shutil.rmtree(os.path.join(lc, d))
         inc = os.path.join(lc, "include")
         for d in os.listdir(inc):
-            if "windows" not in d: shutil.rmtree(os.path.join(inc, d))
+            want = "windows" in d if plat == "windows" else (d.endswith(("linux-gnu", "linux-any", "-glibc")) or d in ("generic-glibc", "any-linux-any", "x86-linux-any", "x86-linux-gnu"))
+            if not want: shutil.rmtree(os.path.join(inc, d))
         for d in ("libcxx", "libcxxabi", "libunwind", "libtsan", "docs", "build-web", "init"):
             shutil.rmtree(os.path.join(zg, "lib", d), ignore_errors=True)
-        with zipfile.ZipFile(tcz, "w", zipfile.ZIP_DEFLATED, compresslevel=9) as z:
+        with zipfile.ZipFile(out_zip, "w", zipfile.ZIP_DEFLATED, compresslevel=9) as z:
-            add_tree(z, py, "python", set())
+            add_tree(z, py, "python", set(), modes=plat == "linux")
-            add_tree(z, zg, "zig", set())
+            add_tree(z, zg, "zig", set(), modes=plat == "linux")
-print("toolchain:", tcz, os.path.getsize(tcz) // (1 << 20), "MB")
+
+
+tc = {}
+for plat, name, reuse in (("windows", "toolchain-win64.zip", a.toolchain_zip), ("linux", "toolchain-linux64.zip", a.toolchain_linux_zip)):
+    tc[plat] = os.path.join(out, name)
+    if reuse:
+        if os.path.abspath(reuse) != os.path.abspath(tc[plat]): shutil.copyfile(reuse, tc[plat])
+    else:
+        make_toolchain(plat, tc[plat])
+    print("toolchain (%s):" % plat, tc[plat], os.path.getsize(tc[plat]) // (1 << 20), "MB")
 
 # what the launcher reads (a flat object: it is read without a JSON library)
 man = {
@@ -112,7 +140,8 @@ man = {
     "version": a.version,
     "levels": a.levels,
     "kit_file": os.path.basename(kit), "kit_size": os.path.getsize(kit), "kit_sha256": sha256(kit),
-    "toolchain_file": os.path.basename(tcz), "toolchain_size": os.path.getsize(tcz), "toolchain_sha256": sha256(tcz),
+    "toolchain_file": os.path.basename(tc["windows"]), "toolchain_size": os.path.getsize(tc["windows"]), "toolchain_sha256": sha256(tc["windows"]),
+    "toolchain_linux_file": os.path.basename(tc["linux"]), "toolchain_linux_size": os.path.getsize(tc["linux"]), "toolchain_linux_sha256": sha256(tc["linux"]),
     "toolchain_version": tcj["version"],
 }
 with open(os.path.join(out, "%s.json" % game), "w") as f: json.dump(man, f, indent=1)

release/rac1/rac1.json

@@ -1,12 +0,0 @@
-{
- "game": "rac1",
- "version": "2026.10.14.1",
- "levels": "0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18",
- "kit_file": "rac1-build.zip",
- "kit_size": 5125429,
- "kit_sha256": "b6919dd6d31d1ae8fac65c2859eaa6c084e98b97af36e3a305b4253da40ba5c5",
- "toolchain_file": "toolchain-win64.zip",
- "toolchain_size": 84006129,
- "toolchain_sha256": "2d56894833ca4066faf5ac008d3fa06aa40e2bb1d1fcb8f5bd26f28a70382b63",
- "toolchain_version": "py3.12.7-zig0.16.0-rabbitizer1.16.2"
-}

release/toolchain.json

@@ -1,7 +1,10 @@
 {
-  "comment": "What toolchain-win64.zip is made of (release/make_release.py fetches these when it has no toolchain zip to reuse). All are redistributable: CPython (PSF licence), zig (MIT), rabbitizer (MIT).",
+  "comment": "What toolchain-win64.zip and toolchain-linux64.zip are made of (release/make_release.py fetches these when it has no toolchain zip to reuse). All are redistributable: CPython (PSF licence), zig (MIT), rabbitizer (MIT).",
   "version": "py3.12.7-zig0.16.0-rabbitizer1.16.2",
-  "python": "https://github.com/astral-sh/python-build-standalone/releases/download/20241016/cpython-3.12.7+20241016-x86_64-pc-windows-msvc-install_only.tar.gz",
+  "python": {
+    "windows": "https://github.com/astral-sh/python-build-standalone/releases/download/20241016/cpython-3.12.7+20241016-x86_64-pc-windows-msvc-install_only.tar.gz",
+    "linux": "https://github.com/astral-sh/python-build-standalone/releases/download/20241016/cpython-3.12.7+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz"
+  },
   "ziglang_wheel": "ziglang==0.16.0",
   "rabbitizer_wheel": "rabbitizer==1.16.2"
 }