diff --git a/README.md b/README.md index 70b531c..182f3d6 100644 --- a/README.md +++ b/README.md @@ -2,17 +2,29 @@ Native PC versions of the PlayStation 2 Ratchet & Clank games, built on your own PC from your own disc. -Project Solana is a launcher and a set of static recompilations. Each game's code is translated ahead of time from the game's executable into C, with the decompiled source from OpenRAC where it exists, and compiled into a native Windows program that reads everything else (models, textures, sound, movies) from your disc image at run time. +Project Solana is a launcher and a set of static recompilations. Each game's code is translated ahead of time from the game's executable into C, with the decompiled source from OpenRAC where it exists, and compiled into a native Windows or Linux program that reads everything else (models, textures, sound, movies) from your disc image at run time. **Nothing from the games is distributed here.** The launcher downloads only tools and source code; the game is built on your machine from a disc image you provide. -## Playing +## Playing on Windows 1. Download `Project Solana.exe` from the [latest launcher release](../../releases) and put it in a folder of its own (it keeps its files next to itself). 2. Run it and add your disc image: **Add disc image**, **Scan a folder**, or drag an `.iso` onto the window. 3. Select the game and press **Build**. The first build downloads about 95 MB (Python and the zig compiler, then the game's build kit) and builds the game from your disc: about 15 minutes on a recent 6 to 8 core CPU, an hour or more on an older 4 core one. It only happens once per game version. 4. Press **Play**. +## Playing on Linux and Steam Deck + +The game runs natively on Linux (no Wine or Proton). The Linux launcher is a single script, `launcher/linux/project-solana` (download it from the [latest launcher release](../../releases), then `chmod +x project-solana`): + +``` +./project-solana build ~/Games/ratchet.iso # once: about 95 MB of downloads, then the same build as on Windows +./project-solana play # --upscaled / --native, --fullscreen / --windowed, --vsync (remembered) +./project-solana desktop # adds it to the applications menu; from there, Steam > Add a Non-Steam Game +``` + +It needs `python3` and the **32-bit** SDL2 and OpenGL libraries. Steam installs them, so SteamOS and any PC with Steam already has them; otherwise `./project-solana status` says what to install for your distribution. Everything is kept in `~/.local/share/project-solana`. On a Steam Deck, run the build once in Desktop Mode (Konsole), then add the game to Steam with `desktop`; it starts full screen there, with the Deck's controls as the controller. + ### Supported games | Game | Region | Disc executable | Status | @@ -23,10 +35,10 @@ Project Solana is a launcher and a set of static recompilations. Each game's cod ### Requirements -- Windows 10 or 11, 64-bit +- Windows 10 or 11, 64-bit, or Linux x86-64 (glibc 2.17 or newer, X11 or Wayland) with the 32-bit SDL2 and OpenGL libraries - A CPU with SSE4.2 (Intel since 2008, AMD since 2011) and 8 GB of RAM - A GPU with OpenGL 3.3 (tested: RTX 3060, Upscaled at 3x) -- A controller: Xbox, DualShock 4 or DualSense, over USB or Bluetooth +- A controller: Xbox, DualShock 4 or DualSense, over USB or Bluetooth (on Linux, anything SDL2 knows, including the Steam Deck's controls) - About 2.5 GB free while building, 600 MB after (the game plus the build tools, kept for rebuilds) ### Settings @@ -43,7 +55,8 @@ Resolution (Native, or Upscaled to a whole multiple of the PS2's 448 lines that | Path | What | |---|---| | `launcher/` | `Project Solana.exe`: Win32 + GDI, no dependencies. `python launcher/build_launcher.py` builds it (needs zig: `pip install ziglang`) | -| `games/rac1/` | Ratchet & Clank's build kit: `build.py`, the recompilers (`tools/`), the host engine (`host/`: GS in OpenGL and software, DMA, VU, SPU2 and 989snd, disc, pad, memory card, movies), the wrapper (`launcher/`) | +| `launcher/linux/project-solana` | the Linux launcher (Python 3, no dependencies) | +| `games/rac1/` | Ratchet & Clank's build kit: `build.py`, the recompilers (`tools/`), the host engine (`host/`: GS in OpenGL and software, DMA, VU, SPU2 and 989snd, disc, pad, memory card, movies; Win32 and Linux/SDL2 platform layers), the wrapper (`launcher/`) | | `games/rac1/dev/` | developer tools (GS capture replay, VU reference check); not part of the kit | | `release/make_release.py` | makes a game's release files | | `THIRD_PARTY_NOTICES.md` | pl_mpeg (MIT), jak-project's 989snd tables (ISC) | @@ -56,6 +69,7 @@ The launcher fetches from the release named after the game (`releases/download/r |---|---| | `rac1.json` | version, file names, sizes and SHA-256s, the levels to build | | `toolchain-win64.zip` | Python 3.12 (python-build-standalone), rabbitizer, zig 0.16; shared by all games, kept in `toolchain\` and fetched again only when its version changes | +| `toolchain-linux64.zip` | the same for Linux x86-64 (the Linux launcher's) | | `rac1-build.zip` | `games/rac1` plus the decomp's source as `decomp/` (no `baserom/`) | It checks both zips against their SHA-256s, unpacks them next to itself (`toolchain\`, `games\rac1\`), runs `build.py --platform windows --iso `, and then deletes the intermediate files (about 1.3 GB) and the bytes it read from the disc, keeping `games\rac1\build\windows` (the game, its saves and logs). @@ -65,13 +79,14 @@ It checks both zips against their SHA-256s, unpacks them next to itself (`toolch ## Making a release ``` -python release/make_release.py rac1 --decomp ../rac1-decomp --version 2026.10.13 -gh release create rac1 dist/rac1/rac1.json dist/rac1/rac1-build.zip dist/rac1/toolchain-win64.zip +python release/make_release.py rac1 --decomp ../rac1-decomp --version 2026.10.14.2 \ + --toolchain-zip dist/rac1/toolchain-win64.zip --toolchain-linux-zip dist/rac1/toolchain-linux64.zip # (reuse, or omit to assemble them) +gh release create rac1 dist/rac1/rac1.json dist/rac1/rac1-build.zip dist/rac1/toolchain-win64.zip dist/rac1/toolchain-linux64.zip ``` To update an existing game release, upload the new files over the old ones (`gh release upload rac1 dist/rac1/* --clobber`). Players get them with **Rebuild the game**. -The launcher itself goes in a release of its own (for example `launcher-v1`) with `Project Solana.exe`. +The launchers go in a release of their own (for example `launcher-v1`): `Project Solana.exe` and `project-solana` (Linux). ## Licence diff --git a/games/rac1/build.py b/games/rac1/build.py index 5c86ff0..c29ef62 100644 --- a/games/rac1/build.py +++ b/games/rac1/build.py @@ -28,7 +28,9 @@ ap.add_argument("--levels", default="00") a = ap.parse_args() zig = [shutil.which("zig")] if shutil.which("zig") else [sys.executable, "-m", "ziglang"] -target = {"linux": "x86-linux-musl", "windows": "x86-windows-gnu"}[a.platform] +# Linux: 32-bit glibc 2.17 or newer (any distribution since 2013), linked dynamically so it can load +# the system's SDL2 and OpenGL driver (32-bit: i386 libraries) +target = {"linux": "x86-linux-gnu.2.17", "windows": "x86-windows-gnu"}[a.platform] out = os.path.join(HERE, "build", a.platform) gen = os.path.join(out, "gen") stage = os.path.join(out, "gen.new") @@ -145,6 +147,7 @@ sync(stage, gen) common = ["-target", target, "-O" + a.O, "-g", "-march=nehalem", "-mfpmath=sse", "-malign-double", "-fno-strict-aliasing", "-fwrapv", "-fno-PIE", "-fno-stack-protector", "-I", os.path.join(gen, "include"), "-I", os.path.join(HERE, "host", "include")] +if a.platform == "linux": common += ["-D_FILE_OFFSET_BITS=64"] # (32-bit glibc: disc images are over 2 GB) # frontbin code: keep it permissive, it was written for gcc 2.95 menu_flags = ["-w", "-Wno-error=incompatible-pointer-types", "-Wno-error=int-conversion", "-Wno-error=implicit-function-declaration", "-Wno-error=implicit-int", @@ -228,7 +231,10 @@ if errs: sys.exit("%d files failed to compile" % len(errs)) if a.platform == "linux": exe = os.path.join(out, "rac1_wrapper") - link = zig + ["cc", "-target", target, "-o", exe, objs_rsp("link")] + ["-static", "-no-pie", "-Wl,--image-base=0x40000000"] + # at 0x40000000 like the Windows DLL (lower addresses are the PS2's); its symbols exported + # (-rdynamic) so the crash handler can name functions + link = zig + ["cc", "-target", target, "-o", exe, objs_rsp("link")] + ["-no-pie", "-rdynamic", "-Wl,--image-base=0x40000000", + "-ldl", "-lpthread", "-lm"] r = subprocess.run(link, capture_output=True, text=True) if r.returncode: print(r.stderr[-6000:]); sys.exit("link failed") diff --git a/games/rac1/host/src/gs/gs.c b/games/rac1/host/src/gs/gs.c index c5e0b92..79403f2 100644 --- a/games/rac1/host/src/gs/gs.c +++ b/games/rac1/host/src/gs/gs.c @@ -2095,7 +2095,7 @@ static void uya_gs_gif_(const u32 *d, u32 qwc) /* ---- init and display ---- */ /* UYA_RENDERER=gl|soft (or renderer= in rac1_wrapper.cfg): the GPU draws - * (default where there is GL: Windows) or the software rasterizer; GL falls + * (default) or the software rasterizer; GL falls * back to software if it cannot start */ const char *uya_renderer_cfg; void uya_gs_init(void) @@ -2107,11 +2107,7 @@ void uya_gs_init(void) { const char *r = getenv("UYA_RENDERER"); if (!r) r = uya_renderer_cfg; -#ifdef _WIN32 - if (!r) r = "gl"; -#else - if (!r) r = "soft"; -#endif + if (!r) r = "gl"; /* (with no window, as headless on Linux, GL does not start: software) */ if (!strcmp(r, "gl")) { if (plat_gl_init() && gsgl_init(vram)) { plat_gl_active = 1; uya_log("GS: drawing with OpenGL"); } else uya_log("GS: OpenGL not available, drawing in software"); diff --git a/games/rac1/host/src/platform_linux.c b/games/rac1/host/src/platform_linux.c index df8d80e..b2120fc 100644 --- a/games/rac1/host/src/platform_linux.c +++ b/games/rac1/host/src/platform_linux.c @@ -1,5 +1,6 @@ /* Linux (i386) platform layer: fixed mappings and the int3 trap for calls - * into PS2 code the host does not implement. */ + * into PS2 code the host does not implement; the window, OpenGL, gamepads and + * sound through SDL2 (platform_linux_sdl.h), or headless for testing. */ #ifdef __linux__ #define _GNU_SOURCE #include "uya_runtime.h" @@ -81,10 +82,41 @@ static void on_trap(int sig, siginfo_t *si, void *ctxv) g[REG_ESP] = esp + 4; } +#include +static void *dladdr_base(void) /* the game executable's own load address */ +{ + static void *b; Dl_info di; + if (!b && dladdr((void *)&dladdr_base, &di)) b = di.dli_fbase; + return b; +} +/* "name+0xoff" for an address in the game (its symbols are exported: -rdynamic) */ +static const char *sym_name(uint32_t a, char *buf, int cap) +{ + Dl_info di; + if (dladdr((void *)(uintptr_t)a, &di) && di.dli_sname) snprintf(buf, cap, "%s+0x%X", di.dli_sname, a - (uint32_t)(uintptr_t)di.dli_saddr); + else snprintf(buf, cap, "?"); + return buf; +} static void on_fault(int sig, siginfo_t *si, void *ctxv) { + static volatile int in_crash; + if (__atomic_exchange_n(&in_crash, 1, __ATOMIC_SEQ_CST)) _exit(1); /* (a crash while reporting one) */ ucontext_t *uc = ctxv; - uya_log("crash: signal %d, fault address %p, eip=0x%08X", sig, si->si_addr, (uint32_t)uc->uc_mcontext.gregs[REG_EIP]); + greg_t *g = uc->uc_mcontext.gregs; + char nm[256]; + uint32_t eip = (uint32_t)g[REG_EIP]; + uya_log("crash: signal %d, fault address %p, eip=0x%08X", sig, si->si_addr, eip); + uya_log("crash: in %s", sym_name(eip, nm, sizeof nm)); + { /* the return addresses on the stack, named: the call chain */ + const uint32_t *sp = (const uint32_t *)(uintptr_t)(uint32_t)g[REG_ESP]; + Dl_info di; + for (int i = 0, shown = 0; i < 256 && shown < 12; i++) { + uint32_t v = sp[i]; + if (v >= 0x40000000u && v < 0x80000000u && dladdr((void *)(uintptr_t)v, &di) && di.dli_sname && di.dli_fbase == dladdr_base()) { + uya_log("crash: called from %s", sym_name(v, nm, sizeof nm)); shown++; + } + } + } uya_report(); _exit(1); } @@ -153,10 +185,10 @@ int plat_install_trap_handler(void) #endif #ifdef __linux__ -/* No window on Linux (yet): with UYA_DUMP=dir every 15th presented frame is - * written there as a PPM, so output can be checked. */ +/* Headless (no display, no SDL2, or UYA_HEADLESS / UYA_DUMP set): with UYA_DUMP=dir every + * 15th presented frame is written there as a PPM, so output can be checked. */ #include -int plat_present(const uint32_t *px, int w, int h) +static int hl_present(const uint32_t *px, int w, int h) { static unsigned long n; const char *dir = getenv("UYA_DUMP"); @@ -189,8 +221,7 @@ static const struct { const char *name; unsigned bit; } pad_names[] = { {"lup", 0x10000}, {"ldown", 0x20000}, {"lleft", 0x40000}, {"lright", 0x80000}, {"rup", 0x100000}, {"rdown", 0x200000}, {"rleft", 0x400000}, {"rright", 0x800000}, }; -int plat_pad_sticks(uint8_t st[4]) { (void)st; return 0; } -unsigned plat_pad_buttons(void) +static unsigned hl_buttons(void) { const char *s = getenv("UYA_KEYS"); unsigned now = uya_frame_count(), bits = 0; @@ -211,18 +242,9 @@ unsigned plat_pad_buttons(void) return bits; } /* UYA_SKIPMOVIES=n skips the first n movies (plat_movie_started counts them) */ -static int movies_started; -void plat_movie_started(void) { movies_started++; } -int plat_key_skip(void) -{ - const char *e = getenv("UYA_SKIPMOVIES"); - if (e && movies_started <= atoi(e)) return 1; - return (plat_pad_buttons() & 0x4008) != 0; -} -void plat_poll(void) {} /* Sound on Linux: with UYA_DUMP set, each sound is saved there as a WAV file. */ -void plat_audio_play(const int16_t *lr, size_t frames, int rate, const char *tag) +static void hl_audio_play(const int16_t *lr, size_t frames, int rate, const char *tag) { const char *dir = getenv("UYA_DUMP"); if (!dir) return; @@ -241,7 +263,6 @@ void plat_audio_play(const int16_t *lr, size_t frames, int rate, const char *tag fwrite(lr, 4, frames, f); fclose(f); } -void plat_audio_stop(void) {} #endif #ifdef __linux__ @@ -250,7 +271,7 @@ void plat_audio_stop(void) {} static uint64_t stream_written, stream_t0; uint64_t uya_fixed_clock(void); static uint64_t stream_now(void) { uint64_t v = uya_fixed_clock(); return v ? v : plat_time_us(); } /* UYA_FIXEDTIME: the game's clock */ -void plat_stream_write(const int16_t *lr, int frames) +static void hl_stream_write(const int16_t *lr, int frames) { static FILE *f; static int tried; @@ -263,18 +284,46 @@ void plat_stream_write(const int16_t *lr, int frames) if (f) fwrite(lr, 4, (size_t)frames, f); stream_written += (uint64_t)frames; } -int plat_stream_queued(void) +static int hl_stream_queued(void) { if (!stream_t0) return 0; uint64_t played = (stream_now() - stream_t0) * 48000 / 1000000; return played >= stream_written ? 0 : (int)(stream_written - played); } -/* OpenGL: this build is 32-bit static (musl) and cannot load a GL driver, so - * the GS draws in software (the GL renderer is tested with tools/gs_replay.c) */ -int plat_gl_active; -int plat_gl_init(void) { return 0; } -void plat_gl_current(int on) { (void)on; } -void plat_gl_swap(void) {} -void plat_gl_size(int *w, int *h) { *w = 640; *h = 480; } -void *plat_gl_proc(const char *name) { (void)name; return 0; } +#include "platform_linux_sdl.h" + +/* ---- what the game calls: SDL when there is a display, else headless (testing) ---- */ +int plat_present(const uint32_t *px, int w, int h) { return sdl_on() ? sdl_present(px, w, h) : hl_present(px, w, h); } +int plat_pad_sticks(uint8_t st[4]) { return sdl_on() ? sdl_sticks(st) : 0; } +unsigned plat_pad_buttons(void) { return hl_buttons() | (sdl_on() ? sdl_buttons() : 0); } /* (UYA_KEYS: scripted presses) */ +/* movies: a fresh press of Cross or Start skips (buttons already held when the movie starts do not); + * UYA_SKIPMOVIES=n skips the first n movies (testing) */ +static int movies_started; +static unsigned skip_held; +void plat_movie_started(void) { movies_started++; pump(); skip_held = plat_pad_buttons() & 0x4008; } +int plat_key_skip(void) +{ + const char *e = getenv("UYA_SKIPMOVIES"); + if (e && movies_started <= atoi(e)) return 1; + pump(); + unsigned b = plat_pad_buttons() & 0x4008, fresh = b & ~skip_held; + skip_held = b; + return fresh != 0; +} +void plat_poll(void) +{ + if (!win) return; + pump(); + if (win_closed) { uya_log("window closed"); exit(0); } +} +void plat_audio_play(const int16_t *lr, size_t frames, int rate, const char *tag) { if (sdl_on()) sdl_audio_play(lr, frames, rate); else hl_audio_play(lr, frames, rate, tag); } +void plat_audio_stop(void) { if (sdl_on()) sdl_audio_stop(); } +void plat_stream_write(const int16_t *lr, int frames) { if (sdl_on()) sdl_stream_write(lr, frames); else hl_stream_write(lr, frames); } +int plat_stream_queued(void) { return sdl_on() ? sdl_stream_queued() : hl_stream_queued(); } +/* OpenGL through SDL (none headless: the GS draws in software) */ +int plat_gl_init(void) { return sdl_gl_init(); } +void plat_gl_current(int on) { sdl_gl_current(on); } +void plat_gl_swap(void) { sdl_gl_swap(); } +void plat_gl_size(int *w, int *h) { sdl_gl_size(w, h); } +void *plat_gl_proc(const char *name) { return sdl_gl_proc(name); } #endif diff --git a/games/rac1/host/src/platform_linux_sdl.h b/games/rac1/host/src/platform_linux_sdl.h new file mode 100644 index 0000000..f58eb09 --- /dev/null +++ b/games/rac1/host/src/platform_linux_sdl.h @@ -0,0 +1,310 @@ +/* platform_linux_sdl.h: the window, OpenGL, gamepads and sound on Linux, through SDL2 + * (included once by platform_linux.c). + * + * libSDL2-2.0.so.0 is loaded at run time (dlopen), so the game builds without SDL's + * headers and still runs headless (testing) where there is no SDL or no display. Only + * the few SDL2 declarations used are written out here (SDL2's ABI is stable). + * + * window one resizable window, 640x480 to start; F11 or UYA_FULLSCREEN=1 for full screen + * OpenGL a 3.3 core context for the GS renderer (gs_gl.c); vsync with UYA_VSYNC=1 + * gamepads SDL's game controller API: Xbox, PlayStation, Switch and Steam Deck controls + * by position (south = Cross, east = Circle, west = Square, north = Triangle) + * sound two queued audio devices: one-shot sounds (plat_audio_play), and the 48 kHz + * effects stream (plat_stream_write) */ +#include + +/* ---- SDL2 declarations (SDL.h, 2.0.x) ---- */ +typedef struct SDL_Window SDL_Window; +typedef struct SDL_GameController SDL_GameController; +typedef void *SDL_GLContext; +typedef uint32_t SDL_AudioDeviceID; +typedef struct { int freq; uint16_t format; uint8_t channels, silence; uint16_t samples, padding; uint32_t size; void *callback, *userdata; } SDL_AudioSpec; +typedef struct { int x, y, w, h; } SDL_Rect; +typedef struct { uint32_t flags; void *format; int w, h, pitch; void *pixels; } SDL_Surface_head; /* (the start of SDL_Surface) */ +#define SDL_INIT_AUDIO 0x10u +#define SDL_INIT_VIDEO 0x20u +#define SDL_INIT_GAMECONTROLLER 0x2000u +#define SDL_WINDOWPOS_CENTERED 0x2FFF0000 +#define SDL_WINDOW_FULLSCREEN_DESKTOP 0x1001u +#define SDL_WINDOW_OPENGL 0x2u +#define SDL_WINDOW_SHOWN 0x4u +#define SDL_WINDOW_RESIZABLE 0x20u +#define SDL_WINDOW_ALLOW_HIGHDPI 0x2000u +#define SDL_QUIT_EV 0x100u +#define SDL_WINDOWEVENT_EV 0x200u +#define SDL_KEYDOWN_EV 0x300u +#define SDL_WINDOWEVENT_CLOSE 14 +#define SDL_SCANCODE_F9 66 +#define SDL_SCANCODE_F11 68 +#define SDL_GL_DOUBLEBUFFER 5 +#define SDL_GL_CONTEXT_MAJOR_VERSION 17 +#define SDL_GL_CONTEXT_MINOR_VERSION 18 +#define SDL_GL_CONTEXT_PROFILE_MASK 21 +#define SDL_GL_CONTEXT_PROFILE_CORE 1 +#define AUDIO_S16LSB 0x8010 +#define SDL_PIXELFORMAT_RGB888 0x16161804u /* (XRGB8888: the frames' BGRX bytes) */ + +static struct { + int (*Init)(uint32_t); + const char *(*GetError)(void); + int (*SetHint)(const char *, const char *); + SDL_Window *(*CreateWindow)(const char *, int, int, int, int, uint32_t); + void (*SetWindowTitle)(SDL_Window *, const char *); + int (*SetWindowFullscreen)(SDL_Window *, uint32_t); + uint32_t (*GetWindowFlags)(SDL_Window *); + void (*GetWindowSize)(SDL_Window *, int *, int *); + int (*PollEvent)(void *); + void (*PumpEvents)(void); + int (*ShowCursor)(int); + int (*GL_SetAttribute)(int, int); + SDL_GLContext (*GL_CreateContext)(SDL_Window *); + int (*GL_MakeCurrent)(SDL_Window *, SDL_GLContext); + void (*GL_SwapWindow)(SDL_Window *); + void *(*GL_GetProcAddress)(const char *); + int (*GL_SetSwapInterval)(int); + void (*GL_GetDrawableSize)(SDL_Window *, int *, int *); + void *(*GetWindowSurface)(SDL_Window *); + int (*UpdateWindowSurface)(SDL_Window *); + void *(*CreateRGBSurfaceWithFormatFrom)(void *, int, int, int, int, uint32_t); + void (*FreeSurface)(void *); + int (*FillRect)(void *, const SDL_Rect *, uint32_t); + int (*UpperBlitScaled)(void *, const SDL_Rect *, void *, SDL_Rect *); + int (*NumJoysticks)(void); + int (*IsGameController)(int); + SDL_GameController *(*GameControllerOpen)(int); + void (*GameControllerClose)(SDL_GameController *); + int (*GameControllerGetAttached)(SDL_GameController *); + const char *(*GameControllerName)(SDL_GameController *); + uint8_t (*GameControllerGetButton)(SDL_GameController *, int); + int16_t (*GameControllerGetAxis)(SDL_GameController *, int); + SDL_AudioDeviceID (*OpenAudioDevice)(const char *, int, const SDL_AudioSpec *, SDL_AudioSpec *, int); + void (*CloseAudioDevice)(SDL_AudioDeviceID); + void (*PauseAudioDevice)(SDL_AudioDeviceID, int); + int (*QueueAudio)(SDL_AudioDeviceID, const void *, uint32_t); + uint32_t (*GetQueuedAudioSize)(SDL_AudioDeviceID); + void (*ClearQueuedAudio)(SDL_AudioDeviceID); +} sdl; + +/* 1: SDL is there and a display is (the game draws in a window); 0: headless */ +static int sdl_state = -1; +static int sdl_on(void) +{ + if (sdl_state >= 0) return sdl_state; + sdl_state = 0; + if (getenv("UYA_HEADLESS") || getenv("UYA_DUMP")) return 0; /* (testing: frames and sound to files) */ + if (!getenv("DISPLAY") && !getenv("WAYLAND_DISPLAY")) { uya_log("no display: running without a window"); return 0; } + void *h = dlopen("libSDL2-2.0.so.0", RTLD_NOW | RTLD_GLOBAL); + if (!h) h = dlopen("libSDL2.so", RTLD_NOW | RTLD_GLOBAL); + if (!h) { uya_log("SDL2 not found (%s): install the 32-bit SDL2 library (libSDL2-2.0.so.0, i386)", dlerror()); return 0; } + static const char *const names[] = { + "SDL_Init", "SDL_GetError", "SDL_SetHint", "SDL_CreateWindow", "SDL_SetWindowTitle", "SDL_SetWindowFullscreen", + "SDL_GetWindowFlags", "SDL_GetWindowSize", "SDL_PollEvent", "SDL_PumpEvents", "SDL_ShowCursor", + "SDL_GL_SetAttribute", "SDL_GL_CreateContext", "SDL_GL_MakeCurrent", "SDL_GL_SwapWindow", "SDL_GL_GetProcAddress", + "SDL_GL_SetSwapInterval", "SDL_GL_GetDrawableSize", "SDL_GetWindowSurface", "SDL_UpdateWindowSurface", + "SDL_CreateRGBSurfaceWithFormatFrom", "SDL_FreeSurface", "SDL_FillRect", "SDL_UpperBlitScaled", + "SDL_NumJoysticks", "SDL_IsGameController", "SDL_GameControllerOpen", "SDL_GameControllerClose", + "SDL_GameControllerGetAttached", "SDL_GameControllerName", "SDL_GameControllerGetButton", "SDL_GameControllerGetAxis", + "SDL_OpenAudioDevice", "SDL_CloseAudioDevice", "SDL_PauseAudioDevice", "SDL_QueueAudio", "SDL_GetQueuedAudioSize", + "SDL_ClearQueuedAudio", + }; + void **fp = (void **)&sdl; + for (unsigned i = 0; i < sizeof names / sizeof names[0]; i++) + if (!(fp[i] = dlsym(h, names[i]))) { uya_log("SDL2: %s missing (SDL 2.0.10 or newer needed)", names[i]); return 0; } + sdl.SetHint("SDL_JOYSTICK_ALLOW_BACKGROUND_EVENTS", "1"); + sdl.SetHint("SDL_VIDEO_X11_NET_WM_BYPASS_COMPOSITOR", "0"); + if (sdl.Init(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER | SDL_INIT_AUDIO) != 0 && sdl.Init(SDL_INIT_VIDEO | SDL_INIT_GAMECONTROLLER) != 0) { + uya_log("SDL2: cannot start (%s)", sdl.GetError()); + return 0; + } + sdl_state = 1; + return 1; +} + +/* ---- the window ---- */ +static SDL_Window *win; +static SDL_GLContext glc; +int plat_gl_active; +static void win_create(uint32_t extra) +{ + if (win || !sdl_on()) return; + uint32_t fl = SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE | SDL_WINDOW_ALLOW_HIGHDPI | extra; + const char *fs = getenv("UYA_FULLSCREEN"); + if (fs && atoi(fs)) fl |= SDL_WINDOW_FULLSCREEN_DESKTOP; + win = sdl.CreateWindow("Project Solana", SDL_WINDOWPOS_CENTERED, SDL_WINDOWPOS_CENTERED, 640, 480, fl); + if (!win) uya_log("SDL2: no window (%s)", sdl.GetError()); + else if (fl & SDL_WINDOW_FULLSCREEN_DESKTOP) sdl.ShowCursor(0); +} + +static void toggle_fullscreen(void) +{ + if (!win) return; + int fs = (sdl.GetWindowFlags(win) & SDL_WINDOW_FULLSCREEN_DESKTOP) == SDL_WINDOW_FULLSCREEN_DESKTOP; + sdl.SetWindowFullscreen(win, fs ? 0 : SDL_WINDOW_FULLSCREEN_DESKTOP); + sdl.ShowCursor(fs); +} + +static volatile int win_closed; +static void pump(void) +{ + if (!win) return; + union { uint32_t type; uint8_t b[64]; struct { uint32_t type, ts, wid; uint8_t state, repeat, p2, p3; int32_t scancode, sym; } key; + struct { uint32_t type, ts, wid; uint8_t event; } w; uint8_t pad[128]; } ev; + while (sdl.PollEvent(&ev)) { + if (ev.type == SDL_QUIT_EV) win_closed = 1; + else if (ev.type == SDL_WINDOWEVENT_EV && ev.w.event == SDL_WINDOWEVENT_CLOSE) win_closed = 1; + else if (ev.type == SDL_KEYDOWN_EV && !ev.key.repeat) { + if (ev.key.scancode == SDL_SCANCODE_F9) { void uya_gs_rec_hotkey(void); uya_gs_rec_hotkey(); } /* testing: record the GS input */ + if (ev.key.scancode == SDL_SCANCODE_F11) toggle_fullscreen(); + } + } +} + +/* ---- OpenGL ---- */ +void *sdl_gl_proc(const char *name) { return sdl.GL_GetProcAddress ? sdl.GL_GetProcAddress(name) : 0; } +static int sdl_gl_init(void) +{ + if (!sdl_on()) return 0; + sdl.GL_SetAttribute(SDL_GL_CONTEXT_MAJOR_VERSION, 3); + sdl.GL_SetAttribute(SDL_GL_CONTEXT_MINOR_VERSION, 3); + sdl.GL_SetAttribute(SDL_GL_CONTEXT_PROFILE_MASK, SDL_GL_CONTEXT_PROFILE_CORE); + sdl.GL_SetAttribute(SDL_GL_DOUBLEBUFFER, 1); + win_create(SDL_WINDOW_OPENGL); + if (!win) return 0; + glc = sdl.GL_CreateContext(win); + if (!glc) { uya_log("GL: no OpenGL 3.3 core context (%s)", sdl.GetError()); return 0; } + { /* UYA_VSYNC=1: swaps wait for the monitor (the game paces itself anyway) */ + const char *e = getenv("UYA_VSYNC"); + sdl.GL_SetSwapInterval(e ? atoi(e) : 0); + } + return 1; +} +static void sdl_gl_current(int on) { if (glc) sdl.GL_MakeCurrent(win, on ? glc : 0); } +static void sdl_gl_swap(void) { if (glc) sdl.GL_SwapWindow(win); } +static void sdl_gl_size(int *w, int *h) +{ + *w = 640; *h = 480; + if (win && glc) sdl.GL_GetDrawableSize(win, w, h); /* pixels (HiDPI) */ + else if (win) sdl.GetWindowSize(win, w, h); +} + +/* the frame rate in the title, as on Windows */ +static void title_fps(void) +{ + extern unsigned long long nsends; + static uint64_t t0; static unsigned long long s0; + uint64_t t = plat_time_us(); + if (!t0) { t0 = t; s0 = nsends; return; } + if (t - t0 < 1000000) return; + char title[64]; + snprintf(title, sizeof title, "Project Solana - %.0f FPS", (double)(nsends - s0) / ((double)(t - t0) / 1e6)); + sdl.SetWindowTitle(win, title); + t0 = t; s0 = nsends; +} + +/* the software renderer's frames (when there is no OpenGL): scaled into the window, 4:3 */ +static int sdl_present(const uint32_t *px, int w, int h) +{ + win_create(0); + pump(); + if (win_closed) return 0; + if (!win) return 1; + title_fps(); + if (plat_gl_active || !px) return 1; /* (GL: the GS thread draws the window) */ + void *dst = sdl.GetWindowSurface(win); + void *src = sdl.CreateRGBSurfaceWithFormatFrom((void *)px, w, h, 32, w * 4, SDL_PIXELFORMAT_RGB888); + if (dst && src) { + const SDL_Surface_head *d = dst; + int vw = d->w, vh = d->w * 3 / 4; + if (vh > d->h) { vh = d->h; vw = d->h * 4 / 3; } + SDL_Rect r = { (d->w - vw) / 2, (d->h - vh) / 2, vw, vh }; + sdl.FillRect(dst, 0, 0); + sdl.UpperBlitScaled(src, 0, dst, &r); + sdl.UpdateWindowSurface(win); + } + if (src) sdl.FreeSurface(src); + return 1; +} + +/* ---- gamepads ---- */ +static SDL_GameController *pad; +static uint64_t pad_next; +static SDL_GameController *pad_get(void) +{ + if (pad && !sdl.GameControllerGetAttached(pad)) { uya_log("gamepad: disconnected"); sdl.GameControllerClose(pad); pad = 0; } + if (pad) return pad; + uint64_t t = plat_time_us(); + if (t < pad_next) return 0; + pad_next = t + 1000000; /* looked for every second */ + int n = sdl.NumJoysticks(); + for (int i = 0; i < n && !pad; i++) + if (sdl.IsGameController(i) && (pad = sdl.GameControllerOpen(i))) uya_log("gamepad: %s", sdl.GameControllerName(pad)); + return pad; +} +static unsigned sdl_buttons(void) +{ + if (!sdl_on()) return 0; + if (!win) sdl.PumpEvents(); + SDL_GameController *g = pad_get(); + if (!g) return 0; + /* SDL_GameControllerButton: A B X Y Back Guide Start LS RS LB RB Up Down Left Right */ + static const unsigned map[15] = { 0x4000, 0x2000, 0x8000, 0x1000, 0x0001, 0, 0x0008, 0x0002, 0x0004, 0x0400, 0x0800, 0x0010, 0x0040, 0x0080, 0x0020 }; + unsigned b = 0; + for (int i = 0; i < 15; i++) if (map[i] && sdl.GameControllerGetButton(g, i)) b |= map[i]; + if (sdl.GameControllerGetAxis(g, 4) > 8000) b |= 0x0100; /* L2 */ + if (sdl.GameControllerGetAxis(g, 5) > 8000) b |= 0x0200; /* R2 */ + return b; +} +static uint8_t sdl_axis(int16_t v) /* -32768..32767 -> 0..255, 0x80 centre, with a dead zone; down is + (as the PS2) */ +{ + int x = v / 256; + if (x > -24 && x < 24) return 0x7F; + x += 0x80; + return (uint8_t)(x < 0 ? 0 : x > 255 ? 255 : x); +} +static int sdl_sticks(uint8_t st[4]) +{ + if (!sdl_on()) return 0; + SDL_GameController *g = pad_get(); + if (!g) return 0; + for (int i = 0; i < 4; i++) st[i] = sdl_axis(sdl.GameControllerGetAxis(g, i)); + return 1; +} + +/* ---- sound ---- */ +static SDL_AudioDeviceID one_dev, stream_dev; +static int stream_failed; +static SDL_AudioDeviceID audio_open(int rate) +{ + SDL_AudioSpec want, have; + memset(&want, 0, sizeof want); + want.freq = rate; want.format = AUDIO_S16LSB; want.channels = 2; want.samples = 1024; + SDL_AudioDeviceID d = sdl.OpenAudioDevice(0, 0, &want, &have, 0); /* (SDL converts to what the device takes) */ + if (!d) { uya_log("sound: no output device (%s)", sdl.GetError()); return 0; } + sdl.PauseAudioDevice(d, 0); + return d; +} +static void sdl_audio_stop(void) +{ + if (!one_dev) return; + sdl.ClearQueuedAudio(one_dev); + sdl.CloseAudioDevice(one_dev); + one_dev = 0; +} +static void sdl_audio_play(const int16_t *lr, size_t frames, int rate) +{ + sdl_audio_stop(); + if (!(one_dev = audio_open(rate))) return; + sdl.QueueAudio(one_dev, lr, (uint32_t)(frames * 4)); +} +#define STREAM_MAX_FRAMES (32 * 1024) /* as the Windows ring: beyond it, writes are dropped */ +static int sdl_stream_queued(void) +{ + if (!stream_dev) return stream_failed ? 1 << 20 : 0; + return (int)(sdl.GetQueuedAudioSize(stream_dev) / 4); +} +static void sdl_stream_write(const int16_t *lr, int frames) +{ + if (!stream_dev && !stream_failed && !(stream_dev = audio_open(48000))) stream_failed = 1; + if (!stream_dev || sdl_stream_queued() + frames > STREAM_MAX_FRAMES) return; + sdl.QueueAudio(stream_dev, lr, (uint32_t)frames * 4); +} diff --git a/launcher/linux/project-solana b/launcher/linux/project-solana new file mode 100644 index 0000000..daf609c --- /dev/null +++ b/launcher/linux/project-solana @@ -0,0 +1,319 @@ +#!/usr/bin/env python3 +"""Project Solana for Linux: builds the game from your own disc and plays it. + + project-solana build DISC.iso identify the disc, fetch the build files, build the game (once) + project-solana play [options] play it (--upscaled / --native, --fullscreen / --windowed, --vsync) + project-solana status what is installed, and whether the 32-bit libraries it needs are there + project-solana desktop add Project Solana to your applications menu (and so to Steam) + project-solana log print the last game log (rac1_log.txt) + +Like the Windows launcher: nothing from the game is downloaded. The build files (the +recompiler, the runtime, the decomp's source) come from the project's GitHub release, the +game itself from your disc image. Everything lives in ~/.local/share/project-solana +(or $XDG_DATA_HOME/project-solana); `play` remembers your last options. + +Needs: python3 (3.6 or newer), an x86-64 CPU with SSE4.2, and the 32-bit (i386) SDL2 and +OpenGL libraries, which Steam installs (SteamOS and any PC with Steam have them).""" +import argparse, configparser, hashlib, json, os, shutil, struct, subprocess, sys, time, urllib.request, zipfile + +RELEASE_URL = "https://github.com/OpenRAC/project-solana/releases/download/" +GAMES = { # serial (BOOT2 in SYSTEM.CNF): what this version is + "SCES_509.16": {"title": "Ratchet & Clank (PAL)", "kit": "rac1", "md5": "d86429e2eca4e1da7d7f1cc656af2574", "exe": "rac1_wrapper"}, +} +OTHER = {"SCUS_971.99": "Ratchet & Clank (NTSC-U)", "SCPS_150.37": "Ratchet & Clank (NTSC-J)", + "SCUS_972.68": "Ratchet & Clank: Going Commando (NTSC-U)", "SCES_516.07": "Ratchet & Clank 2 (PAL)", + "SCUS_973.53": "Ratchet & Clank: Up Your Arsenal (NTSC-U)", "SCES_524.56": "Ratchet & Clank 3 (PAL)", + "SCUS_974.65": "Ratchet: Deadlocked (NTSC-U)", "SCES_532.85": "Ratchet: Gladiator (PAL)"} + +HOME = os.path.join(os.environ.get("XDG_DATA_HOME") or os.path.expanduser("~/.local/share"), "project-solana") +INI = os.path.join(HOME, "solana.ini") + + +def say(*a): print(*a, flush=True) +def die(msg): sys.exit("project-solana: " + msg) + + +def settings(): + c = configparser.ConfigParser() + c.read(INI) + for s in ("disc", "play", "build"): + if not c.has_section(s): c.add_section(s) + return c + + +def save(c): + os.makedirs(HOME, exist_ok=True) + with open(INI, "w") as f: c.write(f) + + +# ---- the disc: ISO 9660, SYSTEM.CNF's BOOT2, and that executable's MD5 (as the Windows launcher) ---- +def iso_root_find(f, root, name): + o = 0 + while o < len(root): + ln = root[o] + if not ln: o = (o // 2048 + 1) * 2048; continue + fl = root[o + 32] + fn = root[o + 33:o + 33 + fl].decode("latin-1") + if fn.split(";")[0].upper() == name.upper(): + return struct.unpack_from(" 0.5: + last = time.time() + print("\r %s: %.0f of %.0f MB " % (what, got / 1048576, total / 1048576) if total else "\r %s: %.0f MB " % (what, got / 1048576), end="", flush=True) + if what: print("\r %s: done%s" % (what, " " * 30), flush=True) + if out: out.close(); os.replace(dest + ".part", dest); return None + return b"".join(buf) + + +def sha256(p): + h = hashlib.sha256() + with open(p, "rb") as f: + for b in iter(lambda: f.read(1 << 20), b""): h.update(b) + return h.hexdigest() + + +def unzip(zp, dest, what): + with zipfile.ZipFile(zp) as z: + n = len(z.infolist()) + for i, info in enumerate(z.infolist()): + z.extract(info, dest) + mode = (info.external_attr >> 16) & 0o777 + if mode and not info.is_dir(): os.chmod(os.path.join(dest, info.filename), mode) + if i % 200 == 0: print("\r unpacking %s: %d of %d files" % (what, i + 1, n), end="", flush=True) + print("\r unpacking %s: done%s" % (what, " " * 20), flush=True) + + +def rmtree(p): + if os.path.isdir(p): shutil.rmtree(p, ignore_errors=True) + elif os.path.exists(p): os.remove(p) + + +# ---- the 32-bit libraries the game loads ---- +LIBDIRS = ["/usr/lib32", "/usr/lib/i386-linux-gnu", "/lib/i386-linux-gnu", "/usr/lib/i686-linux-gnu", "/lib32"] + + +def have_lib(name): + if any(os.path.exists(os.path.join(d, name)) for d in LIBDIRS): return True + try: # (Fedora and others keep 32-bit libraries in /usr/lib: ask the linker cache) + out = subprocess.run(["ldconfig", "-p"], capture_output=True, text=True).stdout + return any(name in l and "x86-64" not in l and "64bit" not in l for l in out.splitlines()) + except OSError: + return False + + +def check_libs(quiet=False): + missing = [n for n in ("libSDL2-2.0.so.0", "libGL.so.1") if not have_lib(n)] + if missing and not quiet: + say("\nThe game is a 32-bit program and needs these 32-bit libraries: %s" % ", ".join(missing)) + say("Installing Steam brings them. Otherwise:") + say(" Debian, Ubuntu, Mint, Pop!_OS: sudo dpkg --add-architecture i386 && sudo apt update && sudo apt install libsdl2-2.0-0:i386 libgl1:i386 libgl1-mesa-dri:i386") + say(" Fedora: sudo dnf install SDL2.i686 mesa-libGL.i686 mesa-dri-drivers.i686") + say(" Arch, Manjaro, EndeavourOS: enable [multilib], then sudo pacman -S lib32-sdl2 lib32-mesa") + say(" openSUSE: sudo zypper install libSDL2-2_0-0-32bit Mesa-libGL1-32bit") + say(" (NVIDIA's own driver: also its 32-bit OpenGL library, e.g. lib32-nvidia-utils on Arch)") + return not missing + + +# ---- build ---- +def progress_line(line): + p = line.split() + if len(p) >= 4 and p[0] == "[progress]": + ph, i, n = p[1], int(p[2]), max(int(p[3]), 1) + label = {"prepare": "preparing", "levels": "translating the levels", "compile": "compiling", "link": "linking", "done": "done"}.get(ph, ph) + print("\r %s (%d of %d)%s" % (label, i + (ph == "levels"), n, " " * 10) if ph in ("levels", "compile") else "\r %s%s" % (label, " " * 30), end="", flush=True) + return True + return False + + +def build(args): + iso = os.path.abspath(os.path.expanduser(args.disc)) + if not os.path.isfile(iso): die("no such file: %s" % iso) + say("Disc: %s" % iso) + try: serial, md5 = identify(iso) + except (OSError, ValueError) as e: die("%s: %s" % (iso, e)) + g = GAMES.get(serial) + if not g: die("%s (%s) is not supported yet." % (OTHER.get(serial, "this game"), serial) if serial in OTHER else "not a game Project Solana knows (%s)" % serial) + if g["md5"] and md5 != g["md5"]: + die("this is %s, but another version of it (executable MD5 %s): the build is made for %s" % (g["title"], md5, g["md5"])) + say("Game: %s" % g["title"]) + check_libs() + c = settings() + rel = (args.release_url or c.get("build", "release_url", fallback="") or RELEASE_URL).rstrip("/") + "/" + kit = g["kit"] + src = (lambda f: rel + kit + "/" + f) if rel.startswith("http") else (lambda f: os.path.join(rel, f)) + + say("Checking for the latest build files") + man = json.loads(fetch(src(kit + ".json")).decode()) + tc_dir = os.path.join(HOME, "toolchain") + os.makedirs(HOME, exist_ok=True) + have = open(os.path.join(tc_dir, "version.txt")).read().strip() if os.path.exists(os.path.join(tc_dir, "version.txt")) else "" + if have != man["toolchain_version"]: + say("Getting the build tools (Python and the zig compiler, %d MB, kept for later builds)" % (int(man["toolchain_linux_size"]) >> 20)) + zp = os.path.join(HOME, man["toolchain_linux_file"]) + fetch(src(man["toolchain_linux_file"]), zp, "build tools", int(man["toolchain_linux_size"])) + if sha256(zp) != man["toolchain_linux_sha256"]: os.remove(zp); die("the build tools did not download correctly (checksum): try again") + rmtree(tc_dir); unzip(zp, tc_dir, "build tools"); os.remove(zp) + open(os.path.join(tc_dir, "version.txt"), "w").write(man["toolchain_version"]) + + gdir = os.path.join(HOME, "games", kit) + kv = os.path.join(gdir, "kit_version.txt") + old = open(kv).read().strip() if os.path.exists(kv) else "" + if old != man["version"] or args.clean: + say("Getting the build kit %s (%d MB)" % (man["version"], int(man["kit_size"]) >> 20)) + zp = os.path.join(HOME, man["kit_file"]) + fetch(src(man["kit_file"]), zp, "build kit", int(man["kit_size"])) + if sha256(zp) != man["kit_sha256"]: os.remove(zp); die("the build kit did not download correctly (checksum): try again") + if os.path.isdir(gdir): # everything but the build output (and its saves); a new kit version builds from scratch + for e in os.listdir(gdir): + if e != "build": rmtree(os.path.join(gdir, e)) + rmtree(os.path.join(gdir, "build", "linux", "gen")); rmtree(os.path.join(gdir, "build", "linux", "obj")) + unzip(zp, gdir, "build kit"); os.remove(zp) + + levels = c.get("build", "levels", fallback="") or man.get("levels") or ",".join(str(i) for i in range(19)) + cpus = os.cpu_count() or 2 + try: ram_gb = os.sysconf("SC_PAGE_SIZE") * os.sysconf("SC_PHYS_PAGES") / (1 << 30) + except (ValueError, OSError): ram_gb = 8 + jobs = args.jobs or max(1, min(cpus, int(ram_gb / 1.5))) + say("Building the game from your disc (%d jobs): about 15 minutes on a recent 6 to 8 core CPU, longer on older ones" % jobs) + env = dict(os.environ, PATH=os.path.join(tc_dir, "zig") + os.pathsep + os.environ.get("PATH", ""), PYTHONUTF8="1", + ZIG_GLOBAL_CACHE_DIR=os.path.join(gdir, "zig-cache"), ZIG_LOCAL_CACHE_DIR=os.path.join(gdir, "zig-cache")) + py = os.path.join(tc_dir, "python", "bin", "python3") + cmd = [py, "-u", "build.py", "--platform", "linux", "--levels", levels, "--iso", iso, "--repo", "decomp", "-j", str(jobs)] + log = open(os.path.join(gdir, "build_log.txt"), "w") + p = subprocess.Popen(cmd, cwd=gdir, env=env, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, bufsize=1) + for line in p.stdout: + log.write(line) + if not progress_line(line) and ("error" in line.lower() or "failed" in line.lower()): print("\n " + line.rstrip()) + p.wait(); log.close(); print() + if p.returncode: die("the build stopped (see %s)" % os.path.join(gdir, "build_log.txt")) + for j in ("build/linux/gen", "build/linux/gen.new", "build/linux/obj", "build/levels", "build/disc", "zig-cache"): rmtree(os.path.join(gdir, j)) + open(kv, "w").write(man["version"]) + c.set("disc", kit, iso); c.set("build", "built_" + kit, man["version"]); save(c) + say("Built. Ready to play: project-solana play") + + +# ---- play ---- +def play(args): + c = settings() + kit = "rac1" + iso = c.get("disc", kit, fallback="") + exe = os.path.join(HOME, "games", kit, "build", "linux", GAMES["SCES_509.16"]["exe"]) + if not os.path.exists(exe) or not iso: die("not built yet: run project-solana build /path/to/your/disc.iso") + if not os.path.exists(iso): die("the disc image has moved (%s): build again with its new path" % iso) + pc = c["play"] + for k, v in (("resolution", args.resolution), ("fullscreen", args.fullscreen), ("vsync", args.vsync)): + if v is not None: pc[k] = str(v) + save(c) + env = dict(os.environ) + deck = os.environ.get("SteamDeck") == "1" + if pc.get("resolution", "upscaled") == "upscaled": + try: # (as the Windows launcher: render at about the screen's height) + out = subprocess.run(["xrandr", "--current"], capture_output=True, text=True, timeout=5).stdout + h = max(int(l.split("current ")[1].split(" x ")[1].split(",")[0]) for l in out.splitlines() if "current " in l) + except Exception: + h = 800 if deck else 1080 + env["UYA_SCALE"] = str(min(4, max(2, -(-h // 448)))) + full = pc.get("fullscreen", "1" if deck else "0") in ("1", "True", "true") + if full: env["UYA_FULLSCREEN"] = "1" + if pc.get("vsync", "0") in ("1", "True", "true"): env["UYA_VSYNC"] = "1" + if not check_libs(quiet=True): check_libs(); die("install the 32-bit libraries above, then play again") + os.chdir(os.path.dirname(exe)) + os.execve(exe, [exe, "-", iso], env) + + +def status(args): + c = settings() + say("Data folder: %s" % HOME) + tv = os.path.join(HOME, "toolchain", "version.txt") + say("Build tools: %s" % (open(tv).read().strip() if os.path.exists(tv) else "not downloaded")) + for kit in ("rac1",): + b = c.get("build", "built_" + kit, fallback="") + say("%s: %s, disc %s" % (kit, ("built from kit " + b) if b else "not built", c.get("disc", kit, fallback="(none)"))) + say("32-bit SDL2 and OpenGL: %s" % ("found" if check_libs(quiet=True) else "missing")) + if not check_libs(quiet=True): check_libs() + + +def desktop(args): + me = os.path.abspath(sys.argv[0]) + d = os.path.join(os.environ.get("XDG_DATA_HOME") or os.path.expanduser("~/.local/share"), "applications") + os.makedirs(d, exist_ok=True) + p = os.path.join(d, "project-solana-rac1.desktop") + with open(p, "w") as f: + f.write("[Desktop Entry]\nType=Application\nName=Ratchet & Clank (Project Solana)\nComment=Built from your own disc\n" + "Exec=\"%s\" play\nTerminal=false\nCategories=Game;\n" % me) + say("Added %s" % p) + say("Steam: Games > Add a Non-Steam Game to My Library, and pick \"Ratchet & Clank (Project Solana)\".") + + +def show_log(args): + p = os.path.join(HOME, "games", "rac1", "build", "linux", "rac1_log.txt") + if not os.path.exists(p): die("no log yet: it is written when the game runs") + sys.stdout.write(open(p, errors="replace").read()) + + +def main(): + ap = argparse.ArgumentParser(prog="project-solana", description="Build Ratchet & Clank from your own disc, then play it.") + sp = ap.add_subparsers(dest="cmd") + b = sp.add_parser("build", help="build the game from your disc image") + b.add_argument("disc"); b.add_argument("-j", "--jobs", type=int, default=0) + b.add_argument("--clean", action="store_true", help="fetch the build kit again and build from scratch") + b.add_argument("--release-url", default="", help=argparse.SUPPRESS) + p = sp.add_parser("play", help="play (options are remembered)") + g = p.add_mutually_exclusive_group() + g.add_argument("--upscaled", dest="resolution", action="store_const", const="upscaled", help="render at about your screen's resolution (default)") + g.add_argument("--native", dest="resolution", action="store_const", const="native", help="the PS2's own resolution") + f = p.add_mutually_exclusive_group() + f.add_argument("--fullscreen", dest="fullscreen", action="store_const", const=1, help="full screen (F11 switches; default on Steam Deck)") + f.add_argument("--windowed", dest="fullscreen", action="store_const", const=0) + v = p.add_mutually_exclusive_group() + v.add_argument("--vsync", dest="vsync", action="store_const", const=1) + v.add_argument("--no-vsync", dest="vsync", action="store_const", const=0) + sp.add_parser("status", help="what is installed"); sp.add_parser("desktop", help="add it to the applications menu") + sp.add_parser("log", help="print the last game log") + a = ap.parse_args() + if sys.platform != "linux": die("this is the Linux launcher: on Windows, use Project Solana.exe") + {"build": build, "play": play, "status": status, "desktop": desktop, "log": show_log}.get(a.cmd, lambda _: ap.print_help())(a) + + +if __name__ == "__main__": + main() diff --git a/launcher/project-solana b/launcher/project-solana new file mode 100644 index 0000000..daf609c --- /dev/null +++ b/launcher/project-solana @@ -0,0 +1,319 @@ +#!/usr/bin/env python3 +"""Project Solana for Linux: builds the game from your own disc and plays it. + + project-solana build DISC.iso identify the disc, fetch the build files, build the game (once) + project-solana play [options] play it (--upscaled / --native, --fullscreen / --windowed, --vsync) + project-solana status what is installed, and whether the 32-bit libraries it needs are there + project-solana desktop add Project Solana to your applications menu (and so to Steam) + project-solana log print the last game log (rac1_log.txt) + +Like the Windows launcher: nothing from the game is downloaded. The build files (the +recompiler, the runtime, the decomp's source) come from the project's GitHub release, the +game itself from your disc image. Everything lives in ~/.local/share/project-solana +(or $XDG_DATA_HOME/project-solana); `play` remembers your last options. + +Needs: python3 (3.6 or newer), an x86-64 CPU with SSE4.2, and the 32-bit (i386) SDL2 and +OpenGL libraries, which Steam installs (SteamOS and any PC with Steam have them).""" +import argparse, configparser, hashlib, json, os, shutil, struct, subprocess, sys, time, urllib.request, zipfile + +RELEASE_URL = "https://github.com/OpenRAC/project-solana/releases/download/" +GAMES = { # serial (BOOT2 in SYSTEM.CNF): what this version is + "SCES_509.16": {"title": "Ratchet & Clank (PAL)", "kit": "rac1", "md5": "d86429e2eca4e1da7d7f1cc656af2574", "exe": "rac1_wrapper"}, +} +OTHER = {"SCUS_971.99": "Ratchet & Clank (NTSC-U)", "SCPS_150.37": "Ratchet & Clank (NTSC-J)", + "SCUS_972.68": "Ratchet & Clank: Going Commando (NTSC-U)", "SCES_516.07": "Ratchet & Clank 2 (PAL)", + "SCUS_973.53": "Ratchet & Clank: Up Your Arsenal (NTSC-U)", "SCES_524.56": "Ratchet & Clank 3 (PAL)", + "SCUS_974.65": "Ratchet: Deadlocked (NTSC-U)", "SCES_532.85": "Ratchet: Gladiator (PAL)"} + +HOME = os.path.join(os.environ.get("XDG_DATA_HOME") or os.path.expanduser("~/.local/share"), "project-solana") +INI = os.path.join(HOME, "solana.ini") + + +def say(*a): print(*a, flush=True) +def die(msg): sys.exit("project-solana: " + msg) + + +def settings(): + c = configparser.ConfigParser() + c.read(INI) + for s in ("disc", "play", "build"): + if not c.has_section(s): c.add_section(s) + return c + + +def save(c): + os.makedirs(HOME, exist_ok=True) + with open(INI, "w") as f: c.write(f) + + +# ---- the disc: ISO 9660, SYSTEM.CNF's BOOT2, and that executable's MD5 (as the Windows launcher) ---- +def iso_root_find(f, root, name): + o = 0 + while o < len(root): + ln = root[o] + if not ln: o = (o // 2048 + 1) * 2048; continue + fl = root[o + 32] + fn = root[o + 33:o + 33 + fl].decode("latin-1") + if fn.split(";")[0].upper() == name.upper(): + return struct.unpack_from(" 0.5: + last = time.time() + print("\r %s: %.0f of %.0f MB " % (what, got / 1048576, total / 1048576) if total else "\r %s: %.0f MB " % (what, got / 1048576), end="", flush=True) + if what: print("\r %s: done%s" % (what, " " * 30), flush=True) + if out: out.close(); os.replace(dest + ".part", dest); return None + return b"".join(buf) + + +def sha256(p): + h = hashlib.sha256() + with open(p, "rb") as f: + for b in iter(lambda: f.read(1 << 20), b""): h.update(b) + return h.hexdigest() + + +def unzip(zp, dest, what): + with zipfile.ZipFile(zp) as z: + n = len(z.infolist()) + for i, info in enumerate(z.infolist()): + z.extract(info, dest) + mode = (info.external_attr >> 16) & 0o777 + if mode and not info.is_dir(): os.chmod(os.path.join(dest, info.filename), mode) + if i % 200 == 0: print("\r unpacking %s: %d of %d files" % (what, i + 1, n), end="", flush=True) + print("\r unpacking %s: done%s" % (what, " " * 20), flush=True) + + +def rmtree(p): + if os.path.isdir(p): shutil.rmtree(p, ignore_errors=True) + elif os.path.exists(p): os.remove(p) + + +# ---- the 32-bit libraries the game loads ---- +LIBDIRS = ["/usr/lib32", "/usr/lib/i386-linux-gnu", "/lib/i386-linux-gnu", "/usr/lib/i686-linux-gnu", "/lib32"] + + +def have_lib(name): + if any(os.path.exists(os.path.join(d, name)) for d in LIBDIRS): return True + try: # (Fedora and others keep 32-bit libraries in /usr/lib: ask the linker cache) + out = subprocess.run(["ldconfig", "-p"], capture_output=True, text=True).stdout + return any(name in l and "x86-64" not in l and "64bit" not in l for l in out.splitlines()) + except OSError: + return False + + +def check_libs(quiet=False): + missing = [n for n in ("libSDL2-2.0.so.0", "libGL.so.1") if not have_lib(n)] + if missing and not quiet: + say("\nThe game is a 32-bit program and needs these 32-bit libraries: %s" % ", ".join(missing)) + say("Installing Steam brings them. Otherwise:") + say(" Debian, Ubuntu, Mint, Pop!_OS: sudo dpkg --add-architecture i386 && sudo apt update && sudo apt install libsdl2-2.0-0:i386 libgl1:i386 libgl1-mesa-dri:i386") + say(" Fedora: sudo dnf install SDL2.i686 mesa-libGL.i686 mesa-dri-drivers.i686") + say(" Arch, Manjaro, EndeavourOS: enable [multilib], then sudo pacman -S lib32-sdl2 lib32-mesa") + say(" openSUSE: sudo zypper install libSDL2-2_0-0-32bit Mesa-libGL1-32bit") + say(" (NVIDIA's own driver: also its 32-bit OpenGL library, e.g. lib32-nvidia-utils on Arch)") + return not missing + + +# ---- build ---- +def progress_line(line): + p = line.split() + if len(p) >= 4 and p[0] == "[progress]": + ph, i, n = p[1], int(p[2]), max(int(p[3]), 1) + label = {"prepare": "preparing", "levels": "translating the levels", "compile": "compiling", "link": "linking", "done": "done"}.get(ph, ph) + print("\r %s (%d of %d)%s" % (label, i + (ph == "levels"), n, " " * 10) if ph in ("levels", "compile") else "\r %s%s" % (label, " " * 30), end="", flush=True) + return True + return False + + +def build(args): + iso = os.path.abspath(os.path.expanduser(args.disc)) + if not os.path.isfile(iso): die("no such file: %s" % iso) + say("Disc: %s" % iso) + try: serial, md5 = identify(iso) + except (OSError, ValueError) as e: die("%s: %s" % (iso, e)) + g = GAMES.get(serial) + if not g: die("%s (%s) is not supported yet." % (OTHER.get(serial, "this game"), serial) if serial in OTHER else "not a game Project Solana knows (%s)" % serial) + if g["md5"] and md5 != g["md5"]: + die("this is %s, but another version of it (executable MD5 %s): the build is made for %s" % (g["title"], md5, g["md5"])) + say("Game: %s" % g["title"]) + check_libs() + c = settings() + rel = (args.release_url or c.get("build", "release_url", fallback="") or RELEASE_URL).rstrip("/") + "/" + kit = g["kit"] + src = (lambda f: rel + kit + "/" + f) if rel.startswith("http") else (lambda f: os.path.join(rel, f)) + + say("Checking for the latest build files") + man = json.loads(fetch(src(kit + ".json")).decode()) + tc_dir = os.path.join(HOME, "toolchain") + os.makedirs(HOME, exist_ok=True) + have = open(os.path.join(tc_dir, "version.txt")).read().strip() if os.path.exists(os.path.join(tc_dir, "version.txt")) else "" + if have != man["toolchain_version"]: + say("Getting the build tools (Python and the zig compiler, %d MB, kept for later builds)" % (int(man["toolchain_linux_size"]) >> 20)) + zp = os.path.join(HOME, man["toolchain_linux_file"]) + fetch(src(man["toolchain_linux_file"]), zp, "build tools", int(man["toolchain_linux_size"])) + if sha256(zp) != man["toolchain_linux_sha256"]: os.remove(zp); die("the build tools did not download correctly (checksum): try again") + rmtree(tc_dir); unzip(zp, tc_dir, "build tools"); os.remove(zp) + open(os.path.join(tc_dir, "version.txt"), "w").write(man["toolchain_version"]) + + gdir = os.path.join(HOME, "games", kit) + kv = os.path.join(gdir, "kit_version.txt") + old = open(kv).read().strip() if os.path.exists(kv) else "" + if old != man["version"] or args.clean: + say("Getting the build kit %s (%d MB)" % (man["version"], int(man["kit_size"]) >> 20)) + zp = os.path.join(HOME, man["kit_file"]) + fetch(src(man["kit_file"]), zp, "build kit", int(man["kit_size"])) + if sha256(zp) != man["kit_sha256"]: os.remove(zp); die("the build kit did not download correctly (checksum): try again") + if os.path.isdir(gdir): # everything but the build output (and its saves); a new kit version builds from scratch + for e in os.listdir(gdir): + if e != "build": rmtree(os.path.join(gdir, e)) + rmtree(os.path.join(gdir, "build", "linux", "gen")); rmtree(os.path.join(gdir, "build", "linux", "obj")) + unzip(zp, gdir, "build kit"); os.remove(zp) + + levels = c.get("build", "levels", fallback="") or man.get("levels") or ",".join(str(i) for i in range(19)) + cpus = os.cpu_count() or 2 + try: ram_gb = os.sysconf("SC_PAGE_SIZE") * os.sysconf("SC_PHYS_PAGES") / (1 << 30) + except (ValueError, OSError): ram_gb = 8 + jobs = args.jobs or max(1, min(cpus, int(ram_gb / 1.5))) + say("Building the game from your disc (%d jobs): about 15 minutes on a recent 6 to 8 core CPU, longer on older ones" % jobs) + env = dict(os.environ, PATH=os.path.join(tc_dir, "zig") + os.pathsep + os.environ.get("PATH", ""), PYTHONUTF8="1", + ZIG_GLOBAL_CACHE_DIR=os.path.join(gdir, "zig-cache"), ZIG_LOCAL_CACHE_DIR=os.path.join(gdir, "zig-cache")) + py = os.path.join(tc_dir, "python", "bin", "python3") + cmd = [py, "-u", "build.py", "--platform", "linux", "--levels", levels, "--iso", iso, "--repo", "decomp", "-j", str(jobs)] + log = open(os.path.join(gdir, "build_log.txt"), "w") + p = subprocess.Popen(cmd, cwd=gdir, env=env, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True, bufsize=1) + for line in p.stdout: + log.write(line) + if not progress_line(line) and ("error" in line.lower() or "failed" in line.lower()): print("\n " + line.rstrip()) + p.wait(); log.close(); print() + if p.returncode: die("the build stopped (see %s)" % os.path.join(gdir, "build_log.txt")) + for j in ("build/linux/gen", "build/linux/gen.new", "build/linux/obj", "build/levels", "build/disc", "zig-cache"): rmtree(os.path.join(gdir, j)) + open(kv, "w").write(man["version"]) + c.set("disc", kit, iso); c.set("build", "built_" + kit, man["version"]); save(c) + say("Built. Ready to play: project-solana play") + + +# ---- play ---- +def play(args): + c = settings() + kit = "rac1" + iso = c.get("disc", kit, fallback="") + exe = os.path.join(HOME, "games", kit, "build", "linux", GAMES["SCES_509.16"]["exe"]) + if not os.path.exists(exe) or not iso: die("not built yet: run project-solana build /path/to/your/disc.iso") + if not os.path.exists(iso): die("the disc image has moved (%s): build again with its new path" % iso) + pc = c["play"] + for k, v in (("resolution", args.resolution), ("fullscreen", args.fullscreen), ("vsync", args.vsync)): + if v is not None: pc[k] = str(v) + save(c) + env = dict(os.environ) + deck = os.environ.get("SteamDeck") == "1" + if pc.get("resolution", "upscaled") == "upscaled": + try: # (as the Windows launcher: render at about the screen's height) + out = subprocess.run(["xrandr", "--current"], capture_output=True, text=True, timeout=5).stdout + h = max(int(l.split("current ")[1].split(" x ")[1].split(",")[0]) for l in out.splitlines() if "current " in l) + except Exception: + h = 800 if deck else 1080 + env["UYA_SCALE"] = str(min(4, max(2, -(-h // 448)))) + full = pc.get("fullscreen", "1" if deck else "0") in ("1", "True", "true") + if full: env["UYA_FULLSCREEN"] = "1" + if pc.get("vsync", "0") in ("1", "True", "true"): env["UYA_VSYNC"] = "1" + if not check_libs(quiet=True): check_libs(); die("install the 32-bit libraries above, then play again") + os.chdir(os.path.dirname(exe)) + os.execve(exe, [exe, "-", iso], env) + + +def status(args): + c = settings() + say("Data folder: %s" % HOME) + tv = os.path.join(HOME, "toolchain", "version.txt") + say("Build tools: %s" % (open(tv).read().strip() if os.path.exists(tv) else "not downloaded")) + for kit in ("rac1",): + b = c.get("build", "built_" + kit, fallback="") + say("%s: %s, disc %s" % (kit, ("built from kit " + b) if b else "not built", c.get("disc", kit, fallback="(none)"))) + say("32-bit SDL2 and OpenGL: %s" % ("found" if check_libs(quiet=True) else "missing")) + if not check_libs(quiet=True): check_libs() + + +def desktop(args): + me = os.path.abspath(sys.argv[0]) + d = os.path.join(os.environ.get("XDG_DATA_HOME") or os.path.expanduser("~/.local/share"), "applications") + os.makedirs(d, exist_ok=True) + p = os.path.join(d, "project-solana-rac1.desktop") + with open(p, "w") as f: + f.write("[Desktop Entry]\nType=Application\nName=Ratchet & Clank (Project Solana)\nComment=Built from your own disc\n" + "Exec=\"%s\" play\nTerminal=false\nCategories=Game;\n" % me) + say("Added %s" % p) + say("Steam: Games > Add a Non-Steam Game to My Library, and pick \"Ratchet & Clank (Project Solana)\".") + + +def show_log(args): + p = os.path.join(HOME, "games", "rac1", "build", "linux", "rac1_log.txt") + if not os.path.exists(p): die("no log yet: it is written when the game runs") + sys.stdout.write(open(p, errors="replace").read()) + + +def main(): + ap = argparse.ArgumentParser(prog="project-solana", description="Build Ratchet & Clank from your own disc, then play it.") + sp = ap.add_subparsers(dest="cmd") + b = sp.add_parser("build", help="build the game from your disc image") + b.add_argument("disc"); b.add_argument("-j", "--jobs", type=int, default=0) + b.add_argument("--clean", action="store_true", help="fetch the build kit again and build from scratch") + b.add_argument("--release-url", default="", help=argparse.SUPPRESS) + p = sp.add_parser("play", help="play (options are remembered)") + g = p.add_mutually_exclusive_group() + g.add_argument("--upscaled", dest="resolution", action="store_const", const="upscaled", help="render at about your screen's resolution (default)") + g.add_argument("--native", dest="resolution", action="store_const", const="native", help="the PS2's own resolution") + f = p.add_mutually_exclusive_group() + f.add_argument("--fullscreen", dest="fullscreen", action="store_const", const=1, help="full screen (F11 switches; default on Steam Deck)") + f.add_argument("--windowed", dest="fullscreen", action="store_const", const=0) + v = p.add_mutually_exclusive_group() + v.add_argument("--vsync", dest="vsync", action="store_const", const=1) + v.add_argument("--no-vsync", dest="vsync", action="store_const", const=0) + sp.add_parser("status", help="what is installed"); sp.add_parser("desktop", help="add it to the applications menu") + sp.add_parser("log", help="print the last game log") + a = ap.parse_args() + if sys.platform != "linux": die("this is the Linux launcher: on Windows, use Project Solana.exe") + {"build": build, "play": play, "status": status, "desktop": desktop, "log": show_log}.get(a.cmd, lambda _: ap.print_help())(a) + + +if __name__ == "__main__": + main() diff --git a/rac1.json b/rac1.json new file mode 100644 index 0000000..377db23 --- /dev/null +++ b/rac1.json @@ -0,0 +1,12 @@ +{ + "game": "rac1", + "version": "2026.10.14.1", + "levels": "0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18", + "kit_file": "rac1-build.zip", + "kit_size": 5125429, + "kit_sha256": "b6919dd6d31d1ae8fac65c2859eaa6c084e98b97af36e3a305b4253da40ba5c5", + "toolchain_file": "toolchain-win64.zip", + "toolchain_size": 84006129, + "toolchain_sha256": "2d56894833ca4066faf5ac008d3fa06aa40e2bb1d1fcb8f5bd26f28a70382b63", + "toolchain_version": "py3.12.7-zig0.16.0-rabbitizer1.16.2" +} \ No newline at end of file diff --git a/release/make_release.py b/release/make_release.py index 333fb07..1d91e92 100644 --- a/release/make_release.py +++ b/release/make_release.py @@ -2,7 +2,7 @@ """make_release.py: makes the files of a game's GitHub release (the tag is the game, e.g. "rac1"). usage: python release/make_release.py rac1 --decomp PATH_TO_RAC1_DECOMP [--version V] - [--toolchain-zip toolchain-win64.zip] + [--toolchain-zip toolchain-win64.zip] [--toolchain-linux-zip toolchain-linux64.zip] Writes dist/rac1/: rac1.json what the launcher's Build button fetches (version, files, SHA-256s, levels) @@ -10,8 +10,9 @@ Writes dist/rac1/: source as decomp/ (everything but baserom/ and .git: no game files) toolchain-win64.zip Python, rabbitizer and zig for Windows x64 (release/toolchain.json); reused from --toolchain-zip when given, else fetched (PyPI and GitHub) and assembled + toolchain-linux64.zip the same for Linux x86-64 (--toolchain-linux-zip to reuse one) -Upload the three files to the release named after the game (gh release create rac1 dist/rac1/*). +Upload the four files to the release named after the game (gh release create rac1 dist/rac1/*). Players never download anything from the game itself: the build reads their own disc image.""" import argparse, datetime, glob, hashlib, io, json, os, shutil, subprocess, sys, tarfile, tempfile, zipfile @@ -21,7 +22,8 @@ ap = argparse.ArgumentParser() ap.add_argument("game") ap.add_argument("--decomp", required=True, help="the game's decomp repo (only its source is packed; baserom/ is left out)") ap.add_argument("--version", default=datetime.date.today().strftime("%Y.%m.%d")) -ap.add_argument("--toolchain-zip", default=None) +ap.add_argument("--toolchain-zip", default=None, help="reuse this toolchain-win64.zip") +ap.add_argument("--toolchain-linux-zip", default=None, help="reuse this toolchain-linux64.zip") ap.add_argument("--levels", default=",".join(str(i) for i in range(19))) a = ap.parse_args() @@ -35,7 +37,7 @@ SKIP_DIRS = {"__pycache__", ".git", "build", "dev", "decomp", "zig-cache"} DECOMP_SKIP = {"baserom", ".git", ".github"} -def add_tree(z, top, arc, skip): +def add_tree(z, top, arc, skip, modes=False): for root, dirs, files in os.walk(top): dirs[:] = sorted(d for d in dirs if d not in skip) for f in sorted(files): @@ -45,7 +47,7 @@ def add_tree(z, top, arc, skip): # newer, so a retry of the same kit reuses them (a new kit version starts clean) zi = zipfile.ZipInfo(os.path.join(arc, os.path.relpath(p, top)).replace(os.sep, "/"), date_time=(2020, 1, 1, 0, 0, 0)) zi.compress_type = zipfile.ZIP_DEFLATED - zi.external_attr = 0o644 << 16 + zi.external_attr = ((os.stat(p).st_mode & 0o777) if modes else 0o644) << 16 # (Linux: executables stay executable) with open(p, "rb") as fh: z.writestr(zi, fh.read(), compresslevel=9) @@ -67,44 +69,70 @@ with zipfile.ZipFile(kit, "w", zipfile.ZIP_DEFLATED, compresslevel=9) as z: if n.startswith("decomp/baserom/") or n.endswith(("SCES_509.16", ".iso")): sys.exit("refusing to pack %s" % n) print("kit:", kit, os.path.getsize(kit) // 1024, "KB") -# the toolchain +# the toolchains: Python, rabbitizer and zig, for building on Windows (x64) and on Linux (x86-64) tcj = json.load(open(os.path.join(HERE, "toolchain.json"))) -tcz = os.path.join(out, "toolchain-win64.zip") -if a.toolchain_zip: - if os.path.abspath(a.toolchain_zip) != os.path.abspath(tcz): shutil.copyfile(a.toolchain_zip, tcz) -else: + + +def make_toolchain(plat, out_zip): import urllib.request with tempfile.TemporaryDirectory() as t: - print("fetching", tcj["python"]) - data = urllib.request.urlopen(tcj["python"]).read() + print("fetching", tcj["python"][plat]) + data = urllib.request.urlopen(tcj["python"][plat]).read() tarfile.open(fileobj=io.BytesIO(data)).extractall(t) py = os.path.join(t, "python") for pat in ("*.pdb", "**/*.pdb"): for p in glob.glob(os.path.join(py, pat), recursive=True): os.remove(p) - for d in ("tcl", "Lib/test", "Lib/idlelib", "Lib/tkinter", "Lib/turtledemo", "Lib/ensurepip", "include", "libs", "Scripts"): + site = os.path.join(py, "Lib", "site-packages") if plat == "windows" else glob.glob(os.path.join(py, "lib", "python3.*", "site-packages"))[0] + stdlib = os.path.join(py, "Lib") if plat == "windows" else os.path.dirname(site) + for d in ("test", "idlelib", "tkinter", "turtledemo", "ensurepip", "lib2to3", "config-3.12-x86_64-linux-gnu"): + shutil.rmtree(os.path.join(stdlib, d), ignore_errors=True) + for d in ("tcl", "include", "libs", "Scripts", "share"): shutil.rmtree(os.path.join(py, d), ignore_errors=True) + for p in glob.glob(os.path.join(py, "lib", "libtcl*")) + glob.glob(os.path.join(py, "lib", "libtk*")) + glob.glob(os.path.join(py, "lib", "tcl*")) + glob.glob(os.path.join(py, "lib", "tk*")): + shutil.rmtree(p, ignore_errors=True) if os.path.isdir(p) else os.remove(p) w = os.path.join(t, "wheels") - subprocess.run([sys.executable, "-m", "pip", "download", "--no-deps", "--platform", "win_amd64", "--only-binary=:all:", "-d", w, + tags = ["--platform", "win_amd64"] if plat == "windows" else ["--platform", "manylinux_2_17_x86_64", "--platform", "manylinux2014_x86_64", "--platform", "manylinux_2_12_x86_64"] + subprocess.run([sys.executable, "-m", "pip", "download", "--no-deps", *tags, "--python-version", "3.12", "--only-binary=:all:", "-d", w, tcj["ziglang_wheel"], tcj["rabbitizer_wheel"]], check=True) - for whl in glob.glob(os.path.join(w, "rabbitizer-*.whl")): zipfile.ZipFile(whl).extractall(os.path.join(py, "Lib", "site-packages")) + for whl in glob.glob(os.path.join(w, "rabbitizer-*.whl")): zipfile.ZipFile(whl).extractall(site) for whl in glob.glob(os.path.join(w, "ziglang-*.whl")): zipfile.ZipFile(whl).extractall(os.path.join(t, "zw")) zg = os.path.join(t, "zig") shutil.move(os.path.join(t, "zw", "ziglang"), zg) for f in ("__init__.py", "__main__.py"): if os.path.exists(os.path.join(zg, f)): os.remove(os.path.join(zg, f)) - # the C library sources and headers of other systems are not needed for x86-windows-gnu + if plat == "linux": + os.chmod(os.path.join(zg, "zig"), 0o755) + # symlinks would be stored as second copies, and the binaries carry debug symbols (hundreds of MB) + for root, dirs, files in os.walk(py): + for f in files: + q = os.path.join(root, f) + if os.path.islink(q) and f.startswith("libpython"): os.remove(q) # (libpython3.12.so: for linking against it) + for q in glob.glob(os.path.join(py, "lib", "libpython3*.so*")) + [os.path.join(zg, "zig")]: + subprocess.run(["strip", "--strip-unneeded", q], check=True) + # the C library sources and headers of other systems: x86-windows-gnu needs mingw, x86-linux-gnu glibc + keep_libc = ("mingw", "include") if plat == "windows" else ("glibc", "include", "include-linux") lc = os.path.join(zg, "lib", "libc") for d in os.listdir(lc): - if d in ("glibc", "musl", "darwin", "wasi", "freebsd", "netbsd", "openbsd"): shutil.rmtree(os.path.join(lc, d)) + if d in ("glibc", "musl", "darwin", "wasi", "freebsd", "netbsd", "openbsd", "mingw") and d not in keep_libc: shutil.rmtree(os.path.join(lc, d)) inc = os.path.join(lc, "include") for d in os.listdir(inc): - if "windows" not in d: shutil.rmtree(os.path.join(inc, d)) + want = "windows" in d if plat == "windows" else (d.endswith(("linux-gnu", "linux-any", "-glibc")) or d in ("generic-glibc", "any-linux-any", "x86-linux-any", "x86-linux-gnu")) + if not want: shutil.rmtree(os.path.join(inc, d)) for d in ("libcxx", "libcxxabi", "libunwind", "libtsan", "docs", "build-web", "init"): shutil.rmtree(os.path.join(zg, "lib", d), ignore_errors=True) - with zipfile.ZipFile(tcz, "w", zipfile.ZIP_DEFLATED, compresslevel=9) as z: - add_tree(z, py, "python", set()) - add_tree(z, zg, "zig", set()) -print("toolchain:", tcz, os.path.getsize(tcz) // (1 << 20), "MB") + with zipfile.ZipFile(out_zip, "w", zipfile.ZIP_DEFLATED, compresslevel=9) as z: + add_tree(z, py, "python", set(), modes=plat == "linux") + add_tree(z, zg, "zig", set(), modes=plat == "linux") + + +tc = {} +for plat, name, reuse in (("windows", "toolchain-win64.zip", a.toolchain_zip), ("linux", "toolchain-linux64.zip", a.toolchain_linux_zip)): + tc[plat] = os.path.join(out, name) + if reuse: + if os.path.abspath(reuse) != os.path.abspath(tc[plat]): shutil.copyfile(reuse, tc[plat]) + else: + make_toolchain(plat, tc[plat]) + print("toolchain (%s):" % plat, tc[plat], os.path.getsize(tc[plat]) // (1 << 20), "MB") # what the launcher reads (a flat object: it is read without a JSON library) man = { @@ -112,7 +140,8 @@ man = { "version": a.version, "levels": a.levels, "kit_file": os.path.basename(kit), "kit_size": os.path.getsize(kit), "kit_sha256": sha256(kit), - "toolchain_file": os.path.basename(tcz), "toolchain_size": os.path.getsize(tcz), "toolchain_sha256": sha256(tcz), + "toolchain_file": os.path.basename(tc["windows"]), "toolchain_size": os.path.getsize(tc["windows"]), "toolchain_sha256": sha256(tc["windows"]), + "toolchain_linux_file": os.path.basename(tc["linux"]), "toolchain_linux_size": os.path.getsize(tc["linux"]), "toolchain_linux_sha256": sha256(tc["linux"]), "toolchain_version": tcj["version"], } with open(os.path.join(out, "%s.json" % game), "w") as f: json.dump(man, f, indent=1) diff --git a/release/rac1/rac1-build.zip b/release/rac1/rac1-build.zip index 8fdda1e..c61cac9 100644 Binary files a/release/rac1/rac1-build.zip and b/release/rac1/rac1-build.zip differ diff --git a/release/rac1/rac1.json b/release/rac1/rac1.json index 377db23..0a5228d 100644 --- a/release/rac1/rac1.json +++ b/release/rac1/rac1.json @@ -1,12 +1,15 @@ { "game": "rac1", - "version": "2026.10.14.1", + "version": "2026.10.14.2", "levels": "0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18", "kit_file": "rac1-build.zip", - "kit_size": 5125429, - "kit_sha256": "b6919dd6d31d1ae8fac65c2859eaa6c084e98b97af36e3a305b4253da40ba5c5", + "kit_size": 5131215, + "kit_sha256": "440ee0db6288b4f10ac13806ca24a352b7331cebf06e91335f5c7e469dd26ba8", "toolchain_file": "toolchain-win64.zip", "toolchain_size": 84006129, "toolchain_sha256": "2d56894833ca4066faf5ac008d3fa06aa40e2bb1d1fcb8f5bd26f28a70382b63", + "toolchain_linux_file": "toolchain-linux64.zip", + "toolchain_linux_size": 83952276, + "toolchain_linux_sha256": "b6502c5c606b28cfda4587b6818084e8f8304a52f7d906408b70f664602e50bf", "toolchain_version": "py3.12.7-zig0.16.0-rabbitizer1.16.2" } \ No newline at end of file diff --git a/release/rac1/toolchain-linux64.zip b/release/rac1/toolchain-linux64.zip new file mode 100644 index 0000000..da6b952 Binary files /dev/null and b/release/rac1/toolchain-linux64.zip differ diff --git a/release/toolchain.json b/release/toolchain.json index 988b6ce..b34c323 100644 --- a/release/toolchain.json +++ b/release/toolchain.json @@ -1,7 +1,10 @@ { - "comment": "What toolchain-win64.zip is made of (release/make_release.py fetches these when it has no toolchain zip to reuse). All are redistributable: CPython (PSF licence), zig (MIT), rabbitizer (MIT).", + "comment": "What toolchain-win64.zip and toolchain-linux64.zip are made of (release/make_release.py fetches these when it has no toolchain zip to reuse). All are redistributable: CPython (PSF licence), zig (MIT), rabbitizer (MIT).", "version": "py3.12.7-zig0.16.0-rabbitizer1.16.2", - "python": "https://github.com/astral-sh/python-build-standalone/releases/download/20241016/cpython-3.12.7+20241016-x86_64-pc-windows-msvc-install_only.tar.gz", + "python": { + "windows": "https://github.com/astral-sh/python-build-standalone/releases/download/20241016/cpython-3.12.7+20241016-x86_64-pc-windows-msvc-install_only.tar.gz", + "linux": "https://github.com/astral-sh/python-build-standalone/releases/download/20241016/cpython-3.12.7+20241016-x86_64-unknown-linux-gnu-install_only.tar.gz" + }, "ziglang_wheel": "ziglang==0.16.0", "rabbitizer_wheel": "rabbitizer==1.16.2" }