Files

T

ReaperOfVeriod 027373e65b Add the disc pipeline: identify, unpack and verify

Phase 1 of the roadmap. The disc is now turned into verified asset files, and
the result is checked against a committed hash table rather than against
whatever happens to be on disk.

  gc-extract identify disc.iso        SCUS_972.68 v1.01 NTSC, ELF SHA-1 matches
  gc-extract unpack   disc.iso out/   5135 files, 4,500,106,960 bytes, 10.8s
  gc-extract verify   out/ table.tsv  OK: 5135 files match

All 92 G/*.WAD containers parse. MPEG1 and MPEG2 are raw video and pass
through whole; the other 90 are indexed containers whose extents are extracted
individually.

Formats recovered and written up in docs/formats/:

- iso9660.rs. R&C2 lists its assets in the filesystem, unlike R&C1, so a
  directory walk is the primary asset index. Three details that all fail
  quietly rather than loudly: directory records store every 32-bit field twice
  (little-endian then big-endian), the CD001 signature is at sector offset 1
  rather than 0x8001, and the directory flag at record offset 25 is
  authoritative where the identifier's trailing slash is not.

- global_wad.rs. Container headers hold a table of extents. All 92 indexed
  containers use {offset in sectors, size in bytes}. Measured, not assumed: the
  sector-count reading places every entry of all 27 LEVEL<n> and all 27
  AUDIO<n> containers out of range, while the byte reading places all of them
  correctly. SCENE<n> is the one exception, a flat sector list whose extents
  are sized by the next value.

- wad.rs. Insomniac's LZ77 stream: four flag-selected packet classes, with 0-3
  trailing literal bytes per packet carried in unused low bits. This is not a
  bit stream, which is why it resists the usual LZ77 reading.

- buf.rs. A bounds-checked reader, so untrusted 2003 disc data produces typed
  errors rather than panics.

gc-extract carries the three stages and a SHA-1 implementation with FIPS 180-4
test vectors, so identification works without pulling in a crypto dependency.

Verification: 54 unit tests, plus four integration tests that run only when
GC_DISC names a disc. Those four check identification against the recorded
values, cross-check the Rust identifier against tools/identify-disc.py, parse
all 92 containers, and confirm a full unpack matches the committed table. The
committed table is 316 KB of sizes and SHA-1s for all 5,135 files: hashes only,
no game content, so it is safe to version and it turns "matches retail" into
something a test can assert.

docs/plan/disc-pipeline.md records two wrong turns, because both are easy to
repeat and both failed by producing plausible wrong output rather than an
error: the container table was read backwards, and the LZ77 stream was modelled
as a bit stream. The first survived because it happened to be correct on the
one container used for manual checking.

2026-10-09 16:08:30 +02:00

8.2 KiB

Raw Blame History

Roadmap

Status as of 2026-10-09.

A from-scratch native Rust reimplementation of Ratchet & Clank: Going Commando (PS2). Each phase ends in a gate — something checkable, not a feeling that it is going well. Phases are ordered so that the riskiest unknowns are retired early, while they are still cheap to retire.

Where we are

Target disc identified and verified (SCUS_972.68 v1.01, NTSC 60 Hz)
Workspace builds (cargo check --workspace)
Repository hygiene in place (pre-commit guard, .gitignore, licence record)
Phase 1 complete. The disc pipeline runs end to end and is verified against a committed hash table: 5,135 files, 4,500,106,960 bytes, every one of the 92 asset containers parsing.
Phase 2 next: the asset format parsers.

The disc being the exact build the reference projects pin is the single biggest favourable fact so far: no disc conversion, no regional timing work, and no re-derivation of the reference baseline.

Phase 0 — Foundation ✅

Done.

Disc identified: SCUS_972.68 v1.01, USA Black Label, NTSC 60 Hz. Boot ELF SHA-1 bacda860cbd6723142e86db56f197e47e728970f, entry 0x00131AE8 — the entry point independently matches the value recorded by rac2-gc-decomp, which cross-confirms the identification.
tools/identify-disc.py reproduces the identification against any disc image and distinguishes a truncated dump from a non-PS2 image.
Five-crate workspace scaffolding, Bevy 0.19.1 pinned, builds clean.
.githooks/pre-commit blocks game data. Verified against seven cases: .iso, renamed disc image, WAD container, ADX container, extracted/, a 60 MiB file, and legitimate source.
AGENTS.md and PROVENANCE.md record conventions and licences.

Open from this phase

alsa-lib-devel is not installed, so Bevy's host audio backend is disabled in Cargo.toml. Audio is the last phase; revisit then.
Bevy feature set is deliberately minimal. Re-add features as needed rather than switching default-features back on wholesale.

Phase 1 — Disc pipeline ✅

Done. Gate met: every extracted file matches the committed hash table.

iso9660 reader (R&C2 lists assets as real files, unlike R&C1)
G/*.WAD container header parsing — 92 containers, three table shapes
WAD LZ77 decompression (wad.rs, four packet classes)
Unpack every container, bounds-checking each extent
SHA-1 table for all 5,135 files (hashes only, never content)
verify stage, plus integration tests against a real disc
Format specs written: disc-layout.md, global-wad.md, wad.md
Work record with the negative results: plan/disc-pipeline.md

gc-extract identify disc.iso        -> SCUS_972.68 v1.01 NTSC, ELF SHA-1 matches
gc-extract unpack   disc.iso out/   -> 5135 files, 4,500,106,960 bytes, 10.8s
gc-extract verify   out/ table.tsv -> OK: 5135 files match

Not done, deliberately: the table of contents at sector 1001 is documented but not read. Enumerating the ISO filesystem is simpler and sufficient; the ToC matters when per-level overlay loading is understood in Phase 3.

Two wrong turns are recorded in plan/disc-pipeline.md, because both would be easy to repeat: the container pair table read backwards, and the LZ77 stream mistaken for a bit stream. Both produced plausible-looking wrong output before failing, which is the failure mode worth guarding against.

Phase 2 — Format parsers

Goal: every asset type parses into plain Rust values, tested against real disc data.

Phase 1 hands over the extents; this phase works out what each one contains. global_wad.md records which pair in each container corresponds to which asset, and that table starts empty.

tfrag — terrain: LOD chains, strips, morph targets
tie — instanced environment geometry, fat vertices
shrub — foliage and decorative meshes
moby — dynamic actors; moby_anim — skeletal animation
collision — cells, packed vertices, faces
occlusion — the PVS grid
sky, volumes, textures (CSM1 CLUT + PS2 alpha)
level_overlay — the seven-section parser and relocation from Phase 3

Gate: every parser has a golden test against bytes from the real disc, and reports mismatched header counts as failures rather than warnings.

Discipline: bound every read. Typed errors, no panics, no reads past the buffer.

Phase 3 — Overlay system

Goal: make 27 level overlays tractable.

Every overlay carries the same seven sections (.lit, .bss, .data, lvl.vtbl, lvl.camvtbl, lvl.sndvtbl, .text) and the same engine code, linked at different addresses. ReRAC solves this by masking relocations — j/ jal targets, lui immediates of main-RAM addresses, the %lo that completes them, and $gp-relative operands — then matching the masked words to find a reference function's copy on another overlay.

Port the masking and code-identity logic
Relocation::func and Relocation::data
$gp for this game (0x001AEFF0, against R&C1's 0x166c00)
Extend the level table from 19 to 27

Gate: every level-01 function's relocated copy is found on all 27 overlays; cross-checked against an independent cluster hash.

Why this is high-value: it turns "27 overlays of unknown code" into a table lookup, and it is the main thing inherited from ReRAC.

Phase 4 — Engine core

Goal: draw a frame.

Graphics Synthesizer in wgpu, with the GS's own rules: its round(log2(z/n) + K) mip selection, blend and alpha-test behaviour, alpha 0x80 meaning 1.0, CLAMP wrap
VU0/VU1 skinning and lighting on the PS2 float model
LOD/morph/cull per TfragProc and TieProc
Occlusion culling, fog, shadows
GIF/VIF packet stream interpretation

Gate: CPU and GPU paths agree on every vertex of a reference level — the same identity check ReRAC runs.

Risk: VU1 microcode is binary in the game's .vutext section, so it is reverse engineered rather than parsed. This is the hardest single item in the project and the main threat to the timeline. Note that ReRAC's own moby VU1 analysis trips their disassembler on two opcodes.

Phase 5 — Veldin vertical slice

Goal: one planet, playable start to finish.

Ratchet's controller on a spherical planet
Chargeboots: hold/release timing, chained jumps, dash
Wrench and the camera
Collision and the HUD

Gate: Veldin is completable and visually matches PCSX2 frame-for-frame under replayed input.

This is the go/no-go for the project. If Veldin works, the other 26 levels are content. If it does not, the reason is discovered while it is still cheap. Spherical gravity is the thing most likely to break, because it invalidates R&C1's movement and camera assumptions wholesale.

Phase 6 — Remaining overlays and the shell

Per-level moby class dispatch via lvl.vtbl
Remaining 26 levels
Planets, travel, the Gadgetron vendor, RPG upgrades
Weapon modification bitmasks (lock-on, shock, acid, …)
Minigames and challenges
The 25-state memory-card FSM and its UI
IFF chunk saves, including import of a R&C1 save to unlock legacy weapons

Phase 7 — Audio and video

989snd v2 audio decode and playback (not CRI ADX — a different codec)
SPU2 mixing, reverb zones, level sounds
IPU/MPEG-2 video for the title screen and cutscenes

Phase 8 — Fidelity pass

Differential testing against PCSX2 across every level:

Recorded-input frame comparison, per planet
Geometry, fog, blend, alpha, texture sampling
Audio sync and 60 Hz timing
Behavioural comparison: traversal routes, boss timing, RNG

Principles

A gate is pass/fail, not a percentage. "Looks fine" is not progress.
Negative results are written down. A ruled-out approach saves the next attempt as much as a working one.
The float model is not negotiable. Approximate is not the same as faithful, and the difference is visible in how the game feels.
Fidelity over tidiness. Where the original does something that looks like a bug, reproduce it and say why in a comment. See AGENTS.md.

8.2 KiB Raw Blame History