Files
Going-Commando/docs/roadmap.md
T
ReaperOfVeriod 572cd1898d Record a defect packet slicing found: the tie block terminator is early
Range measured: every tie LOD block on the disc, 2,177 blocks.

The last packet's end is known two ways. Its length comes from its record's
packet_quadwords, and every other record's length is confirmed by the next
record's pointer -- so only the last packet of each table is unconfirmed.
Comparing its computed end against where parse_tie_block stopped: the last
packet's computed end lands 2 to 16 bytes past the located 0xf6, on all 2,150
blocks measured.

Two readings fit. Either the terminator was located too early -- inside the
last packet's data -- or the last packet's length over-predicts by a small
amount. The first is likelier: a length error on the last record of every table
would have to be a coincidence. Neither is settled here, and the test asserts
the measurement rather than the reading.

Why the existing test missed it: "2,177 of 2,177 blocks end on 0xf6" is
satisfied by a false terminator exactly as well as a true one -- it checks
that the byte found IS a 0xf6, not that it is the block's end. A test that
confirms the shape it walked to will pass on the wrong byte.

The cost: the block extent is short by that much, so payload and padding
under-report and anything computed from them is correspondingly short. The
block starts are unaffected and RAM-confirmed.

Recorded as a limitation with the measurement attached, not fixed. A terminator
packet data cannot imitate is its own task, and guessing one would be the
plausible-sounding alternative this project has been bitten by. A packets()
accessor was tried and removed for the same reason: with the extent short it
silently yields the wrong answer, and an API that is quietly wrong is worse
than no API.
2026-10-13 17:40:31 +02:00

40 KiB

Roadmap

Status as of 2026-10-11.

A from-scratch native Rust reimplementation of Ratchet & Clank: Going Commando (PS2). Each phase ends in a gate — something checkable, not a feeling that it is going well. Phases are ordered so that the riskiest unknowns are retired early, while they are still cheap to retire.

Where we are

  • Target disc identified and verified (SCUS_972.68 v1.01, NTSC 60 Hz)
  • Workspace builds (cargo check --workspace)
  • Repository hygiene in place (pre-commit guard, .gitignore, licence record)
  • Phase 1 complete. The disc pipeline runs end to end and is verified against a committed hash table, every one of the 92 asset containers parsing. The pipeline now also writes each level's chunk descriptors where they exist (3 of 27 levels).
  • Phase 2 in progress. Level containers are parsed through their typed header, and the data extent is parsed as a container of byte ranges. The geometry decompresses: level 0 yields 16,969,296 bytes of level content from a 9.7 MB stream.
  • Parsers with real-disc tests, per asset type. level_wad, data, core_index class tables, tfrag (format, topology, textures, LOD, normals, colour), occlusion, sky, moby, collision framing and header, shrub (blob, packets, palette, billboard), the post-shrub chunks container, sound_bank framing and level_overlay. 238 tests green against the real disc. What is open is naming, not locating -- see the Phase 2 list for the four types that still have no decoded fields: tie vertices, collision cells, SBlk records, volumes and moby_anim.
  • Lighting verified against the running game. PCSX2 2.8.2 with savestates for level 0 and level 1. The tfrag lighting pass is byte-exact on both against the game's own RAM. Setup and method in plan/pcsx2-verification.md.
  • First drawable frame. gc-engine loads a level, applies LOD selection and morphing for the camera, groups triangles by texture and draws them under a Bevy fly-camera. Level 0 renders as 26,758 textured triangles across 92 textures with correct UVs. Set GC_ENGINE_SHOT=<path.ppm> to capture a frame and exit.
  • The origin bug this exposed. Positions are i16 offsets from a per-patch origin, so taking the wrong STROW collapsed the whole level to a +-32 unit blob at the world origin while every parser test still passed: the vertex count matched, the triangle count matched, only the place was wrong. Fixed by taking the STROW immediately preceding the first position unpack rather than the last one.
  • Per-vertex colour. The tfrag's RGBA array now modulates the texture, drawn unlit so the frame shows real albedo rather than a faked lighting model. Verified against the stored values (level 0 patch 0 runs (25,26,26) to (108,144,141)), which are genuinely dark.
  • Tfrag normals. The 256-entry sine table is decoded from the level overlay and normals are built per vertex and handed to the renderer. The overlay has no ELF header, so the table cannot be reached through a program header, and its offset differs per level (level 0 0xb650c, level 5 0xcca30); it is located by matching the whole 256-entry curve. Normals are negated as stored and get the same axis swap as positions. The pipeline now also writes overlay.bin.
  • The PS2 float model. gc-engine::ps2 is the EE's FPU on raw bits: no denormals, no NaN or infinity, truncation toward zero, the adder's operand alignment, and rsqrt as sqrt-then-divide. MAX reads back in Rust as NaN, which is the sharpest available statement of why this is not f32.
  • The LightTfrags pass is written and unit-tested against the PS2 model: PEXT5 base colour, select set blending, re-normalised blended directions, and base + A*max(dA, dA*wA) + B*max(dB, dB*wB).
  • The level light bank parses and level 0 is fully lit. gameplay is a WAD stream; the extractor had been writing it compressed, which is why the bank framing found garbage. All 80,274 vertices of level 0 now go through LightTfrags. The renderer still counts and reports how many vertices it lit, because a silent fallback to stored colours would look plausible and be wrong.
  • The shrub region is bounded, and with it the whole lump. Shrub's end was the last unlocated region bound: it is the start of the post-class chunk container, found by a marker-paired signature in core_data (the index carries no successor offset). The container closes shrub on the 24 shrub levels and tie on the three without. All 358 shrub class bodies are now bounded and are whole 8-byte records, so shrub measurements are trustworthy for the first time -- the "3.5M records" figure from the unbounded final span is replaced by a measured 9,600.
  • Lighting magnitudes are verified by the PCSX2 comparison loop. gc-trace compare-tfrag-light runs the pass over level 0's disc data and diffs it against the game's own lit colours in EE RAM (live over PINE, or from a savestate): all 162,400 colour bytes match, across 833/833 headers relocated and the light bank byte-identical. Two bugs it found: the normal's z was not negated (dark but plausible, 95.88% agreement), and records with no light set drew transparent black instead of the base colour. See docs/plan/pcsx2-verification.md.
  • Then the GS register set and the guard-band clip test.

The disc being the exact build the reference projects pin is the single biggest favourable fact so far: no disc conversion, no regional timing work, and no re-derivation of the reference baseline.


Phase 0 — Foundation ✅

Done.

  • Disc identified: SCUS_972.68 v1.01, USA Black Label, NTSC 60 Hz. Boot ELF SHA-1 bacda860cbd6723142e86db56f197e47e728970f, entry 0x00131AE8 — the entry point independently matches the value recorded by rac2-gc-decomp, which cross-confirms the identification.
  • tools/identify-disc.py reproduces the identification against any disc image and distinguishes a truncated dump from a non-PS2 image.
  • Five-crate workspace scaffolding, Bevy 0.19.1 pinned, builds clean.
  • .githooks/pre-commit blocks game data. Verified against seven cases: .iso, renamed disc image, WAD container, ADX container, extracted/, a 60 MiB file, and legitimate source.
  • AGENTS.md and PROVENANCE.md record conventions and licences.

Open from this phase

  • alsa-lib-devel is not installed, so Bevy's host audio backend is disabled in Cargo.toml. Audio is the last phase; revisit then.
  • Bevy feature set is deliberately minimal. Re-add features as needed rather than switching default-features back on wholesale.

Phase 1 — Disc pipeline ✅

Done. Gate met: every extracted file matches the committed hash table.

  • iso9660 reader (R&C2 lists assets as real files, unlike R&C1)
  • G/*.WAD container header parsing — 92 containers, three table shapes
  • WAD LZ77 decompression (wad.rs, four packet classes)
  • Unpack every container, bounds-checking each extent
  • SHA-1 table for all 5,135 files (hashes only, never content)
  • verify stage, plus integration tests against a real disc
  • Format specs written: disc-layout.md, global-wad.md, wad.md
  • Work record with the negative results: plan/disc-pipeline.md
gc-extract identify disc.iso        -> SCUS_972.68 v1.01 NTSC, ELF SHA-1 matches
gc-extract unpack   disc.iso out/   -> 5135 files, 4,500,106,960 bytes, 10.8s
gc-extract verify   out/ table.tsv -> OK: 5135 files match

Not done, deliberately: the table of contents at sector 1001 is documented but not read. Enumerating the ISO filesystem is simpler and sufficient; the ToC matters when per-level overlay loading is understood in Phase 3.

Two wrong turns are recorded in plan/disc-pipeline.md, because both would be easy to repeat: the container pair table read backwards, and the LZ77 stream mistaken for a bit stream. Both produced plausible-looking wrong output before failing, which is the failure mode worth guarding against.


Phase 2 — Format parsers

Goal: every asset type parses into plain Rust values, tested against real disc data.

Phase 1 hands over the extents; this phase works out what each one contains.

Done so far:

  • level_wad.rs — the 0x60-byte header on all 27 level containers, naming four extents plus three chunk descriptors. Verified: extents tile each container in file order (27 of 27).
  • Two facts the format documentation does not state, both measured here: sizes in this header are sector counts where the generic G/ table uses byte counts, and LEVEL21.WAD reports level id 30, so container order is not level identity.

In progress:

  • data — the extent header locates core_index and core_data; the geometry decompresses and is verified on level 0.

  • core_index class tables — moby, tie and shrub. They were ruled out for want of an entry size, which is knowable exactly: each table ends precisely where the next begins, and that chain closes on 27 of 27 levels. 32 bytes per moby/tie entry, 48 per shrub. Across the disc: 5,796 moby, 1,864 tie, 358 shrub classes. Identifying tfrag by its own structure is what led here.

  • occlusion and sky offsets located and their first bytes read.

  • Region layout of core_data established: tfrag, occlusion, sky, collision and textures_base are offsets, strictly increasing on all 27 levels. Terrain is [0, occlusion) and carries inline texels.

  • Region layout closed: the chunk container's marker-paired signature ends the last class region, and the constant trailer ends the container. Every byte of level-specific core_data now belongs to a named region.

  • tfrag vertex format: recovered. The tfrags block header gives a table offset (0x40) and a patch count; each patch is a 0x40-byte header whose data offset is relative to the table, not to the block. Positions are V3_16 payloads inside a VIF1 command list, split across three lists per patch (common, LOD 0+1, LOD 0). The whole vector format is encoded in the command byte and every element size follows from it. Verified on all 27 levels: 18,102 patches, 821,439 vertices, each patch yielding exactly its own vert_count. See plan/level-assets.md and gc-formats::tfrag.

  • tfrag topology: recovered. Vertex-info (UVs plus LOD parenting), the strip command lists and the index arrays, classified by unpack format and destination VU address. Strip indices address the vertex-info array rather than positions. Every patch's strips expand to exactly its own tri_count: 18,102 patches, 809,879 triangles, all indices in range. See plan/level-assets.md and gc-formats::tfrag.

  • tfrag texture bindings and per-vertex data. The ad-gifs decode to a texture index, wrap modes, minification filter and LOD bias; the level index gained a texture table (level_core.rs) for those indices to land in. Per-vertex RGBA and light/normal records recovered from their fixed offsets. All 27 levels: 47,950 ad-gifs, every index in range, 90% of the level texture tables referenced.

  • Level textures decode. 8-bit indexed texels plus a 256-entry CSM1 CLUT from gs_ram, honouring the PS2 CLUT block swap and the 0x80 alpha scale. The pipeline now writes gs_ram.bin, which it had been skipping. All 1,595 tfrag textures on all 27 levels decode; 96.8% fully opaque and 96.3% with real detail. Verified visually via the new gc-extract dump-texture, because a wrong lookup still yields a well-formed image of the wrong thing.

  • tfrag LOD selection. Switch distances scale as [6L, 4L, 2L] from the block header's float; six draw modes result from testing the bounding sphere against trunc(D * 1024), so a patch straddling a threshold takes the coarser mode and distance never moves back to finer detail. Morph weights run 0..1 across each tier's band, and collapse replaces an entry with its first parent rather than blending it.

  • tfrag parent indices. The four V4_8 arrays are classified by VU address, but the addresses collide, and each V4_8 element holds four indices so an array is count * 4 bytes rather than count. With unk_06/unk_0a suppressing absent tiers, all 370,349 LOD entries across the 27 levels resolve: 29,043 LOD-01 and 341,306 LOD-0, zero failures. See plan/level-assets.md for the two corrections.

  • gameplay — it is a WAD stream, exactly like core_data; the extractor was writing it compressed. Decompressed, level 0 is 1,872,256 bytes and holds 5 directional light sets. The earlier "does not decode with the WAD LZ77 decoder, bit order and header length ruled out" note was wrong about the cause: the decoder was right and the input was not what it looked like. Corrected in plan/level-assets.md.

  • [~] sound_bank — the SBlk head is now characterised: "SBlk", 1, 4, "DAW\0", 0 constant on 27/27, and +0x14 is the only varying word, with its low 16 bits always zero (0x0092..=0x0103). It tracks body length at ~100 bytes/unit and is not a count -- ratio 91.2..110.8, and no stride makes it exact. The absence of a stride is asserted too. Container framing parsed (SoundBankHeader::parse, parse_sound_bank); SBlk and DAW bounds verified on 27/27. The record fields inside SBlk are not decoded, and DAW shows no detectable stride, consistent with ADPCM. See the note under level_overlay below for what the game's own strings say about loading.

  • data — not geometry. It is the level's file descriptor: its first table carries the exact byte lengths of overlay.bin, core_index.bin and gs_ram.bin at fixed u32 indices 1, 3 and 5 on every level checked. What follows the descriptor is still unknown.

  • tfrag — complete on the disc side. Vertex format, topology, strips (TfragStrip), texture bindings, per-vertex colour and normals, LOD selection, parent indices, morphing and collapse. 36 format tests, 9 texture tests, 23 engine tests. The morphing is a blend toward the parent vertices' midpoint, not toward separate morph targets — there are no morph targets in this format, which is what the old "morph targets unrecovered" note wrongly implied.

  • The tfrag table record has exactly two relocated pointer fields, at +0x10 and +0x38, verified against PCSX2 across all 833 records: the loader adds the table's RAM address (0x848040 = block base plus table_offset, matching TfragHeader::data's own "relative to the table") to +0x10; every other byte of every record is verbatim, and 1,665 fields differ where 1,666 were possible. They are the existing TfragHeader::data and ::cube_ofs -- what RAM adds is that both are pointers, not plain offsets. All 833 patches parse, carrying 43,613 vertex-info entries; 499 of 833 have at least one morphing entry.

  • Behind those pointers, every patch's bytes are rewritten -- and it is not morphing. Across all 833 patches: the 334 with no morphing entries differ by a median 7.80%, the 499 with morphing entries by 8.12%, and 0 of 833 are byte-identical to disc. Morphing does not predict which bytes change. Nor is it a relocation: the differences fall in short structured runs with no common u32 addend. The loader regenerates each patch's VU packet stream for the LOD tier it selected, which happens to every patch regardless of morphing.

  • What tfrag is not yet verified is that the morph matches resolve_vertex's formula. Reading a weight back out of RAM does not work -- the positions live inside regenerated VU packets, so there is no stable per-vertex correspondence. The LOD switch distances, morph weight and collapse edge remain derived from the data (tiers classified from the four V4_8 parent arrays, weight running zero to one across each band); verifying them needs the camera distance at the savestate so the weight can be computed and compared against the rendered result.

  • The tie class header's +0x0c word is the three LOD record counts — LOD 0 at +0x0c, LOD 1 at +0x0d, LOD 2 at +0x0e — verified on 1,864 of 1,864 classes by reading the counts independently from the blocks' own record tables. This was filed as "R&C1's normals offset, read past, not decoded", which was the right action on the wrong map: R&C1 carries the same quadruple at +0x20 and it is not an offset on R&C2. The fourth byte is unidentified and is not the shader total (disagrees on 1,827 of 1,864).

  • [~] The tie block's located 0xf6 terminator is inside the last packet's data -- the last packet's computed end lands 2 to 16 bytes past it on all 2,150 blocks measured. So the block's extent is short by that much (the 0xf6 test could not see it: a false terminator passes that test). The block starts are unaffected and RAM-confirmed. The packet lengths themselves are verified for every record but the last.

  • tie — fully chained. resolve_tie_class walks both indirections: the class table is in core_index.bin, and the entry's offset_in_asset_wad addresses core_data.bin (not core_data.wad). All 1,864 tie classes resolve with o_class agreeing on every one. A packets word of zero means no block at that LOD -- 2,177 real blocks, 1,551 absent slots.

  • Tie blocks are self-delimiting: parse_tie_block walks to the 0xf6 terminator that every real block carries (2,177 of 2,177), followed by zero padding. This is what makes blocks parseable at all -- packets[1] bounds one on only 461 of 1,864 classes.

  • The tie 16-byte record is R&C1's TiePacketHeader. Identified not by resemblance but by R&C1's own rule, bfc_distance == 5 * shader_count, holding on 15,219 of 15,219 records across all 2,177 blocks. A block is [16-byte header][record table][vertex payload], and the header word at +0x00 is the byte offset from the block start to the payload. The earlier "variable-length, not 16-byte records" reading was wrong: the word is not an element count, which is why dividing the payload by it gave 0 of 922.

  • The tie vertex element is 16 bytes, measured over 95,612 vertices on levels 0-2 and settled by comparison: a 12-bit-fixed texcoord hit rate of 82.5% at 16 bytes against 47.8% at 8 and 44.8% at 12.

  • i16[0..2] are the vertex positions: LOD1/LOD2 of one class share 96.1% of their position triples, while LOD1 of A against LOD2 of B shares 0.4% -- a factor of 240, asserted including the ratio. Axis order and scale unnamed.

  • i16[3] is a byte-sized index, not a coordinate: 201 distinct values in 0..=254 inside the ordinary layout, whose high byte is the layout tag (93.4% of elements are 0x00, 5.6% are 0x03, 146 selector values in all). Asserted.

  • The tie texcoord field is located: i16[4]/i16[5] of the 16-byte element, scoring 73.6% for 12-bit fixedness over 36,501 level-0 vertices, against 1.1% for (0,1) and 4.8% for (2,3). The (6,7) pair scores higher at 76.6% and is not a texcoord -- it is the near-constant 4096, 0 pair, which is trivially two multiples of 4,096.

  • i16[3] selects the texcoord encoding, and it is specifically the high byte. Grouped across 95,612 vertices on levels 0-2: high byte 0x00 covers 89,341 vertices (93.4%) at 92% 12-bit fixed, while every other high byte scores 0%. The split is on the byte, not the value -- i16[3] of 1 and of 771 differ only in the high byte and behave completely differently. Within the all-zero case, i16[3] == 0 exactly (5,851) is a third distribution at 62%. Three cases: 0x00/0x00, 0x00/1..255, and non-zero high byte.

  • The tie packet's length follows from its record: vert_ofs + vert_size + scissor_size quadwords, on 12,927 of 12,927 record pairs whose strip_count is zero and 0 of 115 whose it is not -- an exact partition, so the formula is safe when strip_count == 0 and knows nothing otherwise. This unblocks slicing the payload per packet. It also corrects this file's own retraction of the same formula: the "0 of 13,042" re-measurement had swapped strip_count for scissor_ofs and blamed the block set.

  • [~] The remaining tie vertex fields are not named, and two readings are now excluded. i16[6]/i16[7] are constant across a whole draw packet in 43%/34% of records, so they are usually per-packet rather than per-vertex. i16[3] is per-vertex (50 distinct values per packet) but not a skin or joint index: vertices sharing it are 0.95x farther apart than vertices differing in it, against a control that halves. It is also unshared across LODs. Recorded as not understood rather than guessed. i16 range with 10,803-13,421 distinct values each. i16[0..2] are most likely position, but that is not established.

  • [~] shrub — class table done (48-byte entries, not 32; reading shrub at 32 still gives 231 sane-looking entries of 358, which is the hazard). Geometry header parsed and checked field by field on all 358 spans: +0x24 equals the entry's own o_class, +0x2c is an in-span data offset, +0x16/+0x18 are zero, and +0x1c names a billboard on exactly the 25 classes whose core-index entry carries billboard texture fields. Shrub does not use moby's part-table format. The body is a 24-entry palette of unit-length s16 normals (squared length within 65,535 of 32767² on all 8,592 entries — a random triple would miss by a billion) plus zero padding records; the billboard record carries K, MMIN and the TEX0/MIPTBP addresses as R&C1's does. What remains open: the three floats in the billboard record, size, mode_bits, count, and where the mesh geometry lives — R&C1 puts the mesh packets in this blob and R&C2's blob holds only the palette.

  • The tie LOD block is a table of 16-byte records. The header's word at +0x00 is the byte offset from the block start to the vertex payload -- the record table's length -- not a count, which is why the payload never divides by it. Each record is a leading u32 pointer relative to the table start plus twelve verbatim field bytes, relocated at load to table + word.

  • The record is R&C1's TiePacketHeader. R&C1's own rule bfc_distance == 5 * shader_count holds on all 15,219 records of all 2,177 blocks on all 27 levels. R&C1's rule for the colour offset holds on none of them, so the games use that field differently.

  • The tie relocation verifies on two levels. gc-trace verify-tie-records checks it against PCSX2's RAM: 87 of 87 blocks on level 0 and 104 of 104 on level 1, zero mismatches, which is what makes the record layout a finding rather than a reading of one block.

  • The shrub mesh geometry is in the class blob. A packet table at +0x40 names count packets between the table and the palette, and they tile on 358 of 358 spans (2,239 packets), which makes count the packet count. Confirmed against the running game, which relocates the table's offsets to absolute addresses and data_offset likewise.

  • moby — per-class model geometry. A class span opens with a u32 pointing at a part table of 16-byte records; parts tile exactly, with vertex_offset + 16 * vertex_count == next_part holding for 43,467 of 43,467 parts across every class on all 27 levels. That identity is what makes the record size fall out of the data rather than being assumed. 1,270,802 vertices over 4,560 class spans; 116 spans carry no geometry. Vertices are 16-byte VIF-packed half-float quads, handed back raw.

  • moby_anim — still open, and the obvious reading is ruled out. An instance table would have to scale with the level's moby class count, but per-chunk-type record counts are level-independent -- the same types report 90, 77, 50 and 47 on every level tried, and the largest type totals 2,430 = 90 x 27. Nothing in the chunk set is keyed by moby class. The types also pair up (two types both reporting 90 on every level, another pair both 77), which looks like two views of one fixed table rather than two independent tables.

  • collision — container framing and header fields. A 0x40-byte header self-describes: it carries its own size and the payload size, and header + payload stays inside core_data on all 27 levels. 41,481,520 payload bytes across 19 levels; 8 declare zero length. Header bytes +8 reads zero on 22 levels and a large value on 7/9/12/13/14; the payload's two leading u16 are non-zero on all 19 non-empty levels (1-74 and 13-112) and are reported as CollisionHeader::first/::second.

  • The collision cell layout is not decoded. The payload does not walk as fixed-stride cells. On level 4 a table starts at payload +5932 and runs 17 entries -- deltas rise then fall -- against a 75-cell grid, with no arithmetic relating the two. A 64-byte RAM stride looked promising and is disproved -- the longest such run is 27 records of identity matrices, one per level.

  • The collision payload is essentially verbatim in RAM, not reshaped. Level 0's sits at RAM 0xac6800 for a disc address of 0x27e800: 903,212 of 903,248 bytes identical, first difference at +0xdc82c, with 6 bytes differing in the final 36. Asserted by a savestate-gated test that compares the whole payload. This overturns the earlier claim that the grid dimensions are absent from RAM: level 0's 11 x 33 are the first two bytes of that block. The earlier negative came from searching for the dimensions as a u32 pair when they are two u16s -- a bad pattern, not a property of the data. The payload contains ascending u32 offset tables from +1024 onward, and disc-side decoding is no longer blocked on reshaping. See plan/pcsx2-verification.md.

  • occlusion — the PVS grid. A z/y/x tree of u16 node offsets over 4x4x4-unit cells, populated cells naming 128-byte masks. 412,521 cells over 140,939 masks across the 24 levels that have one; the mask array ends within 48 bytes of the block boundary on every level. 3 carry no occlusion data, and an offset of 0 means absent rather than "at the start" — the latter parses as a grid whose mask array happens to fit, which is how it nearly went wrong.

  • sky — 149 sub-objects across all 27 levels, each with a count, a four-float bounding sphere and texture dimensions. The offset table at +0x20 is self-limiting: entries run while strictly inside the region. The word at +0x0c is not a count -- it disagrees with the table on most levels and reads 196,617 on level 2 against 4 sub-objects.

  • The overlay's load base is field_00 - 16 = 1,735,792, derived from the overlay's own profiler table (whose absolute addresses point back at its names, on all 27 levels) and confirmed in RAM: all four profiler names land where the base predicts. Any overlay offset can now be carried to a RAM address. The overlay is relocated, not copied: only 37.67% is byte-identical at the base, so tables inside it move.

  • [~] volumes — still not identified, but a candidate exists. Every level's gameplay.bin carries a table of 144-byte float records with world positions at +0x0c, preceded by an all-zero record (levels 0-9: 23 to 612 records). That is the shape of a volume trigger region and it is not tied to anything else in the file -- but camera bounds, spawn points and path nodes would look identical, and there is no engine-side Volume string to discriminate with. Recorded as candidate, not volumes. Two detector bugs on the way are written up: a validity test that counted zero as world-plausible scored zero-filled memory perfectly, and the 144-byte stride rested on four eyeballed offsets that the gap histogram did not support. A promising lead also failed: level 0's data.bin references one candidate offset exactly once (chance ~0.003), but none of seven other levels' candidates are referenced by their own data.bin, so the table is reached by walking structure and nothing links it to volumes. Not located. Nothing examined so far presents itself as a volume: core_data, the chunk region and gameplay.bin all yield structures that are accounted for by something else.

  • [~] level_overlay — header, constant block and level data parsed (gc_formats::overlay, 27/27). +0x10..+0x1220 -- 4,624 bytes -- is byte-identical on every level; the word at +0x0c is not the file length, falling below it on 17 levels and above on 10. The debug string pool it opens with carries Camera_CollPrimTest WARNING! - grid out of bounds! (RAR), independent support for reading the collision payload's two leading u16s as grid dimensions. +0x00 is the load base of the section this file is, 0x1A7C80, constant on all 27 levels because every level's .lit loads there. +0x0c is not the .lit size -- it falls below the file length on 16 of 27 levels, by up to 8.9%, so it is excluded by sign -- and it is still unnamed. The remaining header words are unnamed and the level-specific tail is not structured yet; the Phase 3 seven-section parser and relocation are untouched, though the seven sections are now named (lvl.vtbl, lvl.camvtbl, lvl.sndvtbl).

  • The post-shrub chunk container. A 16-byte header, a 10-byte zero gap, then offsets tagged 0xF0000000 | low24 resolving to chunk_start + low; header byte 14 is the record count, equal to offsets.len() on 1,123 of 1,123 chunks across all 27 levels. 77 distinct chunk types, 1,509 chunks, 34,491 records. Moby class IDs enrich at 9,493 hits against 246.1 expected with contiguous per-type arrays -- but the arrays land in a different place for each type, so no single pooled scan finds them.

  • Chunk record framing is closed, but the uniformity claim in it is retracted: only 257 of 1,509 chunks have a uniform record length, and 1,252 are variable within the chunk. "1,509 of 1,509" was resolve slicing at the chunk's first offset delta and the predicate confirming the parser back to itself. An earlier withdrawal ("variable-length was a pooled artifact") was wrong too -- the records were always variable. The record's leading byte is a per-chunk phase counter (bucket_rule, 1,262 of 1,509 chunks) and on 1,235 of them both leading bytes encode the record's own index in mixed radix (index_rule), so the leading bytes carry no per-record information.

  • The loader's transformation, from PCSX2 RAM. Disc offsets 0xf0000000 | low24 are rewritten in RAM to absolute EE pointers 0xf1850000 | low24 -- tag replaced by the chunk's load base, low 24 bits untouched. The payload itself is copied verbatim. The index finding is corroborated live: the byte at +2 takes exactly 0, 16, ... 240, the 16 values index_rule predicts.

  • Corrected: a record starts two bytes before the table's offset. The 02 00 read in RAM as a loader-inserted prefix is the record's own leading word, already on the disc at every record. The loaded table simply points two bytes earlier than the disc's, at that word. It is a field: uniform across a chunk's records on 1,439 of 1,509 chunks, fixed within all 71 types, values {1, 2, 4, 6, 8, 10}. Disproved readings: an entry count (57 distinct fractional "sizes") and a distinct-leading-byte count (0 of 1,506 agree).

  • [~] The chunk record fields are not decoded, and no type is named -- but the stride table is now measured: 74 signatures, 55 distinct strides, 52..640, every stride a multiple of 4, and 73 of 74 signatures use a single stride. The exception is pinned: [75 42 ef f2 71 b5] at strides {76, 92, 104}. Asserted including the exception, so "every type is uniform" cannot rot. A per-type layout found for one type (48 bytes of data then 60 permanently zero) does not generalise -- 0 of 9 types fit that shape. 65 of the 74 signatures are byte-identical in all 27 levels (9 are level-specific, 0 ragged), so most of the container is static game tables rather than level content -- asserted. Consequence for method: the 65 cannot be decoded by comparing levels, because there is nothing to compare. [75 42 ef] is both level-specific and the three-stride type, so it is where per-type decoding should start -- and decoding it found a parser defect: it makes three chunks per level, one of which has record offsets stepping 92, 88, 84, 92, 92, 92, 84, 88 on every level. resolve slices every record at the chunk's single stride, so four of the eight records over-read by 4 and 8 bytes and include the next record's head. Records still start correctly and bounds checking holds, so it is a data-correctness bug, not a safety one. The "strides {76, 92, 104}" description is a misdescription of three chunk shapes, one of them variable. Measured across all chunks rather than that one signature, the pattern is the norm: 1,252 of 1,509 chunks have non-uniform record offsets, across 66 of 74 signatures, with every deviation a word-aligned multiple of 4 and most exactly +/-4. Which model this is remains open -- genuinely variable-length records and fixed records with inter-record padding produce identical offset tables, and the offsets cannot tell them apart. A fix that slices each record against its successor was written and reverted: it implements one of the two models and the evidence does not establish it. It also revealed that three tests asserting "1,509 uniform record length" were passing only because the parser forced uniformity by slicing at a single stride. Settling it needs a field whose end is independently observable. The body is dense (74% of f32 reads plausible against a 49% random control) with an index-like tail -- 12 of 163 u16 slots in the largest type stay below 0x1000 -- but no count field exists in the first 64 body bytes at any width or stride tried. Only 9.4% of u16 slots are per-chunk constant, so records are not templates.

  • Moby class arrays are contiguous per type, and the pooled scan was wrong. See rule 3a in AGENTS.md: the original claim of a u16 array at slots 129-140 was the largest-stride type dominating an aggregate. Per type, every one of the 26 places its array somewhere else.

Gate: every parser has a golden test against bytes from the real disc, and reports mismatched header counts as failures rather than warnings.

Discipline: bound every read. Typed errors, no panics, no reads past the buffer.


Phase 3 — Overlay system

Goal: make 27 level overlays tractable.

Every overlay carries the same seven sections (.lit, .bss, .data, lvl.vtbl, lvl.camvtbl, lvl.sndvtbl, .text) and the same engine code, linked at different addresses. ReRAC solves this by masking relocations — j/ jal targets, lui immediates of main-RAM addresses, the %lo that completes them, and $gp-relative operands — then matching the masked words to find a reference function's copy on another overlay.

  • Port the masking and code-identity logic
  • Relocation::func and Relocation::data
  • $gp for this game (0x001AEFF0, against R&C1's 0x166c00) -- confirmed by OpenRAC/rac2-gc-decomp docs/LEVELS.md for all 27 levels
  • Extend the level table from 19 to 27

Gate: every level-01 function's relocated copy is found on all 27 overlays; cross-checked against an independent cluster hash.

Why this is high-value: it turns "27 overlays of unknown code" into a table lookup, and it is the main thing inherited from ReRAC.


Phase 4 — Engine core

Goal: draw a frame.

  • Graphics Synthesizer in wgpu, with the GS's own rules: its round(log2(z/n) + K) mip selection, blend and alpha-test behaviour, alpha 0x80 meaning 1.0, CLAMP wrap
  • VU0/VU1 skinning and lighting on the PS2 float model
  • LOD/morph/cull per TfragProc and TieProc
  • Occlusion culling, fog, shadows
  • GIF/VIF packet stream interpretation

Gate: CPU and GPU paths agree on every vertex of a reference level — the same identity check ReRAC runs.

Risk: VU1 microcode is binary in the game's .vutext section, so it is reverse engineered rather than parsed. This is the hardest single item in the project and the main threat to the timeline. Note that ReRAC's own moby VU1 analysis trips their disassembler on two opcodes.


Phase 5 — Veldin vertical slice

Goal: one planet, playable start to finish.

  • Ratchet's controller on a spherical planet
  • Chargeboots: hold/release timing, chained jumps, dash
  • Wrench and the camera
  • Collision and the HUD

Gate: Veldin is completable and visually matches PCSX2 frame-for-frame under replayed input.

This is the go/no-go for the project. If Veldin works, the other 26 levels are content. If it does not, the reason is discovered while it is still cheap. Spherical gravity is the thing most likely to break, because it invalidates R&C1's movement and camera assumptions wholesale.


Phase 6 — Remaining overlays and the shell

  • Per-level moby class dispatch via lvl.vtbl
  • Remaining 26 levels
  • Planets, travel, the Gadgetron vendor, RPG upgrades
  • Weapon modification bitmasks (lock-on, shock, acid, …)
  • Minigames and challenges
  • The 25-state memory-card FSM and its UI
  • IFF chunk saves, including import of a R&C1 save to unlock legacy weapons

Phase 7 — Audio and video

  • 989snd v2 audio decode and playback (not CRI ADX — a different codec)
  • SPU2 mixing, reverb zones, level sounds
  • IPU/MPEG-2 video for the title screen and cutscenes

Phase 8 — Fidelity pass

Differential testing against PCSX2 across every level:

  • Recorded-input frame comparison, per planet
  • Geometry, fog, blend, alpha, texture sampling
  • Audio sync and 60 Hz timing
  • Behavioural comparison: traversal routes, boss timing, RNG

Principles

  • A gate is pass/fail, not a percentage. "Looks fine" is not progress.
  • Negative results are written down. A ruled-out approach saves the next attempt as much as a working one.
  • The float model is not negotiable. Approximate is not the same as faithful, and the difference is visible in how the game feels.
  • Fidelity over tidiness. Where the original does something that looks like a bug, reproduce it and say why in a comment. See AGENTS.md.