Picked the tie flat encoding to close, sincee5cd3dfleft it as the largest unidentified field. What came out was a retraction of my own earlier claim. The texcoord slot was recorded at i16[4]/i16[5] on a 73.6% "both multiples of 4096" score. Two problems, either of which would be enough. First, that 73.6% was computed on the 16-byte element reading thate5cd3dfretracted. Half the records are VIF packets and half are flat, so the figure describes neither population cleanly. Second, and decisive on its own: measured on the FLAT records only -- levels 0-2, 930 records, 47,909 vertices -- the question that separates a 12.4 fixed-point texcoord from anything else is how many of a slot's distinct values are multiples of 4096 at all. For a texcoord it is all of them. slot distinct of which are multiples of 4096 i16[4] 1,488 16 i16[5] 1,261 16 i16[6] 966 4 i16[7] 489 5 One percent. The dominant PAIRS do look like UV corners -- (0,0) 5,773, (0,4096) 5,500, (4096,4096) 5,059, (4096,0) 4,521 -- and that is precisely why the claim was persuasive. But it is a statement about the most frequent values, not about the field: 1,472 of the 1,488 values in i16[4] are not multiples of 4096 at all. This is the moby class-ID artifact one level down. A small set of dominant values that happen to satisfy the test, scored as though the whole population did. The arithmetic was right about the popular values and the conclusion was wrong about the field. The retraction commit itself warned about exactly this shape, which makes the repeat more useful than the original claim was. So the texcoord slot of a flat tie record is NOT identified. What survives is the boundary work -- vert_ofs + vert_size + scissor_size is the region length -- and the LOD1/LOD2 sharing result, which is stride-insensitive and survives the stride retraction. Both docs updated: the roadmap's two [x] claims about the texcoord field and about i16[3] selecting its encoding are now [~] with the evidence, and the format file has a retraction section giving the table above. I did not find a replacement reading for i16[4..7]. Three discriminators were tried -- LOD sharing, multiple-of-4096 rate, value-set size -- and each came back ambiguous or negative. That is worth recording as the state rather than generating a fourth hypothesis. 258 tests.
44 KiB
Roadmap
Status as of 2026-10-11.
A from-scratch native Rust reimplementation of Ratchet & Clank: Going Commando (PS2). Each phase ends in a gate — something checkable, not a feeling that it is going well. Phases are ordered so that the riskiest unknowns are retired early, while they are still cheap to retire.
Where we are
- Target disc identified and verified (
SCUS_972.68v1.01, NTSC 60 Hz) - Workspace builds (
cargo check --workspace) - Repository hygiene in place (pre-commit guard,
.gitignore, licence record) - Phase 1 complete. The disc pipeline runs end to end and is verified against a committed hash table, every one of the 92 asset containers parsing. The pipeline now also writes each level's chunk descriptors where they exist (3 of 27 levels).
- Phase 2 in progress. Level containers are parsed through their typed
header, and the
dataextent is parsed as a container of byte ranges. The geometry decompresses: level 0 yields 16,969,296 bytes of level content from a 9.7 MB stream. - Parsers with real-disc tests, per asset type.
level_wad,data,core_indexclass tables,tfrag(format, topology, textures, LOD, normals, colour),occlusion,sky,moby,collisionframing and header,shrub(blob, packets, palette, billboard), the post-shrubchunkscontainer,sound_bankframing andlevel_overlay. 238 tests green against the real disc. What is open is naming, not locating -- see the Phase 2 list for the four types that still have no decoded fields:tievertices, collision cells,SBlkrecords,volumesandmoby_anim. - Lighting verified against the running game. PCSX2 2.8.2 with savestates
for level 0 and level 1. The tfrag lighting pass is byte-exact on both
against the game's own RAM. Setup and method in
plan/pcsx2-verification.md. - First drawable frame.
gc-engineloads a level, applies LOD selection and morphing for the camera, groups triangles by texture and draws them under a Bevy fly-camera. Level 0 renders as 26,758 textured triangles across 92 textures with correct UVs. SetGC_ENGINE_SHOT=<path.ppm>to capture a frame and exit. - The origin bug this exposed. Positions are i16 offsets from a per-patch origin, so taking the wrong STROW collapsed the whole level to a +-32 unit blob at the world origin while every parser test still passed: the vertex count matched, the triangle count matched, only the place was wrong. Fixed by taking the STROW immediately preceding the first position unpack rather than the last one.
- Per-vertex colour. The tfrag's RGBA array now modulates the texture,
drawn unlit so the frame shows real albedo rather than a faked lighting
model. Verified against the stored values (level 0 patch 0 runs
(25,26,26)to(108,144,141)), which are genuinely dark. - Tfrag normals. The 256-entry sine table is decoded from the level
overlay and normals are built per vertex and handed to the renderer. The
overlay has no ELF header, so the table cannot be reached through a
program header, and its offset differs per level (level 0
0xb650c, level 50xcca30); it is located by matching the whole 256-entry curve. Normals are negated as stored and get the same axis swap as positions. The pipeline now also writesoverlay.bin. - The PS2 float model.
gc-engine::ps2is the EE's FPU on raw bits: no denormals, no NaN or infinity, truncation toward zero, the adder's operand alignment, andrsqrtas sqrt-then-divide.MAXreads back in Rust as NaN, which is the sharpest available statement of why this is notf32. - The
LightTfragspass is written and unit-tested against the PS2 model: PEXT5 base colour,selectset blending, re-normalised blended directions, andbase + A*max(dA, dA*wA) + B*max(dB, dB*wB). - The level light bank parses and level 0 is fully lit.
gameplayis a WAD stream; the extractor had been writing it compressed, which is why the bank framing found garbage. All 80,274 vertices of level 0 now go throughLightTfrags. The renderer still counts and reports how many vertices it lit, because a silent fallback to stored colours would look plausible and be wrong. - The shrub region is bounded, and with it the whole lump. Shrub's end
was the last unlocated region bound: it is the start of the post-class
chunk container, found by a marker-paired signature in
core_data(the index carries no successor offset). The container closes shrub on the 24 shrub levels and tie on the three without. All 358 shrub class bodies are now bounded and are whole 8-byte records, so shrub measurements are trustworthy for the first time -- the "3.5M records" figure from the unbounded final span is replaced by a measured 9,600. - Lighting magnitudes are verified by the PCSX2 comparison loop.
gc-trace compare-tfrag-lightruns the pass over level 0's disc data and diffs it against the game's own lit colours in EE RAM (live over PINE, or from a savestate): all 162,400 colour bytes match, across 833/833 headers relocated and the light bank byte-identical. Two bugs it found: the normal's z was not negated (dark but plausible, 95.88% agreement), and records with no light set drew transparent black instead of the base colour. Seedocs/plan/pcsx2-verification.md. - Then the GS register set and the guard-band clip test.
The disc being the exact build the reference projects pin is the single biggest favourable fact so far: no disc conversion, no regional timing work, and no re-derivation of the reference baseline.
Phase 0 — Foundation ✅
Done.
- Disc identified:
SCUS_972.68v1.01, USA Black Label, NTSC 60 Hz. Boot ELF SHA-1bacda860cbd6723142e86db56f197e47e728970f, entry0x00131AE8— the entry point independently matches the value recorded byrac2-gc-decomp, which cross-confirms the identification. tools/identify-disc.pyreproduces the identification against any disc image and distinguishes a truncated dump from a non-PS2 image.- Five-crate workspace scaffolding, Bevy 0.19.1 pinned, builds clean.
.githooks/pre-commitblocks game data. Verified against seven cases:.iso, renamed disc image, WAD container, ADX container,extracted/, a 60 MiB file, and legitimate source.AGENTS.mdandPROVENANCE.mdrecord conventions and licences.
Open from this phase
alsa-lib-develis not installed, so Bevy's host audio backend is disabled inCargo.toml. Audio is the last phase; revisit then.- Bevy feature set is deliberately minimal. Re-add features as needed rather
than switching
default-featuresback on wholesale.
Phase 1 — Disc pipeline ✅
Done. Gate met: every extracted file matches the committed hash table.
iso9660reader (R&C2 lists assets as real files, unlike R&C1)G/*.WADcontainer header parsing — 92 containers, three table shapes- WAD LZ77 decompression (
wad.rs, four packet classes) - Unpack every container, bounds-checking each extent
- SHA-1 table for all 5,135 files (hashes only, never content)
verifystage, plus integration tests against a real disc- Format specs written:
disc-layout.md,global-wad.md,wad.md - Work record with the negative results:
plan/disc-pipeline.md
gc-extract identify disc.iso -> SCUS_972.68 v1.01 NTSC, ELF SHA-1 matches
gc-extract unpack disc.iso out/ -> 5135 files, 4,500,106,960 bytes, 10.8s
gc-extract verify out/ table.tsv -> OK: 5135 files match
Not done, deliberately: the table of contents at sector 1001 is documented but not read. Enumerating the ISO filesystem is simpler and sufficient; the ToC matters when per-level overlay loading is understood in Phase 3.
Two wrong turns are recorded in plan/disc-pipeline.md, because both would
be easy to repeat: the container pair table read backwards, and the LZ77 stream
mistaken for a bit stream. Both produced plausible-looking wrong output before
failing, which is the failure mode worth guarding against.
Phase 2 — Format parsers
Goal: every asset type parses into plain Rust values, tested against real disc data.
Phase 1 hands over the extents; this phase works out what each one contains.
Done so far:
level_wad.rs— the 0x60-byte header on all 27 level containers, naming four extents plus three chunk descriptors. Verified: extents tile each container in file order (27 of 27).- Two facts the format documentation does not state, both measured here:
sizes in this header are sector counts where the generic
G/table uses byte counts, andLEVEL21.WADreports level id 30, so container order is not level identity.
In progress:
-
data— the extent header locatescore_indexandcore_data; the geometry decompresses and is verified on level 0. -
core_indexclass tables — moby, tie and shrub. They were ruled out for want of an entry size, which is knowable exactly: each table ends precisely where the next begins, and that chain closes on 27 of 27 levels. 32 bytes per moby/tie entry, 48 per shrub. Across the disc: 5,796 moby, 1,864 tie, 358 shrub classes. Identifyingtfragby its own structure is what led here. -
occlusionandskyoffsets located and their first bytes read. -
Region layout of
core_dataestablished: tfrag, occlusion, sky, collision andtextures_baseare offsets, strictly increasing on all 27 levels. Terrain is[0, occlusion)and carries inline texels. -
Region layout closed: the chunk container's marker-paired signature ends the last class region, and the constant trailer ends the container. Every byte of level-specific
core_datanow belongs to a named region. -
tfrag vertex format: recovered. The tfrags block header gives a table offset (
0x40) and a patch count; each patch is a 0x40-byte header whosedataoffset is relative to the table, not to the block. Positions are V3_16 payloads inside a VIF1 command list, split across three lists per patch (common, LOD 0+1, LOD 0). The whole vector format is encoded in the command byte and every element size follows from it. Verified on all 27 levels: 18,102 patches, 821,439 vertices, each patch yielding exactly its ownvert_count. Seeplan/level-assets.mdandgc-formats::tfrag. -
tfrag topology: recovered. Vertex-info (UVs plus LOD parenting), the strip command lists and the index arrays, classified by unpack format and destination VU address. Strip indices address the vertex-info array rather than positions. Every patch's strips expand to exactly its own
tri_count: 18,102 patches, 809,879 triangles, all indices in range. Seeplan/level-assets.mdandgc-formats::tfrag. -
tfrag texture bindings and per-vertex data. The ad-gifs decode to a texture index, wrap modes, minification filter and LOD bias; the level index gained a texture table (
level_core.rs) for those indices to land in. Per-vertex RGBA and light/normal records recovered from their fixed offsets. All 27 levels: 47,950 ad-gifs, every index in range, 90% of the level texture tables referenced. -
Level textures decode. 8-bit indexed texels plus a 256-entry CSM1 CLUT from
gs_ram, honouring the PS2 CLUT block swap and the 0x80 alpha scale. The pipeline now writesgs_ram.bin, which it had been skipping. All 1,595 tfrag textures on all 27 levels decode; 96.8% fully opaque and 96.3% with real detail. Verified visually via the newgc-extract dump-texture, because a wrong lookup still yields a well-formed image of the wrong thing. -
tfrag LOD selection. Switch distances scale as
[6L, 4L, 2L]from the block header's float; six draw modes result from testing the bounding sphere againsttrunc(D * 1024), so a patch straddling a threshold takes the coarser mode and distance never moves back to finer detail. Morph weights run 0..1 across each tier's band, and collapse replaces an entry with its first parent rather than blending it. -
tfrag parent indices. The four V4_8 arrays are classified by VU address, but the addresses collide, and each V4_8 element holds four indices so an array is
count * 4bytes rather thancount. Withunk_06/unk_0asuppressing absent tiers, all 370,349 LOD entries across the 27 levels resolve: 29,043 LOD-01 and 341,306 LOD-0, zero failures. Seeplan/level-assets.mdfor the two corrections. -
gameplay— it is a WAD stream, exactly likecore_data; the extractor was writing it compressed. Decompressed, level 0 is 1,872,256 bytes and holds 5 directional light sets. The earlier "does not decode with the WAD LZ77 decoder, bit order and header length ruled out" note was wrong about the cause: the decoder was right and the input was not what it looked like. Corrected inplan/level-assets.md. -
[~]
sound_bank— theSBlkhead is now characterised:"SBlk", 1, 4, "DAW\0", 0constant on 27/27, and+0x14is the only varying word, with its low 16 bits always zero (0x0092..=0x0103). It tracks body length at ~100 bytes/unit and is not a count -- ratio 91.2..110.8, and no stride makes it exact. The absence of a stride is asserted too. Container framing parsed (SoundBankHeader::parse,parse_sound_bank);SBlkandDAWbounds verified on 27/27. The record fields insideSBlkare not decoded, andDAWshows no detectable stride, consistent with ADPCM. See the note underlevel_overlaybelow for what the game's own strings say about loading. -
data— not geometry. It is the level's file descriptor: its first table carries the exact byte lengths ofoverlay.bin,core_index.binandgs_ram.binat fixedu32indices 1, 3 and 5 on every level checked. What follows the descriptor is still unknown. -
tfrag— complete on the disc side. Vertex format, topology, strips (TfragStrip), texture bindings, per-vertex colour and normals, LOD selection, parent indices, morphing and collapse. 36 format tests, 9 texture tests, 23 engine tests. The morphing is a blend toward the parent vertices' midpoint, not toward separate morph targets — there are no morph targets in this format, which is what the old "morph targets unrecovered" note wrongly implied. -
The tfrag table record has exactly two relocated pointer fields, at
+0x10and+0x38, verified against PCSX2 across all 833 records: the loader adds the table's RAM address (0x848040= block base plustable_offset, matchingTfragHeader::data's own "relative to the table") to+0x10; every other byte of every record is verbatim, and 1,665 fields differ where 1,666 were possible. They are the existingTfragHeader::dataand::cube_ofs-- what RAM adds is that both are pointers, not plain offsets. All 833 patches parse, carrying 43,613 vertex-info entries; 499 of 833 have at least one morphing entry. -
Behind those pointers, every patch's bytes are rewritten -- and it is not morphing. Across all 833 patches: the 334 with no morphing entries differ by a median 7.80%, the 499 with morphing entries by 8.12%, and 0 of 833 are byte-identical to disc. Morphing does not predict which bytes change. Nor is it a relocation: the differences fall in short structured runs with no common
u32addend. The loader regenerates each patch's VU packet stream for the LOD tier it selected, which happens to every patch regardless of morphing. -
What tfrag is not yet verified is that the morph matches
resolve_vertex's formula. Reading a weight back out of RAM does not work -- the positions live inside regenerated VU packets, so there is no stable per-vertex correspondence. The LOD switch distances, morph weight and collapse edge remain derived from the data (tiers classified from the four V4_8 parent arrays, weight running zero to one across each band); verifying them needs the camera distance at the savestate so the weight can be computed and compared against the rendered result. -
The tie class header's
+0x0cword is the three LOD record counts — LOD 0 at+0x0c, LOD 1 at+0x0d, LOD 2 at+0x0e— verified on 1,864 of 1,864 classes by reading the counts independently from the blocks' own record tables. This was filed as "R&C1's normals offset, read past, not decoded", which was the right action on the wrong map: R&C1 carries the same quadruple at+0x20and it is not an offset on R&C2. The fourth byte is unidentified and is not the shader total (disagrees on 1,827 of 1,864). -
[~] The tie block's located
0xf6terminator is inside the last packet's data -- the last packet's computed end lands 2 to 16 bytes past it on all 2,150 blocks measured. So the block's extent is short by that much (the0xf6test could not see it: a false terminator passes that test). The block starts are unaffected and RAM-confirmed. The packet lengths themselves are verified for every record but the last. -
tie— fully chained.resolve_tie_classwalks both indirections: the class table is incore_index.bin, and the entry'soffset_in_asset_wadaddressescore_data.bin(notcore_data.wad). All 1,864 tie classes resolve witho_classagreeing on every one. Apacketsword of zero means no block at that LOD -- 2,177 real blocks, 1,551 absent slots. -
Tie blocks are self-delimiting:
parse_tie_blockwalks to the0xf6terminator that every real block carries (2,177 of 2,177), followed by zero padding. This is what makes blocks parseable at all --packets[1]bounds one on only 461 of 1,864 classes. -
The tie 16-byte record is R&C1's
TiePacketHeader. Identified not by resemblance but by R&C1's own rule,bfc_distance == 5 * shader_count, holding on 15,219 of 15,219 records across all 2,177 blocks. A block is[16-byte header][record table][vertex payload], and the header word at+0x00is the byte offset from the block start to the payload. The earlier "variable-length, not 16-byte records" reading was wrong: the word is not an element count, which is why dividing the payload by it gave 0 of 922. -
The tie vertex element is 16 bytes, measured over 95,612 vertices on levels 0-2 and settled by comparison: a 12-bit-fixed texcoord hit rate of 82.5% at 16 bytes against 47.8% at 8 and 44.8% at 12.
-
i16[0..2]are the vertex positions: LOD1/LOD2 of one class share 96.1% of their position triples, while LOD1 of A against LOD2 of B shares 0.4% -- a factor of 240, asserted including the ratio. Axis order and scale unnamed. -
i16[3]is a byte-sized index, not a coordinate: 201 distinct values in0..=254inside the ordinary layout, whose high byte is the layout tag (93.4% of elements are0x00, 5.6% are0x03, 146 selector values in all). Asserted. -
[~] The tie texcoord field is no longer located. It was recorded at
i16[4]/i16[5]on a 73.6% "both multiples of 4096" score, but that score is contaminated twice over: it was computed on the 16-byte reading thate5cd3dfretracted, and measured on all records rather than the flat ones. Restricted to the flat records, and asking the discriminating question -- how many of the slot's distinct values are multiples of 4096 at all -- gives 16 of 1,488 fori16[4]and 16 of 1,261 fori16[5]. A 12.4 fixed-point texcoord would be all of them. The dominant values do look like UV corners ((0,0),(0,4096),(4096,4096),(4096,0)), which is why it read as convincing; that is a statement about the most frequent values, not about the field -- the same shape as the 305x class-ID artifact. The texcoord slot in a flat tie record is not identified. The earlier note it supersedes read:i16[4]/i16[5]of the 16-byte element, scoring 73.6% for 12-bit fixedness over 36,501 level-0 vertices, against 1.1% for(0,1)and 4.8% for(2,3). The(6,7)pair scores higher at 76.6% and is not a texcoord -- it is the near-constant4096, 0pair, which is trivially two multiples of 4,096. -
[~]
i16[3]was recorded as selecting the texcoord encoding; that rested on the same 16-byte reading and is withdrawn with it. See the note above. The original text read: it is specifically the high byte. Grouped across 95,612 vertices on levels 0-2: high byte0x00covers 89,341 vertices (93.4%) at 92% 12-bit fixed, while every other high byte scores 0%. The split is on the byte, not the value --i16[3]of 1 and of 771 differ only in the high byte and behave completely differently. Within the all-zero case,i16[3] == 0exactly (5,851) is a third distribution at 62%. Three cases:0x00/0x00,0x00/1..255, and non-zero high byte. -
The tie packet's length follows from its record:
vert_ofs + vert_size + scissor_sizequadwords, on 12,927 of 12,927 record pairs whosestrip_countis zero and 0 of 115 whose it is not -- an exact partition, so the formula is safe whenstrip_count == 0and knows nothing otherwise. This unblocks slicing the payload per packet. It also corrects this file's own retraction of the same formula: the "0 of 13,042" re-measurement had swappedstrip_countforscissor_ofsand blamed the block set. -
[~] The remaining tie vertex fields are not named, and two readings are now excluded.
i16[6]/i16[7]are constant across a whole draw packet in 43%/34% of records, so they are usually per-packet rather than per-vertex.i16[3]is per-vertex (50 distinct values per packet) but not a skin or joint index: vertices sharing it are 0.95x farther apart than vertices differing in it, against a control that halves. It is also unshared across LODs. Recorded as not understood rather than guessed.i16range with 10,803-13,421 distinct values each.i16[0..2]are most likely position, but that is not established. -
[~]
shrub— class table done (48-byte entries, not 32; reading shrub at 32 still gives 231 sane-looking entries of 358, which is the hazard). Geometry header parsed and checked field by field on all 358 spans:+0x24equals the entry's owno_class,+0x2cis an in-span data offset,+0x16/+0x18are zero, and+0x1cnames a billboard on exactly the 25 classes whose core-index entry carries billboard texture fields. Shrub does not use moby's part-table format. The body is a 24-entry palette of unit-length s16 normals (squared length within 65,535 of 32767² on all 8,592 entries — a random triple would miss by a billion) plus zero padding records; the billboard record carries K, MMIN and the TEX0/MIPTBP addresses as R&C1's does. What remains open: the three floats in the billboard record,size,mode_bits,count, and where the mesh geometry lives — R&C1 puts the mesh packets in this blob and R&C2's blob holds only the palette. -
The tie LOD block is a table of 16-byte records. The header's word at
+0x00is the byte offset from the block start to the vertex payload -- the record table's length -- not a count, which is why the payload never divides by it. Each record is a leadingu32pointer relative to the table start plus twelve verbatim field bytes, relocated at load totable + word. -
The record is R&C1's
TiePacketHeader. R&C1's own rulebfc_distance == 5 * shader_countholds on all 15,219 records of all 2,177 blocks on all 27 levels. R&C1's rule for the colour offset holds on none of them, so the games use that field differently. -
The tie relocation verifies on two levels.
gc-trace verify-tie-recordschecks it against PCSX2's RAM: 87 of 87 blocks on level 0 and 104 of 104 on level 1, zero mismatches, which is what makes the record layout a finding rather than a reading of one block. -
The shrub mesh geometry is in the class blob. A packet table at
+0x40namescountpackets between the table and the palette, and they tile on 358 of 358 spans (2,239 packets), which makescountthe packet count. Confirmed against the running game, which relocates the table's offsets to absolute addresses anddata_offsetlikewise. -
moby— per-class model geometry. A class span opens with au32pointing at a part table of 16-byte records; parts tile exactly, withvertex_offset + 16 * vertex_count == next_partholding for 43,467 of 43,467 parts across every class on all 27 levels. That identity is what makes the record size fall out of the data rather than being assumed. 1,270,802 vertices over 4,560 class spans; 116 spans carry no geometry. Vertices are 16-byte VIF-packed half-float quads, handed back raw. -
moby_anim— still open, and the obvious reading is ruled out. An instance table would have to scale with the level's moby class count, but per-chunk-type record counts are level-independent -- the same types report 90, 77, 50 and 47 on every level tried, and the largest type totals 2,430 = 90 x 27. Nothing in the chunk set is keyed by moby class. The types also pair up (two types both reporting 90 on every level, another pair both 77), which looks like two views of one fixed table rather than two independent tables. -
collision— container framing and header fields. A 0x40-byte header self-describes: it carries its own size and the payload size, andheader + payloadstays insidecore_dataon all 27 levels. 41,481,520 payload bytes across 19 levels; 8 declare zero length. Header bytes+8reads zero on 22 levels and a large value on 7/9/12/13/14; the payload's two leadingu16are non-zero on all 19 non-empty levels (1-74 and 13-112) and are reported asCollisionHeader::first/::second. -
The collision cell layout is not decoded. The payload does not walk as fixed-stride cells. On level 4 a table starts at payload
+5932and runs 17 entries -- deltas rise then fall -- against a 75-cell grid, with no arithmetic relating the two. A 64-byte RAM stride looked promising and is disproved -- the longest such run is 27 records of identity matrices, one per level. -
The collision payload is essentially verbatim in RAM, not reshaped. Level 0's sits at RAM
0xac6800for a disc address of0x27e800: 903,212 of 903,248 bytes identical, first difference at+0xdc82c, with 6 bytes differing in the final 36. Asserted by a savestate-gated test that compares the whole payload. This overturns the earlier claim that the grid dimensions are absent from RAM: level 0's11 x 33are the first two bytes of that block. The earlier negative came from searching for the dimensions as au32pair when they are twou16s -- a bad pattern, not a property of the data. The payload contains ascendingu32offset tables from+1024onward, and disc-side decoding is no longer blocked on reshaping. Seeplan/pcsx2-verification.md. -
occlusion— the PVS grid. A z/y/x tree of u16 node offsets over 4x4x4-unit cells, populated cells naming 128-byte masks. 412,521 cells over 140,939 masks across the 24 levels that have one; the mask array ends within 48 bytes of the block boundary on every level. 3 carry no occlusion data, and an offset of 0 means absent rather than "at the start" — the latter parses as a grid whose mask array happens to fit, which is how it nearly went wrong. -
sky— 149 sub-objects across all 27 levels, each with a count, a four-float bounding sphere and texture dimensions. The offset table at+0x20is self-limiting: entries run while strictly inside the region. The word at+0x0cis not a count -- it disagrees with the table on most levels and reads 196,617 on level 2 against 4 sub-objects. -
The overlay's load base is
field_00 - 16= 1,735,792, derived from the overlay's own profiler table (whose absolute addresses point back at its names, on all 27 levels) and confirmed in RAM: all four profiler names land where the base predicts. Any overlay offset can now be carried to a RAM address. The overlay is relocated, not copied: only 37.67% is byte-identical at the base, so tables inside it move. -
[~]
volumes— still not identified, but a candidate exists. Every level'sgameplay.bincarries a table of 144-byte float records with world positions at+0x0c, preceded by an all-zero record (levels 0-9: 23 to 612 records). That is the shape of a volume trigger region and it is not tied to anything else in the file -- but camera bounds, spawn points and path nodes would look identical, and there is no engine-sideVolumestring to discriminate with. Recorded as candidate, not volumes. Two detector bugs on the way are written up: a validity test that counted zero as world-plausible scored zero-filled memory perfectly, and the 144-byte stride rested on four eyeballed offsets that the gap histogram did not support. A promising lead also failed: level 0'sdata.binreferences one candidate offset exactly once (chance ~0.003), but none of seven other levels' candidates are referenced by their owndata.bin, so the table is reached by walking structure and nothing links it to volumes. Not located. Nothing examined so far presents itself as a volume:core_data, the chunk region andgameplay.binall yield structures that are accounted for by something else. -
[~]
level_overlay— header, constant block and level data parsed (gc_formats::overlay, 27/27).+0x10..+0x1220-- 4,624 bytes -- is byte-identical on every level; the word at+0x0cis not the file length, falling below it on 17 levels and above on 10. The debug string pool it opens with carriesCamera_CollPrimTest WARNING! - grid out of bounds! (RAR), independent support for reading the collision payload's two leadingu16s as grid dimensions.+0x00is the load base of the section this file is,0x1A7C80, constant on all 27 levels because every level's.litloads there.+0x0cis not the.litsize -- it falls below the file length on 16 of 27 levels, by up to 8.9%, so it is excluded by sign -- and it is still unnamed. The remaining header words are unnamed and the level-specific tail is not structured yet; the Phase 3 seven-section parser and relocation are untouched, though the seven sections are now named (lvl.vtbl,lvl.camvtbl,lvl.sndvtbl). -
The post-shrub chunk container. A 16-byte header, a 10-byte zero gap, then offsets tagged
0xF0000000 | low24resolving tochunk_start + low; header byte 14 is the record count, equal tooffsets.len()on 1,123 of 1,123 chunks across all 27 levels. 77 distinct chunk types, 1,509 chunks, 34,491 records. Moby class IDs enrich at 9,493 hits against 246.1 expected with contiguous per-type arrays -- but the arrays land in a different place for each type, so no single pooled scan finds them. -
Chunk record framing is closed, but the uniformity claim in it is retracted: only 257 of 1,509 chunks have a uniform record length, and 1,252 are variable within the chunk. "1,509 of 1,509" was
resolveslicing at the chunk's first offset delta and the predicate confirming the parser back to itself. An earlier withdrawal ("variable-length was a pooled artifact") was wrong too -- the records were always variable. The record's leading byte is a per-chunk phase counter (bucket_rule, 1,262 of 1,509 chunks) and on 1,235 of them both leading bytes encode the record's own index in mixed radix (index_rule), so the leading bytes carry no per-record information. -
The loader's transformation, from PCSX2 RAM. Disc offsets
0xf0000000 | low24are rewritten in RAM to absolute EE pointers0xf1850000 | low24-- tag replaced by the chunk's load base, low 24 bits untouched. The payload itself is copied verbatim. The index finding is corroborated live: the byte at+2takes exactly0, 16, ... 240, the 16 valuesindex_rulepredicts. -
Corrected: a record starts two bytes before the table's offset. The
02 00read in RAM as a loader-inserted prefix is the record's own leading word, already on the disc at every record. The loaded table simply points two bytes earlier than the disc's, at that word. It is a field: uniform across a chunk's records on 1,439 of 1,509 chunks, fixed within all 71 types, values{1, 2, 4, 6, 8, 10}. Disproved readings: an entry count (57 distinct fractional "sizes") and a distinct-leading-byte count (0 of 1,506 agree). -
[~] The chunk record fields are not decoded, and no type is named -- but the stride table is now measured: 74 signatures, 55 distinct strides, 52..640, every stride a multiple of 4, and 73 of 74 signatures use a single stride. The exception is pinned:
[75 42 ef f2 71 b5]at strides{76, 92, 104}. Asserted including the exception, so "every type is uniform" cannot rot. A per-type layout found for one type (48 bytes of data then 60 permanently zero) does not generalise -- 0 of 9 types fit that shape. 65 of the 74 signatures are byte-identical in all 27 levels (9 are level-specific, 0 ragged), so most of the container is static game tables rather than level content -- asserted. Consequence for method: the 65 cannot be decoded by comparing levels, because there is nothing to compare.[75 42 ef]is both level-specific and the three-stride type, so it is where per-type decoding should start -- and decoding it found a parser defect: it makes three chunks per level, one of which has record offsets stepping92, 88, 84, 92, 92, 92, 84, 88on every level.resolveslices every record at the chunk's single stride, so four of the eight records over-read by 4 and 8 bytes and include the next record's head. Records still start correctly and bounds checking holds, so it is a data-correctness bug, not a safety one. The "strides {76, 92, 104}" description is a misdescription of three chunk shapes, one of them variable. Measured across all chunks rather than that one signature, the pattern is the norm: 1,252 of 1,509 chunks have non-uniform record offsets, across 66 of 74 signatures, with every deviation a word-aligned multiple of 4 and most exactly +/-4. Which model this is remains open -- genuinely variable-length records and fixed records with inter-record padding produce identical offset tables, and the offsets cannot tell them apart. A fix that slices each record against its successor was written and reverted: it implements one of the two models and the evidence does not establish it. It also revealed that three tests asserting "1,509 uniform record length" were passing only because the parser forced uniformity by slicing at a single stride. Settling it needs a field whose end is independently observable. The body is dense (74% off32reads plausible against a 49% random control) with an index-like tail -- 12 of 163u16slots in the largest type stay below0x1000-- but no count field exists in the first 64 body bytes at any width or stride tried. Only 9.4% ofu16slots are per-chunk constant, so records are not templates. -
Moby class arrays are contiguous per type, and the pooled scan was wrong. See rule 3a in
AGENTS.md: the original claim of au16array at slots 129-140 was the largest-stride type dominating an aggregate. Per type, every one of the 26 places its array somewhere else.
Gate: every parser has a golden test against bytes from the real disc, and reports mismatched header counts as failures rather than warnings.
Gate status, checked
Both halves verified by inspection rather than assertion:
- Golden tests. Every module in
gc-formats/srcis referenced by a test or another module's test path; none is unreferenced.gc-extract/tests/disc.rscarries 50 real-disc tests, and theoverlay_relocmask test resolves a level-01 function on all 26 other overlays. - Counts are failures, not warnings. No parser writes warning-style output
outside
#[cfg(test)]-- the onlyeprintln!calls ingc-formatsare inside test modules. Count and length problems surface as typed errors: 42 uses ofErrorKind::Invalid/InvalidLength, includingglobal_wad's "pair table is not a whole number of entries", which is exactly a header count disagreeing with the data.
The gate is met.
Blocked, and what would unblock each
AGENTS.md requires a blocked item to say what would unblock it. These four did not, which is itself a gap; this is that gap closed. None of these is a shortcut or an approximation -- each is a measurement this repository currently cannot make.
- Collision cell layout. The payload is verbatim in RAM and does not walk as
fixed-stride cells; every disc-side segmentation is excluded. Needs the game's
own
Camera_CollPrimTest, which lives in.lit. The route that would work: the code addresses the assert string by something other than a$gp-relative load -- a%hi/%losplit that does not straddle zero, a pointer table, or a different section. A naive$gpsearch returns zero hits and will keep doing so. - Volumes. A 144-byte float record table with world positions at
+0x0cexists in every level and is not referenced by anydata.bin, so it is reached by walking structure, and nothing ties it to volumes. Needs ground truth: a count of level 0's trigger volumes from a map or guide, to match against a table's population. No class name containing "volume" exists on the disc, so the dispatch-table route is closed. moby_anim. The instance-table reading is disproved and nothing replaces it. Needs the class-dispatch table for the classes that reference the animation data, which needsmobyclass records decoded past the header -- a dependency, not a measurement.- tfrag morph. All patch bytes are rewritten by LOD-selected VU-packet regeneration, so the morph is invisible in a savestate. Needs a frame captured at a known camera distance, so the expected blend is computable and can be compared. Requires running the game and driving the camera, not analysing a static savestate.
Discipline: bound every read. Typed errors, no panics, no reads past the buffer.
Phase 3 — Overlay system
Goal: make 27 level overlays tractable.
Every overlay carries the same seven sections (.lit, .bss, .data,
lvl.vtbl, lvl.camvtbl, lvl.sndvtbl, .text) and the same engine code,
linked at different addresses. ReRAC solves this by masking relocations — j/
jal targets, lui immediates of main-RAM addresses, the %lo that completes
them, and $gp-relative operands — then matching the masked words to find a
reference function's copy on another overlay.
- Port the masking and code-identity logic (
gc_formats::overlay_reloc,mask/address_refs/key). Verified against the disc: the masked form of a level-01 function (858 words at0x2a3b88) is found exactly once on each of the other 26 overlays, and 0 times on its own level. Relocation::funcandRelocation::data, plusis_identity,same_codeandcopies$gpfor this game (0x001AEFF0, against R&C1's0x166c00) -- confirmed byOpenRAC/rac2-gc-decompdocs/LEVELS.mdfor all 27 levels- Extend the level table from 19 to 27
Gate: every level-01 function's relocated copy is found on all 27 overlays; cross-checked against an independent cluster hash.
Why this is high-value: it turns "27 overlays of unknown code" into a table lookup, and it is the main thing inherited from ReRAC.
Phase 4 — Engine core
Goal: draw a frame.
- Graphics Synthesizer in wgpu, with the GS's own rules: its
round(log2(z/n) + K)mip selection, blend and alpha-test behaviour, alpha0x80meaning 1.0, CLAMP wrap - VU0/VU1 skinning and lighting on the PS2 float model
- LOD/morph/cull per
TfragProcandTieProc - Occlusion culling, fog, shadows
- GIF/VIF packet stream interpretation
Gate: CPU and GPU paths agree on every vertex of a reference level — the same identity check ReRAC runs.
Risk: VU1 microcode is binary in the game's .vutext section, so it is
reverse engineered rather than parsed. This is the hardest single item in the
project and the main threat to the timeline. Note that ReRAC's own moby VU1
analysis trips their disassembler on two opcodes.
Phase 5 — Veldin vertical slice
Goal: one planet, playable start to finish.
- Ratchet's controller on a spherical planet
- Chargeboots: hold/release timing, chained jumps, dash
- Wrench and the camera
- Collision and the HUD
Gate: Veldin is completable and visually matches PCSX2 frame-for-frame under replayed input.
This is the go/no-go for the project. If Veldin works, the other 26 levels are content. If it does not, the reason is discovered while it is still cheap. Spherical gravity is the thing most likely to break, because it invalidates R&C1's movement and camera assumptions wholesale.
Phase 6 — Remaining overlays and the shell
- Per-level moby class dispatch via
lvl.vtbl - Remaining 26 levels
- Planets, travel, the Gadgetron vendor, RPG upgrades
- Weapon modification bitmasks (lock-on, shock, acid, …)
- Minigames and challenges
- The 25-state memory-card FSM and its UI
- IFF chunk saves, including import of a R&C1 save to unlock legacy weapons
Phase 7 — Audio and video
- 989snd v2 audio decode and playback (not CRI ADX — a different codec)
- SPU2 mixing, reverb zones, level sounds
- IPU/MPEG-2 video for the title screen and cutscenes
Phase 8 — Fidelity pass
Differential testing against PCSX2 across every level:
- Recorded-input frame comparison, per planet
- Geometry, fog, blend, alpha, texture sampling
- Audio sync and 60 Hz timing
- Behavioural comparison: traversal routes, boss timing, RNG
Principles
- A gate is pass/fail, not a percentage. "Looks fine" is not progress.
- Negative results are written down. A ruled-out approach saves the next attempt as much as a working one.
- The float model is not negotiable. Approximate is not the same as faithful, and the difference is visible in how the game feels.
- Fidelity over tidiness. Where the original does something that looks like
a bug, reproduce it and say why in a comment. See
AGENTS.md.