ReaperOfVeriod
3a1156d8e1
The morph was recorded as unverifiable because "reading a weight back out of
RAM does not work -- the positions live inside regenerated VU packets". That
is right about the correspondence and wrong about the cause. The cause is
measurable, and it comes back clean:
patches 833
positions 40,600 disc == ram
positions identical 40,600
block bytes differing 166,465
inside a position unpack 0
inside another unpack 49,484
in a VIF command word 116,981
Range: level 0, all 833 tfrag patches, live PINE RAM against the disc. Two
independent measurements agree. Walking the VIF lists on both copies and
comparing parsed positions gives 40,600 identical and zero changed; a raw
per-byte diff of each patch's own extent classifies every differing byte, and
none of them is in a position unpack. The position ranges total 490,224 bytes.
So the blend is not precomputed into the geometry. The rewrites are packet
framing -- 70% command words -- plus other unpack payloads, which is the loader
rebuilding the VIF packet for the tier it selected while leaving the vertices
alone. The morph happens at draw time in the VU stream.
The standing note becomes a sharper one: there was never a correspondence
problem. The values were identical, so there was nothing to read.
gc-trace gains verify-morph, which does both measurements. TfragGeometry
gains position_ranges and other_ranges so the walk needed to classify a diff
is already paid for, and the ranges come from the same walk the rest of the
parse uses rather than a second one that could drift.
Per-patch extents are the gap to the next patch's data pointer. That is exact
and needs no guess about which header field marks a patch's end; the first
attempt at this compared each patch against the whole rest of the region and
reported 65 million differing bytes, which is what an unbounded comparison
looks like.
tfrag_positions_are_verbatim_and_the_rewrites_are_elsewhere is gated on
GC_SSTATE and asserts the four bounds, plus two guards: the rewrites must be
non-trivial (over 100,000 bytes) and mostly command words, so a RAM image that
was a verbatim copy of the disc -- which would make the zero look good for the
wrong reason -- fails instead.
Suite with the disc and GC_SSTATE: 68 disc tests, 156 gc-formats, 22
gc-trace, 23 gc-engine, 19 gc-game. Clippy clean.
844 lines
53 KiB
Markdown
844 lines
53 KiB
Markdown
# Roadmap
|
|
|
|
Status as of **2026-10-11**.
|
|
|
|
A from-scratch native Rust reimplementation of Ratchet & Clank: Going Commando
|
|
(PS2). Each phase ends in a **gate** — something checkable, not a feeling that
|
|
it is going well. Phases are ordered so that the riskiest unknowns are retired
|
|
early, while they are still cheap to retire.
|
|
|
|
## Where we are
|
|
|
|
- [x] Target disc identified and verified (`SCUS_972.68` v1.01, NTSC 60 Hz)
|
|
- [x] Workspace builds (`cargo check --workspace`)
|
|
- [x] Repository hygiene in place (pre-commit guard, `.gitignore`, licence
|
|
record)
|
|
- [x] **Phase 1 complete.** The disc pipeline runs end to end and is verified
|
|
against a committed hash table, every one of the 92 asset containers
|
|
parsing. The pipeline now also writes each level's chunk descriptors
|
|
where they exist (3 of 27 levels).
|
|
- [x] **Phase 2 in progress.** Level containers are parsed through their typed
|
|
header, and the `data` extent is parsed as a container of byte ranges. The
|
|
geometry decompresses: level 0 yields 16,969,296 bytes of level content
|
|
from a 9.7 MB stream.
|
|
- [x] **Parsers with real-disc tests, per asset type.** `level_wad`, `data`,
|
|
`core_index` class tables, `tfrag` (format, topology, textures, LOD,
|
|
normals, colour), `occlusion`, `sky`, `moby`, `collision` framing and
|
|
header, `shrub` (blob, packets, palette, billboard), the post-shrub
|
|
`chunks` container, `sound_bank` framing and `level_overlay`. 238 tests
|
|
green against the real disc. **What is open is naming, not locating** --
|
|
see the Phase 2 list for the four types that still have no decoded
|
|
fields: `tie` vertices, collision cells, `SBlk` records, `volumes` and
|
|
`moby_anim`.
|
|
- [x] **Lighting verified against the running game.** PCSX2 2.8.2 with savestates
|
|
for level 0 and level 1. The tfrag lighting pass is byte-exact on both
|
|
against the game's own RAM. Setup and method in
|
|
`plan/pcsx2-verification.md`.
|
|
- [x] **First drawable frame.** `gc-engine` loads a level, applies LOD selection
|
|
and morphing for the camera, groups triangles by texture and draws them
|
|
under a Bevy fly-camera. Level 0 renders as 26,758 textured triangles
|
|
across 92 textures with correct UVs. Set `GC_ENGINE_SHOT=<path.ppm>` to
|
|
capture a frame and exit.
|
|
- [ ] **The origin bug this exposed.** Positions are i16 offsets from a
|
|
per-patch origin, so taking the wrong STROW collapsed the whole level to
|
|
a +-32 unit blob at the world origin while every parser test still
|
|
passed: the vertex count matched, the triangle count matched, only the
|
|
*place* was wrong. Fixed by taking the STROW immediately preceding the
|
|
first position unpack rather than the last one.
|
|
- [x] **Per-vertex colour.** The tfrag's RGBA array now modulates the texture,
|
|
drawn unlit so the frame shows real albedo rather than a faked lighting
|
|
model. Verified against the stored values (level 0 patch 0 runs
|
|
`(25,26,26)` to `(108,144,141)`), which are genuinely dark.
|
|
- [x] **Tfrag normals.** The 256-entry sine table is decoded from the level
|
|
overlay and normals are built per vertex and handed to the renderer. The
|
|
overlay has **no ELF header**, so the table cannot be reached through a
|
|
program header, and its offset differs per level (level 0 `0xb650c`,
|
|
level 5 `0xcca30`); it is located by matching the whole 256-entry curve.
|
|
Normals are negated as stored and get the same axis swap as positions.
|
|
The pipeline now also writes `overlay.bin`.
|
|
- [x] **The PS2 float model.** `gc-engine::ps2` is the EE's FPU on raw bits: no
|
|
denormals, no NaN or infinity, truncation toward zero, the adder's
|
|
operand alignment, and `rsqrt` as sqrt-then-divide. `MAX` reads back in
|
|
Rust as NaN, which is the sharpest available statement of why this is
|
|
not `f32`.
|
|
- [x] **The `LightTfrags` pass** is written and unit-tested against the PS2
|
|
model: PEXT5 base colour, `select` set blending, re-normalised blended
|
|
directions, and `base + A*max(dA, dA*wA) + B*max(dB, dB*wB)`.
|
|
- [x] **The level light bank parses and level 0 is fully lit.** `gameplay` is a
|
|
WAD stream; the extractor had been writing it compressed, which is why
|
|
the bank framing found garbage. All 80,274 vertices of level 0 now go
|
|
through `LightTfrags`. The renderer still counts and reports how many
|
|
vertices it lit, because a silent fallback to stored colours would look
|
|
plausible and be wrong.
|
|
- [x] **The shrub region is bounded, and with it the whole lump.** Shrub's end
|
|
was the last unlocated region bound: it is the start of the post-class
|
|
chunk container, found by a marker-paired signature in `core_data` (the
|
|
index carries no successor offset). The container closes shrub on the
|
|
24 shrub levels and tie on the three without. All 358 shrub class bodies
|
|
are now bounded and are whole 8-byte records, so shrub measurements are
|
|
trustworthy for the first time -- the "3.5M records" figure from the
|
|
unbounded final span is replaced by a measured 9,600.
|
|
- [x] **Lighting *magnitudes* are verified** by the PCSX2 comparison loop.
|
|
`gc-trace compare-tfrag-light` runs the pass over level 0's disc data and
|
|
diffs it against the game's own lit colours in EE RAM (live over PINE, or
|
|
from a savestate): **all 162,400 colour bytes match**, across 833/833
|
|
headers relocated and the light bank byte-identical. Two bugs it found:
|
|
the normal's z was not negated (dark but plausible, 95.88% agreement),
|
|
and records with no light set drew transparent black instead of the base
|
|
colour. See `docs/plan/pcsx2-verification.md`.
|
|
- [ ] Then the GS register set and the guard-band clip test.
|
|
|
|
The disc being the exact build the reference projects pin is the single
|
|
biggest favourable fact so far: no disc conversion, no regional timing work,
|
|
and no re-derivation of the reference baseline.
|
|
|
|
---
|
|
|
|
## Phase 0 — Foundation ✅
|
|
|
|
**Done.**
|
|
|
|
- Disc identified: `SCUS_972.68` v1.01, USA Black Label, NTSC 60 Hz. Boot ELF
|
|
SHA-1 `bacda860cbd6723142e86db56f197e47e728970f`, entry `0x00131AE8` — the
|
|
entry point independently matches the value recorded by `rac2-gc-decomp`,
|
|
which cross-confirms the identification.
|
|
- `tools/identify-disc.py` reproduces the identification against any disc image
|
|
and distinguishes a truncated dump from a non-PS2 image.
|
|
- Five-crate workspace scaffolding, Bevy 0.19.1 pinned, builds clean.
|
|
- `.githooks/pre-commit` blocks game data. Verified against seven cases:
|
|
`.iso`, renamed disc image, WAD container, ADX container, `extracted/`,
|
|
a 60 MiB file, and legitimate source.
|
|
- `AGENTS.md` and `PROVENANCE.md` record conventions and licences.
|
|
|
|
### Open from this phase
|
|
|
|
- `alsa-lib-devel` is not installed, so Bevy's host audio backend is disabled in
|
|
`Cargo.toml`. Audio is the last phase; revisit then.
|
|
- Bevy feature set is deliberately minimal. Re-add features as needed rather
|
|
than switching `default-features` back on wholesale.
|
|
|
|
---
|
|
|
|
## Phase 1 — Disc pipeline ✅
|
|
|
|
**Done.** Gate met: every extracted file matches the committed hash table.
|
|
|
|
- [x] `iso9660` reader (R&C2 lists assets as real files, unlike R&C1)
|
|
- [x] `G/*.WAD` container header parsing — 92 containers, three table shapes
|
|
- [x] WAD LZ77 decompression (`wad.rs`, four packet classes)
|
|
- [x] Unpack every container, bounds-checking each extent
|
|
- [x] SHA-1 table for all 5,135 files (hashes only, never content)
|
|
- [x] `verify` stage, plus integration tests against a real disc
|
|
- [x] Format specs written: `disc-layout.md`, `global-wad.md`, `wad.md`
|
|
- [x] Work record with the negative results: `plan/disc-pipeline.md`
|
|
|
|
```text
|
|
gc-extract identify disc.iso -> SCUS_972.68 v1.01 NTSC, ELF SHA-1 matches
|
|
gc-extract unpack disc.iso out/ -> 5135 files, 4,500,106,960 bytes, 10.8s
|
|
gc-extract verify out/ table.tsv -> OK: 5135 files match
|
|
```
|
|
|
|
**Not done, deliberately:** the table of contents at sector 1001 is documented
|
|
but not read. Enumerating the ISO filesystem is simpler and sufficient; the ToC
|
|
matters when per-level overlay loading is understood in Phase 3.
|
|
|
|
**Two wrong turns are recorded** in `plan/disc-pipeline.md`, because both would
|
|
be easy to repeat: the container pair table read backwards, and the LZ77 stream
|
|
mistaken for a bit stream. Both produced plausible-looking wrong output before
|
|
failing, which is the failure mode worth guarding against.
|
|
|
|
---
|
|
|
|
## Phase 2 — Format parsers
|
|
|
|
**Goal:** every asset type parses into plain Rust values, tested against real
|
|
disc data.
|
|
|
|
Phase 1 hands over the extents; this phase works out what each one contains.
|
|
|
|
**Done so far:**
|
|
|
|
- [x] `level_wad.rs` — the 0x60-byte header on all 27 level containers, naming
|
|
four extents plus three chunk descriptors. Verified: extents tile each
|
|
container in file order (27 of 27).
|
|
- [x] Two facts the format documentation does not state, both measured here:
|
|
sizes in this header are **sector counts** where the generic `G/` table
|
|
uses byte counts, and **`LEVEL21.WAD` reports level id 30**, so container
|
|
order is not level identity.
|
|
|
|
**In progress:**
|
|
|
|
- [x] `data` — the extent header locates `core_index` and `core_data`; the
|
|
geometry decompresses and is verified on level 0.
|
|
- [x] `core_index` class tables — moby, tie and shrub. They were ruled out
|
|
for want of an entry size, which is knowable exactly: each table ends
|
|
precisely where the next begins, and that chain closes on 27 of 27
|
|
levels. 32 bytes per moby/tie entry, 48 per shrub. Across the disc:
|
|
**5,796 moby, 1,864 tie, 358 shrub classes**. Identifying `tfrag` by its
|
|
own structure is what led here.
|
|
- [x] `occlusion` and `sky` offsets located and their first bytes read.
|
|
- [x] Region layout of `core_data` established: tfrag, occlusion, sky, collision
|
|
and `textures_base` are offsets, strictly increasing on all 27 levels.
|
|
Terrain is `[0, occlusion)` and carries inline texels.
|
|
- [x] Region layout **closed**: the chunk container's marker-paired signature
|
|
ends the last class region, and the constant trailer ends the container.
|
|
Every byte of level-specific `core_data` now belongs to a named region.
|
|
- [x] **tfrag vertex format: recovered.** The tfrags block header gives a table
|
|
offset (`0x40`) and a patch count; each patch is a 0x40-byte header whose
|
|
`data` offset is relative to the table, not to the block. Positions are
|
|
V3_16 payloads inside a VIF1 command list, split across three lists per
|
|
patch (common, LOD 0+1, LOD 0). The whole vector format is encoded in the
|
|
command byte and every element size follows from it. Verified on all 27
|
|
levels: **18,102 patches, 821,439 vertices**, each patch yielding exactly
|
|
its own `vert_count`. See `plan/level-assets.md` and `gc-formats::tfrag`.
|
|
- [x] **tfrag topology: recovered.** Vertex-info (UVs plus LOD parenting), the
|
|
strip command lists and the index arrays, classified by unpack format and
|
|
destination VU address. Strip indices address the vertex-info array
|
|
rather than positions. Every patch's strips expand to exactly its own
|
|
`tri_count`: **18,102 patches, 809,879 triangles**, all indices in range.
|
|
See `plan/level-assets.md` and `gc-formats::tfrag`.
|
|
- [x] **tfrag texture bindings and per-vertex data.** The ad-gifs decode to a
|
|
texture index, wrap modes, minification filter and LOD bias; the level
|
|
index gained a texture table (`level_core.rs`) for those indices to land
|
|
in. Per-vertex RGBA and light/normal records recovered from their fixed
|
|
offsets. All 27 levels: 47,950 ad-gifs, every index in range, 90% of the
|
|
level texture tables referenced.
|
|
- [x] **Level textures decode.** 8-bit indexed texels plus a 256-entry CSM1
|
|
CLUT from `gs_ram`, honouring the PS2 CLUT block swap and the 0x80 alpha
|
|
scale. The pipeline now writes `gs_ram.bin`, which it had been skipping.
|
|
All 1,595 tfrag textures on all 27 levels decode; 96.8% fully opaque and
|
|
96.3% with real detail. Verified visually via the new
|
|
`gc-extract dump-texture`, because a wrong lookup still yields a
|
|
well-formed image of the wrong thing.
|
|
- [x] **tfrag LOD selection.** Switch distances scale as `[6L, 4L, 2L]` from the
|
|
block header's float; six draw modes result from testing the bounding
|
|
sphere against `trunc(D * 1024)`, so a patch straddling a threshold takes
|
|
the coarser mode and distance never moves back to finer detail. Morph
|
|
weights run 0..1 across each tier's band, and collapse replaces an entry
|
|
with its first parent rather than blending it.
|
|
- [x] **tfrag parent indices.** The four V4_8 arrays are classified by VU
|
|
address, but the addresses collide, and each V4_8 element holds four
|
|
indices so an array is `count * 4` bytes rather than `count`. With
|
|
`unk_06`/`unk_0a` suppressing absent tiers, all 370,349 LOD entries
|
|
across the 27 levels resolve: 29,043 LOD-01 and 341,306 LOD-0, zero
|
|
failures. See `plan/level-assets.md` for the two corrections.
|
|
- [x] `gameplay` — **it is a WAD stream**, exactly like `core_data`; the
|
|
extractor was writing it compressed. Decompressed, level 0 is 1,872,256
|
|
bytes and holds 5 directional light sets. The earlier "does not decode
|
|
with the WAD LZ77 decoder, bit order and header length ruled out" note
|
|
was wrong about the cause: the decoder was right and the input was not
|
|
what it looked like. Corrected in `plan/level-assets.md`.
|
|
- [~] `sound_bank` — the `SBlk` head is now characterised: `"SBlk", 1, 4,
|
|
"DAW\0", 0` constant on 27/27, and `+0x14` is the only varying word, with
|
|
its low 16 bits always zero (`0x0092..=0x0103`). It tracks body length at
|
|
~100 bytes/unit and is **not** a count -- ratio 91.2..110.8, and no stride
|
|
makes it exact. The absence of a stride is asserted too. Container framing parsed (`SoundBankHeader::parse`,
|
|
`parse_sound_bank`); `SBlk` and `DAW` bounds verified on 27/27. The
|
|
**record fields inside `SBlk` are not decoded**, and `DAW` shows no
|
|
detectable stride, consistent with ADPCM. See the note under
|
|
`level_overlay` below for what the game's own strings say about loading.
|
|
- [x] `data` — **not geometry.** It is the level's file descriptor: its
|
|
first table carries the exact byte lengths of `overlay.bin`,
|
|
`core_index.bin` and `gs_ram.bin` at fixed `u32` indices 1, 3 and 5 on
|
|
every level checked. What follows the descriptor is still unknown.
|
|
|
|
- [x] `tfrag` — **complete on the disc side.** Vertex format, topology, strips
|
|
(`TfragStrip`), texture bindings, per-vertex colour and normals, LOD
|
|
selection, parent indices, morphing and collapse. 36 format tests, 9 texture
|
|
tests, 23 engine tests. The morphing is a blend toward the **parent
|
|
vertices'** midpoint, not toward separate morph targets — there are no
|
|
morph targets in this format, which is what the old "morph targets
|
|
unrecovered" note wrongly implied.
|
|
- [x] The tfrag **table record has exactly two relocated pointer fields**, at
|
|
`+0x10` and `+0x38`, verified against PCSX2 across all **833** records: the
|
|
loader adds the **table's** RAM address (`0x848040` = block base plus
|
|
`table_offset`, matching `TfragHeader::data`'s own "relative to the table")
|
|
to `+0x10`; every other byte of every record is verbatim, and 1,665 fields
|
|
differ where 1,666 were possible.
|
|
They are the existing `TfragHeader::data` and `::cube_ofs` -- what RAM adds
|
|
is that both are **pointers**, not plain offsets. All **833** patches parse,
|
|
carrying **43,613** vertex-info entries; **499 of 833** have at least one
|
|
morphing entry.
|
|
- [x] Behind those pointers, **every patch's bytes are rewritten** -- and it is
|
|
**not morphing**. Across all 833 patches: the 334 with **no** morphing
|
|
entries differ by a median **7.80%**, the 499 with morphing entries by
|
|
**8.12%**, and **0 of 833** are byte-identical to disc. Morphing does not
|
|
predict which bytes change. Nor is it a relocation: the differences fall in
|
|
short structured runs with no common `u32` addend. The loader regenerates
|
|
each patch's **VU packet stream for the LOD tier it selected**, which
|
|
happens to every patch regardless of morphing.
|
|
- [x] **The tfrag morph is not a rewrite of the position arrays.** Across all
|
|
**833** patches of level 0, live PINE RAM against the disc: **40,600**
|
|
positions, **40,600 identical, zero changed**. The game does rewrite the
|
|
blocks -- **166,465** bytes differ -- and **none** of them fall inside a
|
|
position unpack: 116,981 land in VIF command words and 49,484 in other
|
|
unpack payloads. The blend is therefore not precomputed into the geometry;
|
|
it happens at draw time in the VU stream, which is why reading a weight back
|
|
out of RAM never worked -- there was nothing there to read. This sharpens
|
|
the earlier "positions live inside regenerated VU packets, so there is no
|
|
stable per-vertex correspondence": the correspondence was never missing, the
|
|
values are just identical. `gc-trace verify-morph` and the
|
|
`GC_SSTATE`-gated test both carry the numbers.
|
|
- [ ] What tfrag is **not** yet verified is that the morph matches
|
|
`resolve_vertex`'s formula. The blend target is still untested against the
|
|
game itself, and now the search space is bounded: it is in the VU stream,
|
|
not in the tfrag data block. Verifying it needs the **camera distance at the
|
|
savestate** so the weight can be computed, or a read of the VU packet stream
|
|
the game actually submits.
|
|
- [x] The tie class header's **`+0x0c` word is the three LOD record
|
|
counts** — LOD 0 at `+0x0c`, LOD 1 at `+0x0d`, LOD 2 at `+0x0e` — verified
|
|
on **1,864 of 1,864** classes by reading the counts independently from
|
|
the blocks' own record tables. This was filed as "R&C1's normals offset,
|
|
read past, not decoded", which was the right action on the wrong map:
|
|
R&C1 carries the same quadruple at `+0x20` and it is not an offset on
|
|
R&C2. The fourth byte is unidentified and is *not* the shader total
|
|
(disagrees on 1,827 of 1,864).
|
|
- [~] The tie block's **located `0xf6` terminator is inside the last packet's
|
|
data** -- the last packet's computed end lands 2 to 16 bytes past it on
|
|
all 2,150 blocks measured. So the block's *extent* is short by that much
|
|
(the `0xf6` test could not see it: a false terminator passes that test).
|
|
The block *starts* are unaffected and RAM-confirmed. The packet lengths
|
|
themselves are verified for every record but the last.
|
|
- [x] `tie` — **fully chained.** `resolve_tie_class` walks both indirections: the
|
|
class table is in `core_index.bin`, and the entry's `offset_in_asset_wad`
|
|
addresses `core_data.bin` (not `core_data.wad`). All **1,864** tie classes
|
|
resolve with `o_class` agreeing on every one. A `packets` word of **zero
|
|
means no block at that LOD** -- 2,177 real blocks, 1,551 absent slots.
|
|
- [x] Tie **blocks are self-delimiting**: `parse_tie_block` walks to the `0xf6`
|
|
terminator that every real block carries (2,177 of 2,177), followed by zero
|
|
padding. This is what makes blocks parseable at all -- `packets[1]` bounds
|
|
one on only 461 of 1,864 classes.
|
|
- [x] The tie **16-byte record is R&C1's `TiePacketHeader`.** Identified not by
|
|
resemblance but by R&C1's own rule, `bfc_distance == 5 * shader_count`,
|
|
holding on **15,219 of 15,219** records across all 2,177 blocks. A block is
|
|
`[16-byte header][record table][vertex payload]`, and the header word at
|
|
`+0x00` is the byte offset from the block start to the payload. The earlier
|
|
"variable-length, not 16-byte records" reading was wrong: the word is not
|
|
an element count, which is why dividing the payload by it gave 0 of 922.
|
|
- [x] The tie **vertex element is 16 bytes**, measured over 95,612 vertices on
|
|
levels 0-2 and settled by comparison: a 12-bit-fixed texcoord hit rate of
|
|
**82.5%** at 16 bytes against 47.8% at 8 and 44.8% at 12.
|
|
- [x] `i16[0..2]` are the **vertex positions**: LOD1/LOD2 of one class share 96.1%
|
|
of their position triples, while LOD1 of A against LOD2 of B shares 0.4% --
|
|
a factor of 240, asserted including the ratio. Axis order and scale unnamed.
|
|
- [x] `i16[3]` is a **byte-sized index**, not a coordinate: 201 distinct values in
|
|
`0..=254` inside the ordinary layout, whose high byte is the layout tag
|
|
(93.4% of elements are `0x00`, 5.6% are `0x03`, 146 selector values in all).
|
|
Asserted.
|
|
- [~] The tie **texcoord field is no longer located.** It was recorded at
|
|
`i16[4]`/`i16[5]` on a 73.6% "both multiples of 4096" score, but that score
|
|
is contaminated twice over: it was computed on the 16-byte reading that
|
|
`e5cd3df` retracted, and measured on all records rather than the flat ones.
|
|
Restricted to the **flat** records, and asking the discriminating question
|
|
-- how many of the slot's distinct values are multiples of 4096 at all --
|
|
gives **16 of 1,488** for `i16[4]` and 16 of 1,261 for `i16[5]`. A 12.4
|
|
fixed-point texcoord would be *all* of them. The dominant values do look
|
|
like UV corners (`(0,0)`, `(0,4096)`, `(4096,4096)`, `(4096,0)`), which is
|
|
why it read as convincing; that is a statement about the most frequent
|
|
values, not about the field -- the same shape as the 305x class-ID
|
|
artifact. The texcoord slot in a flat tie record is **not identified**.
|
|
The earlier note it supersedes read: `i16[4]`/`i16[5]` of the 16-byte
|
|
element, scoring 73.6% for 12-bit fixedness over 36,501 level-0 vertices,
|
|
against 1.1% for `(0,1)` and 4.8% for `(2,3)`. The `(6,7)` pair scores
|
|
*higher* at 76.6% and is **not** a texcoord -- it is the near-constant
|
|
`4096, 0` pair, which is trivially two multiples of 4,096.
|
|
- [~] `i16[3]` was recorded as selecting the texcoord encoding; that rested on
|
|
the same 16-byte reading and is withdrawn with it. See the note above.
|
|
The original text read: it is specifically the
|
|
**high byte**. Grouped across 95,612 vertices on levels 0-2: high byte `0x00`
|
|
covers 89,341 vertices (93.4%) at 92% 12-bit fixed, while **every** other
|
|
high byte scores 0%. The split is on the byte, not the value -- `i16[3]` of 1
|
|
and of 771 differ only in the high byte and behave completely differently.
|
|
Within the all-zero case, `i16[3] == 0` exactly (5,851) is a third
|
|
distribution at 62%. Three cases: `0x00/0x00`, `0x00/1..255`, and non-zero
|
|
high byte.
|
|
- [x] **The tie packet's length follows from its record**:
|
|
`vert_ofs + vert_size + scissor_size` quadwords, on **12,927 of 12,927**
|
|
record pairs whose `strip_count` is zero and **0 of 115** whose it is
|
|
not -- an exact partition, so the formula is safe when `strip_count == 0`
|
|
and knows nothing otherwise. This unblocks slicing the payload per
|
|
packet. It also corrects this file's own retraction of the same formula:
|
|
the "0 of 13,042" re-measurement had swapped `strip_count` for
|
|
`scissor_ofs` and blamed the block set.
|
|
- [~] The remaining tie vertex **fields are not named**, and two readings are now
|
|
excluded. `i16[6]`/`i16[7]` are constant across a whole draw packet in
|
|
43%/34% of records, so they are usually per-packet rather than per-vertex.
|
|
`i16[3]` is per-vertex (50 distinct values per packet) and unshared across
|
|
LODs (75.3% against a 78.7% control). **An earlier note here called it "not
|
|
a joint index" on a 0.95x distance ratio; that was an over-read and is
|
|
withdrawn.** A rigid object bound to joint 0 makes "vertices sharing a
|
|
joint" the whole mesh, so absence of clustering is what a joint index
|
|
predicts. With `MobyInstance.jointCnt` now named, the joint reading is the
|
|
leading candidate, not a disproved one.
|
|
`i16` range with 10,803-13,421 distinct values each. `i16[0..2]` are most
|
|
likely position, but that is not established.
|
|
- [~] `shrub` — class table done (48-byte entries, not 32; reading shrub at
|
|
32 still gives 231 sane-looking entries of 358, which is the hazard).
|
|
Geometry header parsed and checked field by field on all 358 spans:
|
|
`+0x24` equals the entry's own `o_class`, `+0x2c` is an in-span data
|
|
offset, `+0x16`/`+0x18` are zero, and `+0x1c` names a billboard on
|
|
exactly the 25 classes whose core-index entry carries billboard texture
|
|
fields. Shrub does **not** use moby's part-table format. The body is a
|
|
**24-entry palette of unit-length s16 normals** (squared length within
|
|
65,535 of 32767² on all 8,592 entries — a random triple would miss by a
|
|
billion) plus zero padding records; the billboard record carries K, MMIN
|
|
and the TEX0/MIPTBP addresses as R&C1's does. What remains open: the
|
|
three floats in the billboard record, `size`, `mode_bits`, `count`, and
|
|
**where the mesh geometry lives** — R&C1 puts the mesh packets in this
|
|
blob and R&C2's blob holds only the palette.
|
|
- [x] **The tie LOD block is a table of 16-byte records.** The header's word at
|
|
`+0x00` is the byte offset from the block start to the vertex payload --
|
|
the record table's length -- not a count, which is why the payload never
|
|
divides by it. Each record is a leading `u32` pointer **relative to the
|
|
table start** plus twelve verbatim field bytes, relocated at load to
|
|
`table + word`.
|
|
- [x] **The record is R&C1's `TiePacketHeader`.** R&C1's own rule
|
|
`bfc_distance == 5 * shader_count` holds on **all 15,219 records of all
|
|
2,177 blocks on all 27 levels**. R&C1's rule for the colour offset holds
|
|
on **none** of them, so the games use that field differently.
|
|
- [x] **The tie relocation verifies on two levels.** `gc-trace
|
|
verify-tie-records` checks it against PCSX2's RAM: **87 of 87** blocks on
|
|
level 0 and **104 of 104** on level 1, zero mismatches, which is what
|
|
makes the record layout a finding rather than a reading of one block.
|
|
- [x] **The shrub mesh geometry is in the class blob.** A packet table at
|
|
`+0x40` names `count` packets between the table and the palette, and
|
|
they tile on **358 of 358 spans** (2,239 packets), which makes `count`
|
|
the packet count. Confirmed against the running game, which relocates
|
|
the table's offsets to absolute addresses and `data_offset` likewise.
|
|
- [x] `moby` — per-class model geometry. A class span opens with a `u32`
|
|
pointing at a part table of 16-byte records; parts tile exactly, with
|
|
`vertex_offset + 16 * vertex_count == next_part` holding for **43,467 of
|
|
43,467** parts across every class on all 27 levels. That identity is what
|
|
makes the record size fall out of the data rather than being assumed.
|
|
1,270,802 vertices over 4,560 class spans; 116 spans carry no geometry.
|
|
Vertices are 16-byte VIF-packed half-float quads, handed back raw.
|
|
- [x] The **moby class-id space is known**: `lvl.vtbl` (section 3 of the level
|
|
program) is records of `{oClass, handler, auxiliary}`, and across 27 levels
|
|
that is **1,163 distinct ids** reaching `0x13BA` (5,050), median 38 new per
|
|
level. `moby_classes.count` is 238 and **97.6% of real ids are at or above
|
|
it**, so the count was never a valid bound -- which is the mechanism behind
|
|
the chunk class-ID artifact now retracted. Re-scored against the real set,
|
|
no chunk slot separates (best 0.375 against 0.018 chance, a dozen slots in
|
|
the same band). Asserted.
|
|
- [x] **The dispatch tables hold 4,737 behaviours, not 5,822 records.** `lvl.vtbl`
|
|
collapses 5,822 records over 1,162 classes to 4,737 handler addresses, the
|
|
median serving one class and the busiest 34; 400 addresses serve more than
|
|
one. The camera and sound tables are essentially one handler per record
|
|
(217 -> 216, 32 -> 32). A class's behaviour is therefore not one function,
|
|
which matters for any search for a specific class routine. Asserted.
|
|
- [x] **All three dispatch tables decode.** `lvl.vtbl` (3 words), `lvl.camvtbl`
|
|
(5) and `lvl.sndvtbl` (2) each divide their sections **exactly**, with zero
|
|
remainder on all 27 levels -- which is the check that would fail if a record
|
|
width were wrong. 217 camera records carry 14 distinct global ids (max
|
|
`0x1B`), 32 sound records carry 6 (max `0x07`), so both spaces are shared
|
|
across levels rather than renumbered. Handler addresses are function
|
|
entries in the same overlay's `.text`, which is now readable. Asserted.
|
|
- [x] `lvl.vtbl`'s **`oClass` is a global identifier**, not an index into the level's
|
|
class table: 1,163 distinct ids across the game, largest 5,050, against
|
|
`moby_classes.count` of 136..259. **1,133 of 1,163 are out of range** for any
|
|
per-level table. The two are different axes -- a type id, and the models a
|
|
level instantiates -- and the space is shared rather than renumbered
|
|
(1,163 distinct out of 5,878 records; level 0 supplies 20%, not a
|
|
majority). A class can now be recognised across levels by id. Asserted.
|
|
- [x] The **moby instance is named**: `MobyInstance`, 256 bytes, 67 fields,
|
|
transcribed from the decomp's `include/structs.h` into `gc_formats::moby`
|
|
with an offset map. `oClass` at `0xBC` is the dispatch key into `lvl.vtbl`,
|
|
which closes the instance-to-routine link; `animSeq`/`animLayers`/
|
|
`jointCnt`/`jointCache` at `0x40..0x5C` name the instance-side animation
|
|
state. **Not verified against disc bytes** -- mobies are runtime objects and
|
|
nothing of that shape is in the container; the test asserts what can be
|
|
asserted without them (offsets ascend, do not overlap, land exactly on
|
|
0x100). Layout claim from a third party, attributed as such.
|
|
- [ ] `moby_anim` — **unblock attempted and not achieved.** The unblock was "the
|
|
data behind those pointers", and the pointers are RAM addresses, so the
|
|
search moved to the moby class blob: 187 classes on level 0, part-table
|
|
offset `0x80` on 69 and `0x90` on 34, so each carries a `~0x40`-byte header
|
|
before its parts. That header has ~6 constant and ~12 per-class-varying
|
|
fields and **none is yet identifiable** — the varying ones hold values of
|
|
the same order as the span offsets, so the obvious discriminator ("the field
|
|
that points outside the span") does not resolve. Needs the class loader in
|
|
the overlay's `.text`, which is now readable, or a class whose animation is
|
|
known externally. Recorded as a stall with a location, not as progress. An
|
|
instance table would have to scale with the level's moby class count, but
|
|
**per-chunk-type record counts are level-independent** -- the same types
|
|
report 90, 77, 50 and 47 on every level tried, and the largest type
|
|
totals 2,430 = 90 x 27. Nothing in the chunk set is keyed by moby class.
|
|
The types also pair up (two types both reporting 90 on every level,
|
|
another pair both 77), which looks like two views of one fixed table
|
|
rather than two independent tables.
|
|
- [x] `collision` — container framing and header fields. A 0x40-byte header
|
|
self-describes: it carries its own size and the payload size, and
|
|
`header + payload` stays inside `core_data` on all 27 levels. 41,481,520
|
|
payload bytes across 19 levels; 8 declare zero length. Header bytes `+8`
|
|
reads zero on 22 levels and a large value on 7/9/12/13/14; the payload's
|
|
two leading `u16` are non-zero on all 19 non-empty levels (1-74 and
|
|
13-112) and are reported as `CollisionHeader::first`/`::second`.
|
|
- [ ] The collision **cell layout is not decoded.** The payload does not walk
|
|
as fixed-stride cells. On level 4 a table starts at payload `+5932` and
|
|
runs **17 entries** -- deltas rise then fall -- against a 75-cell grid,
|
|
with no arithmetic relating the two. A 64-byte RAM stride looked promising
|
|
and is **disproved** -- the longest such run is 27 records of identity
|
|
matrices, one per level.
|
|
- [x] **The collision payload is essentially verbatim in RAM, not reshaped.**
|
|
Level 0's sits at RAM `0xac6800` for a disc address of `0x27e800`:
|
|
**903,212 of 903,248 bytes identical**, first difference at `+0xdc82c`, with
|
|
**6** bytes differing in the final 36. Asserted by a savestate-gated test
|
|
that compares the whole payload. This **overturns** the earlier claim that the grid dimensions
|
|
are absent from RAM: level 0's `11 x 33` are the first two bytes of that
|
|
block. The earlier negative came from searching for the dimensions as a
|
|
`u32` pair when they are two `u16`s -- a bad pattern, not a property of
|
|
the data. The payload contains ascending `u32` offset tables from `+1024`
|
|
onward, and disc-side decoding is no longer blocked on reshaping. See
|
|
`plan/pcsx2-verification.md`.
|
|
- [x] `occlusion` — the PVS grid. A z/y/x tree of u16 node offsets over 4x4x4-unit
|
|
cells, populated cells naming 128-byte masks. **412,521 cells over
|
|
140,939 masks** across the 24 levels that have one; the mask array ends
|
|
within 48 bytes of the block boundary on every level. 3 carry no occlusion
|
|
data, and an offset of 0 means *absent* rather than "at the start" — the
|
|
latter parses as a grid whose mask array happens to fit, which is how it
|
|
nearly went wrong.
|
|
- [x] `sky` — 149 sub-objects across all 27 levels, each with a count, a
|
|
four-float bounding sphere and texture dimensions. The offset table at
|
|
`+0x20` is self-limiting: entries run while strictly inside the region.
|
|
**The word at `+0x0c` is not a count** -- it disagrees with the table on
|
|
most levels and reads 196,617 on level 2 against 4 sub-objects.
|
|
- [x] The **overlay's load base is `field_00 - 16` = 1,735,792**, derived from
|
|
the overlay's own profiler table (whose absolute addresses point back at
|
|
its names, on all 27 levels) and **confirmed in RAM**: all four profiler
|
|
names land where the base predicts. Any overlay offset can now be carried
|
|
to a RAM address. The overlay is *relocated*, not copied: only 37.67% is
|
|
byte-identical at the base, so tables inside it move.
|
|
- [~] `volumes` — **still not identified, but a candidate exists.** Every level's
|
|
`gameplay.bin` carries a table of **144-byte float records** with world
|
|
positions at `+0x0c`, preceded by an all-zero record (levels 0-9: 23 to 612
|
|
records). That is the shape of a volume trigger region and it is not tied
|
|
to anything else in the file -- but camera bounds, spawn points and path
|
|
nodes would look identical, and there is no engine-side `Volume` string to
|
|
discriminate with. **Recorded as candidate, not volumes.** Two detector
|
|
bugs on the way are written up: a validity test that counted zero as
|
|
world-plausible scored zero-filled memory perfectly, and the 144-byte stride
|
|
rested on four eyeballed offsets that the gap histogram did not support.
|
|
A promising lead also failed: level 0's `data.bin` references one candidate
|
|
offset exactly once (chance ~0.003), but **none** of seven other levels'
|
|
candidates are referenced by their own `data.bin`, so the table is reached
|
|
by walking structure and nothing links it to volumes. Not located. Nothing examined so far presents itself as a
|
|
volume: `core_data`, the chunk region and `gameplay.bin` all yield
|
|
structures that are accounted for by something else.
|
|
- [~] `level_overlay` — header, constant block and level data parsed
|
|
(`gc_formats::overlay`, 27/27). `+0x10..+0x1220` -- 4,624 bytes -- is
|
|
byte-identical on every level; the word at `+0x0c` is **not** the file
|
|
length, falling below it on 17 levels and above on 10. The debug string
|
|
pool it opens with carries `Camera_CollPrimTest WARNING! - grid out of
|
|
bounds! (RAR)`, independent support for reading the collision payload's
|
|
two leading `u16`s as grid dimensions. `+0x00` **is** the load base of
|
|
the section this file is, `0x1A7C80`, constant on all 27 levels because
|
|
every level's `.lit` loads there. `+0x0c` is **not** the `.lit` size -- it
|
|
falls *below* the file length on 16 of 27 levels, by up to 8.9%, so it is
|
|
excluded by sign -- and it is still unnamed. **The remaining header words
|
|
are unnamed and the level-specific tail is not structured yet**; the
|
|
Phase 3 seven-section parser and relocation are untouched, though the
|
|
seven sections are now named (`lvl.vtbl`, `lvl.camvtbl`, `lvl.sndvtbl`).
|
|
- [x] **The post-shrub chunk container.** A 16-byte header, a 10-byte zero gap,
|
|
then offsets tagged `0xF0000000 | low24` resolving to `chunk_start + low`;
|
|
header byte 14 is the record count, equal to `offsets.len()` on 1,123 of
|
|
1,123 chunks across all 27 levels. **77 distinct chunk types, 1,509
|
|
chunks, 34,491 records.** Moby class IDs enrich at 9,493 hits against
|
|
246.1 expected with contiguous per-type arrays -- but the arrays land in a
|
|
different place for each type, so no single pooled scan finds them.
|
|
- [x] **Chunk record framing is closed**, but the uniformity claim in it is
|
|
**retracted**: only **257 of 1,509** chunks have a uniform record length, and
|
|
1,252 are variable within the chunk. "1,509 of 1,509" was `resolve` slicing
|
|
at the chunk's first offset delta and the predicate confirming the parser
|
|
back to itself. An earlier withdrawal ("variable-length was a pooled
|
|
artifact") was wrong too -- the records were always variable. The record's leading byte is a **per-chunk phase
|
|
counter** (`bucket_rule`, 1,262 of 1,509 chunks) and on **1,235** of them
|
|
*both* leading bytes encode the record's own index in mixed radix
|
|
(`index_rule`), so the leading bytes carry no per-record information.
|
|
- [x] **The loader's transformation, from PCSX2 RAM.** Disc offsets
|
|
`0xf0000000 | low24` are rewritten in RAM to absolute EE pointers
|
|
`0xf1850000 | low24` -- tag replaced by the chunk's load base, low 24 bits
|
|
untouched. The payload itself is copied **verbatim**. The index finding
|
|
is corroborated live: the byte at `+2` takes exactly `0, 16, ... 240`, the
|
|
16 values `index_rule` predicts.
|
|
- [x] **Corrected: a record starts two bytes before the table's offset.** The
|
|
`02 00` read in RAM as a loader-inserted prefix is the record's own
|
|
leading word, already on the disc at every record. The loaded table
|
|
simply points two bytes earlier than the disc's, at that word. It is a
|
|
field: uniform across a chunk's records on **1,439 of 1,509** chunks,
|
|
fixed within **all 71 types**, values `{1, 2, 4, 6, 8, 10}`. Disproved
|
|
readings: an entry count (57 distinct fractional "sizes") and a
|
|
distinct-leading-byte count (0 of 1,506 agree).
|
|
- [~] The chunk **record fields are not decoded**, and no type is named -- but the
|
|
stride table is now measured: **74 signatures, 55 distinct strides, 52..640,
|
|
every stride a multiple of 4**, and 73 of 74 signatures use a single stride.
|
|
The exception is pinned: `[75 42 ef f2 71 b5]` at strides `{76, 92, 104}`.
|
|
Asserted including the exception, so "every type is uniform" cannot rot.
|
|
A per-type layout found for one type (48 bytes of data then 60 permanently
|
|
zero) does **not** generalise -- 0 of 9 types fit that shape.
|
|
**65 of the 74 signatures are byte-identical in all 27 levels** (9 are
|
|
level-specific, 0 ragged), so most of the container is static game tables
|
|
rather than level content -- asserted. Consequence for method: the 65
|
|
cannot be decoded by comparing levels, because there is nothing to compare.
|
|
`[75 42 ef]` is both level-specific *and* the three-stride type, so it is
|
|
where per-type decoding should start -- and decoding it found a **parser
|
|
defect**: it makes three chunks per level, one of which has record offsets
|
|
stepping `92, 88, 84, 92, 92, 92, 84, 88` on every level. `resolve` slices
|
|
every record at the chunk's single stride, so four of the eight records
|
|
over-read by 4 and 8 bytes and include the next record's head. Records
|
|
still *start* correctly and bounds checking holds, so it is a data-correctness
|
|
bug, not a safety one. The "strides {76, 92, 104}" description is a
|
|
misdescription of three chunk shapes, one of them variable.
|
|
Measured across **all** chunks rather than that one signature, the pattern
|
|
is the norm: **1,252 of 1,509 chunks have non-uniform record offsets**,
|
|
across 66 of 74 signatures, with every deviation a word-aligned multiple of
|
|
4 and most exactly +/-4. **Which model this is remains open** -- genuinely
|
|
variable-length records and fixed records with inter-record padding produce
|
|
identical offset tables, and the offsets cannot tell them apart. A fix that
|
|
slices each record against its successor was written and reverted: it
|
|
implements one of the two models and the evidence does not establish it.
|
|
It also revealed that three tests asserting "1,509 uniform record length"
|
|
were passing only because the parser forced uniformity by slicing at a
|
|
single stride. Settling it needs a field whose end is independently
|
|
observable. The body
|
|
is dense (74% of `f32` reads plausible against a 49% random control) with
|
|
an index-like tail -- 12 of 163 `u16` slots in the largest type stay below
|
|
`0x1000` -- but **no count field** exists in the first 64 body bytes at any
|
|
width or stride tried. Only **9.4%** of `u16` slots are per-chunk constant,
|
|
so records are not templates.
|
|
- [x] **Moby class arrays are contiguous per type, and the pooled scan was
|
|
wrong.** See rule 3a in `AGENTS.md`: the original claim of a `u16` array at
|
|
slots 129-140 was the largest-stride type dominating an aggregate. Per
|
|
type, every one of the 26 places its array somewhere else.
|
|
|
|
**Gate:** every parser has a golden test against bytes from the real disc, and
|
|
reports mismatched header counts as failures rather than warnings.
|
|
|
|
### Gate status, checked
|
|
|
|
Both halves verified by inspection rather than assertion:
|
|
|
|
* **Golden tests.** Every module in `gc-formats/src` is referenced by a test or
|
|
another module's test path; none is unreferenced. `gc-extract/tests/disc.rs`
|
|
carries 50 real-disc tests, and the `overlay_reloc` mask test resolves a level-01
|
|
function on all 26 other overlays.
|
|
* **Counts are failures, not warnings.** No parser writes warning-style output
|
|
outside `#[cfg(test)]` -- the only `eprintln!` calls in `gc-formats` are inside
|
|
test modules. Count and length problems surface as typed errors: 42 uses of
|
|
`ErrorKind::Invalid` / `InvalidLength`, including `global_wad`'s "pair table is
|
|
not a whole number of entries", which is exactly a header count disagreeing with
|
|
the data.
|
|
|
|
**The gate is met.**
|
|
|
|
### Blocked, and what would unblock each
|
|
|
|
AGENTS.md requires a blocked item to say what would unblock it. These four did
|
|
not, which is itself a gap; this is that gap closed. None of these is a shortcut
|
|
or an approximation -- each is a measurement this repository currently cannot
|
|
make.
|
|
|
|
- **Collision cell layout.** Exhausted from the disc, and the last route is now
|
|
closed on evidence rather than left open. The payload is verbatim in RAM, it
|
|
does not walk as fixed-stride cells, and every disc-side segmentation is
|
|
excluded. The game's own `Camera_CollPrimTest` cannot be reached: the decomp's
|
|
symbol address lands in NOBITS, its assert-table address is a function with no
|
|
pointer arguments, and -- the decisive one -- **the assert string's address
|
|
occurs zero times in the 2,751,268-byte level program**, in either the address
|
|
or `$gp`-relative form, while a positive control in the same search finds the
|
|
`.lit` load base exactly once. The string is dead data; the assertion is
|
|
compiled out in retail. Needs a build with assertions enabled, a debug listing,
|
|
or an independent description of R&C2's collision cell. None is on this disc.
|
|
- **Volumes.** A 144-byte float record table with world positions at `+0x0c`
|
|
exists in every level and is not referenced by any `data.bin`, so it is reached
|
|
by walking structure, and nothing ties it to volumes. Needs ground truth: a
|
|
count of level 0's trigger volumes from a map or guide, to match against a
|
|
table's population. No class name containing "volume" exists on the disc, so the
|
|
dispatch-table route is closed.
|
|
- **`moby_anim`.** The instance-table reading is disproved and nothing replaces
|
|
it. Needs the class-dispatch table for the classes that reference the animation
|
|
data, which needs `moby` class records decoded past the header -- a dependency,
|
|
not a measurement.
|
|
- **tfrag morph.** All patch bytes are rewritten by LOD-selected VU-packet
|
|
regeneration, so the morph is invisible in a savestate. Needs a frame captured
|
|
at a *known* camera distance, so the expected blend is computable and can be
|
|
compared. Requires running the game and driving the camera, not analysing a
|
|
static savestate.
|
|
|
|
**Discipline:** bound every read. Typed errors, no panics, no reads past the
|
|
buffer.
|
|
|
|
---
|
|
|
|
## Phase 3 — Overlay system
|
|
|
|
**Goal:** make 27 level overlays tractable.
|
|
|
|
Every overlay carries the same seven sections (`.lit`, `.bss`, `.data`,
|
|
`lvl.vtbl`, `lvl.camvtbl`, `lvl.sndvtbl`, `.text`) and the same engine code,
|
|
linked at different addresses. ReRAC solves this by masking relocations — `j`/
|
|
`jal` targets, `lui` immediates of main-RAM addresses, the `%lo` that completes
|
|
them, and `$gp`-relative operands — then matching the masked words to find a
|
|
reference function's copy on another overlay.
|
|
|
|
- [x] Port the masking and code-identity logic (`gc_formats::overlay_reloc`,
|
|
`mask` / `address_refs` / `key`). **Verified against the disc**: the masked
|
|
form of a level-01 function (858 words at `0x2a3b88`) is found **exactly
|
|
once** on each of the other 26 overlays, and 0 times on its own level.
|
|
- [x] `Relocation::func` and `Relocation::data`, plus `is_identity`, `same_code`
|
|
and `copies`
|
|
- [x] `$gp` for this game (`0x001AEFF0`, against R&C1's `0x166c00`) -- confirmed
|
|
by `OpenRAC/rac2-gc-decomp` `docs/LEVELS.md` for all 27 levels
|
|
- [x] **The level table has 27 keys, and 21 is not one of them.** ReRAC, which this
|
|
project follows, has `rc_game::map::LEVELS = 19` -- Ratchet & Clank 1's
|
|
count. Reading every container header's own id off the disc gives
|
|
`0..=20, 22..=26, 30`: the files run `LEVEL0.WAD`..`LEVEL26.WAD` but
|
|
`LEVEL21.WAD` reports **30**, so one number was cut and reused by nothing.
|
|
`OpenRAC/rac2-gc-decomp`'s `config/levels/functions.tsv` independently
|
|
yields exactly the same 27 keys with 21 absent across 7,765 function rows --
|
|
ours from container headers, theirs from where functions land across
|
|
overlays. A table keyed by container index would mis-address one level in
|
|
twenty-seven and shift every id-keyed lookup after the gap. Asserted.
|
|
|
|
**Gate:** every level-01 function's relocated copy is found on all 27 overlays;
|
|
cross-checked against an independent cluster hash. **Both halves met.** The
|
|
cross-check is `OpenRAC/rac2-gc-decomp`'s `config/levels/functions.tsv`: 3,452
|
|
functions appear on all 27 levels in 2,984 distinct content fingerprints. Against
|
|
our own masking of the level-0 copies, **our grouping never merges two the
|
|
reference separates -- 0 cases** -- and splits 3,231 pairs the reference keeps
|
|
together, so our clustering is strictly finer rather than contradictory.
|
|
|
|
**Why this is high-value:** it turns "27 overlays of unknown code" into a table
|
|
lookup, and it is the main thing inherited from ReRAC.
|
|
|
|
---
|
|
|
|
## Phase 4 — Engine core
|
|
|
|
**Goal:** draw a frame.
|
|
|
|
- [~] Graphics Synthesizer in wgpu, with the GS's own rules. **`gc_formats::gs`
|
|
now holds three of the four, each with unit tests:** `alpha_to_8` (where
|
|
`0x80` is opaque, and `0x80 & 0x7F == 0` is exactly why it reads as half to
|
|
an 8-bit reader), `clamp01` (CLAMP wrap, with `1.0` kept at the far edge),
|
|
and `mip_level` -- `round(log2(image / on_screen))` clamped to the chain,
|
|
where a *smaller* on-screen size selects a *higher* level. **All four rules
|
|
are now there**, plus `passes_alpha_test` and `BlendMode` for the
|
|
alpha-test and blend halves of the item. Two traps worth naming:
|
|
the GS's `AFAIL` bit reads as "fail if above the reference" and on the
|
|
hardware means the opposite -- it discards where alpha is **below**; and the
|
|
opaque bit must be kept out of the comparison, or `0x80` satisfies every
|
|
threshold and nothing is ever discarded. **The engine is on bevy 0.19, not raw wgpu**, so these rules reach the GPU
|
|
through bevy's sampler rather than wgpu directly. One is now wired: the
|
|
texture sampler was `..default()` with a comment claiming it clamped, and
|
|
bevy's default is `Repeat` -- see `image_of` in `crates/gc-engine`.
|
|
**What the disc says about priorities.** Measured over all 23,168,000 texels of
|
|
every decoded tfrag texture on 27 levels: **97.94% fully opaque, 0.98%
|
|
fully clear, 1.08% partial across 126 distinct values.** So the `0x80 -> 1.0`
|
|
rule is not a corner case -- it is what 22.7 million texels do -- while
|
|
blending touches about 1.08% of them. **These are the same size** -- the cutout
|
|
path handles the 0.98% fully-clear texels, the blend path the 1.08%
|
|
partial -- so an earlier version of this note claiming cutout "matters far
|
|
more" was not supported by the counts it cited. What does make cutout more
|
|
consequential is the *shape* of the failure: a clear texel drawn when it
|
|
should be discarded puts geometry where there is none, which is visible,
|
|
while a partial texel drawn opaque is a slightly wrong soft edge, which is
|
|
not. Cutout first is a claim about which failure you notice, not about how
|
|
many texels are involved.
|
|
|
|
The 126 evenly-populated partial values look like smooth alpha (soft edges,
|
|
foliage, distance fade) rather than a quantised blend ladder.
|
|
|
|
**No draw-path or shader work has started.**
|
|
- [ ] VU0/VU1 skinning and lighting on the PS2 float model
|
|
- [ ] LOD/morph/cull per `TfragProc` and `TieProc`
|
|
- [ ] Occlusion culling, fog, shadows
|
|
- [~] moby_anim: **the loader hunt has a base.** `moby_classes.offset` is
|
|
referenced by 13-72x more level-0 functions than any of its 16 adjacent
|
|
constants, at every immediate-set size -- 72 of 1,082 functions with fewer
|
|
than 16 distinct immediates, where the best neighbour appears once. The
|
|
~300 candidates with 48 or fewer immediates are a real shortlist.
|
|
Disassembly of one (`0x3a8060`) shows `addiu r17, r18, 0xc0` alongside
|
|
`0xd0` and `0xe0` -- a 16-byte-stride table walk at an address offset, with
|
|
the float constant (`lui 0x3f80`) elsewhere. Whether `r18` is
|
|
`core_data + moby_classes.offset` is not yet traced.
|
|
|
|
- [~] GIF/VIF packet stream interpretation. **The tie control region is not a VIF or GIF
|
|
tag stream** -- measured uniform on both framings across 13,042 records, so
|
|
the GS's draw packets are not stored there in standard DMA form. What the
|
|
probe did establish is the packet partition (control / vertices / scissor)
|
|
and a universal invariant, `scissor_ofs == vert_ofs + vert_size`, on 15,219
|
|
of 15,219 records. **`bfc_distance` is also not the control region's length** --
|
|
100% of records exceed it for 2+ shaders, 32% for one -- so the AD-GIF is not
|
|
stored in the tie packet at all. See `docs/plan/level-assets.md`.
|
|
|
|
**Gate:** CPU and GPU paths agree on every vertex of a reference level — the
|
|
same identity check ReRAC runs.
|
|
|
|
**Risk:** VU1 microcode is **binary** in the game's `.vutext` section, so it is
|
|
reverse engineered rather than parsed. This is the hardest single item in the
|
|
project and the main threat to the timeline. Note that ReRAC's own moby VU1
|
|
analysis trips their disassembler on two opcodes.
|
|
|
|
---
|
|
|
|
## Phase 5 — Aranos vertical slice
|
|
|
|
**Goal:** one planet, playable start to finish.
|
|
|
|
- [ ] Ratchet's controller on a **spherical** planet
|
|
- [ ] Chargeboots: hold/release timing, chained jumps, dash
|
|
- [ ] Wrench and the camera
|
|
- [ ] Collision and the HUD
|
|
|
|
**Gate:** Aranos is completable and visually matches PCSX2 frame-for-frame under
|
|
replayed input.
|
|
|
|
**This is the go/no-go for the project.** If Aranos works, the other 26 levels
|
|
are content. If it does not, the reason is discovered while it is still cheap.
|
|
Spherical gravity is the thing most likely to break, because it invalidates the
|
|
movement and camera assumptions the engine inherits from Ratchet & Clank (2002)
|
|
wholesale. That reference is to the *previous game*, not to this disc -- Going
|
|
Commando opens on Aranos, and Veldin appears nowhere in it.
|
|
|
|
---
|
|
|
|
## Phase 6 — Remaining overlays and the shell
|
|
|
|
- [ ] Per-level moby class dispatch via `lvl.vtbl`
|
|
- [ ] Remaining 26 levels
|
|
- [ ] Planets, travel, the Gadgetron vendor, RPG upgrades
|
|
- [ ] Weapon modification bitmasks (lock-on, shock, acid, …)
|
|
- [ ] Minigames and challenges
|
|
- [ ] The 25-state memory-card FSM and its UI
|
|
- [ ] IFF chunk saves, including import of a R&C1 save to unlock legacy weapons
|
|
|
|
---
|
|
|
|
## Phase 7 — Audio and video
|
|
|
|
- [ ] 989snd v2 audio decode and playback (not CRI ADX — a different codec)
|
|
- [ ] SPU2 mixing, reverb zones, level sounds
|
|
- [ ] IPU/MPEG-2 video for the title screen and cutscenes
|
|
|
|
---
|
|
|
|
## Phase 8 — Fidelity pass
|
|
|
|
Differential testing against PCSX2 across every level:
|
|
|
|
- [ ] Recorded-input frame comparison, per planet
|
|
- [ ] Geometry, fog, blend, alpha, texture sampling
|
|
- [ ] Audio sync and 60 Hz timing
|
|
- [ ] Behavioural comparison: traversal routes, boss timing, RNG
|
|
|
|
---
|
|
|
|
## Principles
|
|
|
|
- **A gate is pass/fail, not a percentage.** "Looks fine" is not progress.
|
|
- **Negative results are written down.** A ruled-out approach saves the next
|
|
attempt as much as a working one.
|
|
- **The float model is not negotiable.** Approximate is not the same as
|
|
faithful, and the difference is visible in how the game feels.
|
|
- **Fidelity over tidiness.** Where the original does something that looks like
|
|
a bug, reproduce it and say why in a comment. See `AGENTS.md`. |