diff --git a/docs/roadmap.md b/docs/roadmap.md index 32fcc03..6550aae 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -21,6 +21,19 @@ early, while they are still cheap to retire. header, and the `data` extent is parsed as a container of byte ranges. The geometry decompresses: level 0 yields 16,969,296 bytes of level content from a 9.7 MB stream. +- [x] **Parsers with real-disc tests, per asset type.** `level_wad`, `data`, + `core_index` class tables, `tfrag` (format, topology, textures, LOD, + normals, colour), `occlusion`, `sky`, `moby`, `collision` framing and + header, `shrub` (blob, packets, palette, billboard), the post-shrub + `chunks` container, `sound_bank` framing and `level_overlay`. 227 tests + green against the real disc. **What is open is naming, not locating** -- + see the Phase 2 list for the four types that still have no decoded + fields: `tie` vertices, collision cells, `SBlk` records, `volumes` and + `moby_anim`. +- [x] **Lighting verified against the running game.** PCSX2 2.8.2 with savestates + for level 0 and level 1. The tfrag lighting pass is byte-exact on both + against the game's own RAM. Setup and method in + `plan/pcsx2-verification.md`. - [x] **First drawable frame.** `gc-engine` loads a level, applies LOD selection and morphing for the camera, groups triangles by texture and draws them under a Bevy fly-camera. Level 0 renders as 26,758 textured triangles @@ -215,15 +228,24 @@ Phase 1 hands over the extents; this phase works out what each one contains. with the WAD LZ77 decoder, bit order and header length ruled out" note was wrong about the cause: the decoder was right and the input was not what it looked like. Corrected in `plan/level-assets.md`. -- [ ] `occlusion` — 2 to 25 sectors, small enough to read whole. -- [ ] `sound_bank` — `SBlk` and `DAW` signatures found in all 27. +- [~] `sound_bank` — container framing parsed (`SoundBankHeader::parse`, + `parse_sound_bank`); `SBlk` and `DAW` bounds verified on 27/27. The + **record fields inside `SBlk` are not decoded**, and `DAW` shows no + detectable stride, consistent with ADPCM. See the note under + `level_overlay` below for what the game's own strings say about loading. - [x] `data` — **not geometry.** It is the level's file descriptor: its first table carries the exact byte lengths of `overlay.bin`, `core_index.bin` and `gs_ram.bin` at fixed `u32` indices 1, 3 and 5 on every level checked. What follows the descriptor is still unknown. -- [ ] `tfrag` — terrain: LOD chains, strips, morph targets -- [ ] `tie` — instanced environment geometry, fat vertices +- [~] `tfrag` — vertex format, topology, texture bindings, per-vertex colour + and normals, LOD selection and parent indices are all recovered and + verified. **Still open:** morph targets and the strip/chain bookkeeping. +- [ ] `tie` — class table and the 0x48-byte header agree with R&C1; the + **vertex layout is not decoded**. The class table entries are pointers + into `core_data.wad`, and the header's `packets[0..3]` LOD offsets are + resolved against the lump base -- neither chain has been walked end to + end yet. Level 0 has 74 tie classes, 238 moby and 7 shrub. - [~] `shrub` — class table done (48-byte entries, not 32; reading shrub at 32 still gives 231 sane-looking entries of 358, which is the hazard). Geometry header parsed and checked field by field on all 358 spans: @@ -250,13 +272,30 @@ Phase 1 hands over the extents; this phase works out what each one contains. makes the record size fall out of the data rather than being assumed. 1,270,802 vertices over 4,560 class spans; 116 spans carry no geometry. Vertices are 16-byte VIF-packed half-float quads, handed back raw. -- [ ] `moby_anim` — skeletal animation, still open -- [x] `collision` — container framing. A 0x40-byte header self-describes: it - carries its own size and the payload size, and `header + payload` stays - inside `core_data` on all 27 levels. 41,481,520 payload bytes across 19 - levels; 8 levels declare zero length. The cell layout is **not** decoded - -- the payload does not walk as fixed-stride cells, and variable-length - faces stop partway into the first cell rather than tiling it. +- [ ] `moby_anim` — **still open, and the obvious reading is ruled out.** An + instance table would have to scale with the level's moby class count, but + **per-chunk-type record counts are level-independent** -- the same types + report 90, 77, 50 and 47 on every level tried, and the largest type + totals 2,430 = 90 x 27. Nothing in the chunk set is keyed by moby class. + The types also pair up (two types both reporting 90 on every level, + another pair both 77), which looks like two views of one fixed table + rather than two independent tables. +- [x] `collision` — container framing and header fields. A 0x40-byte header + self-describes: it carries its own size and the payload size, and + `header + payload` stays inside `core_data` on all 27 levels. 41,481,520 + payload bytes across 19 levels; 8 declare zero length. Header bytes `+8` + reads zero on 22 levels and a large value on 7/9/12/13/14; the payload's + two leading `u16` are non-zero on all 19 non-empty levels (1-74 and + 13-112) and are reported as `CollisionHeader::first`/`::second`. +- [ ] The collision **cell layout is not decoded.** The payload does not walk + as fixed-stride cells. On level 4 a table starts at payload `+5932` and + runs **17 entries** -- deltas rise then fall -- against a 75-cell grid, + with no arithmetic relating the two. RAM is reshaped on load: level 1's + `24 x 16` grid dimensions appear in memory, level 0's `11 x 33` appear + nowhere, so disc-side decoding has to work against the reshaped form. + A 64-byte RAM stride looked promising and is **disproved** -- the longest + such run is 27 records of identity matrices, one per level. See + `plan/pcsx2-verification.md`. - [x] `occlusion` — the PVS grid. A z/y/x tree of u16 node offsets over 4x4x4-unit cells, populated cells naming 128-byte masks. **412,521 cells over 140,939 masks** across the 24 levels that have one; the mask array ends @@ -264,8 +303,35 @@ Phase 1 hands over the extents; this phase works out what each one contains. data, and an offset of 0 means *absent* rather than "at the start" — the latter parses as a grid whose mask array happens to fit, which is how it nearly went wrong. -- [ ] `sky`, `volumes`; textures are done (CSM1 CLUT + PS2 alpha) -- [ ] `level_overlay` — the seven-section parser and relocation from Phase 3 +- [x] `sky` — 149 sub-objects across all 27 levels, each with a count, a + four-float bounding sphere and texture dimensions. The offset table at + `+0x20` is self-limiting: entries run while strictly inside the region. + **The word at `+0x0c` is not a count** -- it disagrees with the table on + most levels and reads 196,617 on level 2 against 4 sub-objects. +- [ ] `volumes` — **not located.** Nothing examined so far presents itself as a + volume: `core_data`, the chunk region and `gameplay.bin` all yield + structures that are accounted for by something else. +- [~] `level_overlay` — header, constant block and level data parsed + (`gc_formats::overlay`, 27/27). `+0x10..+0x1220` -- 4,624 bytes -- is + byte-identical on every level; the word at `+0x0c` is **not** the file + length, falling below it on 17 levels and above on 10. The debug string + pool it opens with carries `Camera_CollPrimTest WARNING! - grid out of + bounds! (RAR)`, independent support for reading the collision payload's + two leading `u16`s as grid dimensions. **The header words are not named + and the level-specific tail is not structured yet**; the Phase 3 + seven-section parser and relocation are untouched. +- [x] **The post-shrub chunk container.** A 16-byte header, a 10-byte zero gap, + then offsets tagged `0xF0000000 | low24` resolving to `chunk_start + low`; + header byte 14 is the record count, equal to `offsets.len()` on 1,123 of + 1,123 chunks across all 27 levels. **77 distinct chunk types, 1,123 + chunks, 19,546 records, 50 strides.** Moby class IDs enrich at 9,493 hits + against 246.1 expected with contiguous per-type arrays -- but the arrays + land in a different place for each type, so no single pooled scan finds + them. The record *fields* are not decoded. +- [x] **Moby class arrays are contiguous per type, and the pooled scan was + wrong.** See rule 3a in `AGENTS.md`: the original claim of a `u16` array at + slots 129-140 was the largest-stride type dominating an aggregate. Per + type, every one of the 26 places its array somewhere else. **Gate:** every parser has a golden test against bytes from the real disc, and reports mismatched header counts as failures rather than warnings.