Going-Commando/docs/plan/pcsx2-verification.md

# PCSX2 verification (`gc-trace`)

Status as of **2026-10-11**. The tfrag lighting magnitudes are verified; this
is the file that records how, so the next comparison starts from the
harness instead of from scratch.

Tool: `tools/trace` (`gc-trace`). It never ships, no shipped crate depends on
it, and it reads the game's own memory out of a running emulator, a
savestate, or a raw dump.

## What exists now

```text
gc-trace pine-info                       # protocol handshake
gc-trace dump-ee   <src> [--out FILE]     # 32 MiB EE RAM image
gc-trace find      <src> --bytes HEX      # content search (the locators use it too)
gc-trace read      <src> --addr HEX --len N
gc-trace info      <src>                  # savestate metadata
gc-trace compare-tfrag-light <unpacked-dir> [--level N] <src>
gc-trace verify-morph <unpacked-dir> [--level N] <src>
```

`<src>` is `--pine [slot]`, `--state FILE` / `--state latest`, or `--ee FILE`.
It may sit anywhere on the command line.

The comparison:

1. **Locates the tfrags block in RAM by content** -- the key is tfrag 0's
   bounding sphere, sixteen bytes unique in 32 MiB. A candidate is accepted
   only when the block header matches, **every** header matches the disc on
   the invariant bytes (everything but the relocated `data` pointer and the
   runtime occlusion index), and every `data` pointer is the relocated form
   `table + disc offset`. Anything less is a plausible wrong read.
2. **Locates the light bank the same way**, by its exact section bytes.
3. **Runs the pass** over the same disc data with
   `gc_engine::tfrag_light::light_tfrag`.
4. **Diffs per channel, per vertex**, bucketed by vertex class, with the first
   few mismatching vertices' *terms* (`gc_engine::tfrag_light::
   light_vertex_terms`) printed so a mismatch can be attributed.

## verify-morph: the morph is not in the tfrag data block (2026-10-14)

`verify-morph` answers a different question than `compare-tfrag-light`. It
walks every patch's packet lists **twice** -- once over the disc copy, once over
the RAM copy -- and compares the parsed positions, then diffs the raw bytes of
each patch and classifies every differing byte against the walk's own unpack
ranges.

The result on level 0, live over PINE: 40,600 positions, all identical. The
game does rewrite the blocks -- 166,465 bytes -- and not one lands inside a
position unpack. So the LOD blend is not written back into the geometry.

That sharpens the standing note in the roadmap. "Reading a weight back out of
RAM does not work" was read as the positions being regenerated. They are not
regenerated in *value*: the loader rebuilds the packet framing (116,981
command-word bytes, 70% of the differences) and some other unpack payloads
(49,484), and leaves the vertices alone. The blend happens at draw time in the
VU stream. Nothing was there to read, which is a different finding from there
being a correspondence problem.

## Result: levels 0 and 1, verified 2026-10-11

Level 0 (Aranos), three inputs -- the live emulator over PINE and two PCSX2
savestates from that level, all agreeing byte for byte:

```text
level0: tfrags block at 0x848000, table at 0x848040, 833 tfrags
  headers matching the disc on invariant bytes: 833/833
  data pointers relocated as table + disc offset: 833/833
  light bank in RAM at 0x50ab20, byte-identical to the disc's

vertices compared: 40600 (162400 colour bytes)
  equal:              162400 (100.00%)
  runtime-set: 3130 vertices, 3130 fully equal
  single-set: 37470 vertices, 37470 fully equal
```

Level 1, from savestate slot 2 and cross-checked against the live emulator:

```text
level1: tfrags block at 0x84c000, table at 0x84c040, 401 tfrags
  headers matching the disc on invariant bytes: 401/401
  data pointers relocated as table + disc offset: 401/401
  light bank in RAM at 0x50f4a0, byte-identical to the disc's

vertices compared: 26470 (105880 colour bytes)
  equal:              105880 (100.00%)
  runtime-set: 129 vertices, 129 fully equal
  single-set: 26341 vertices, 26341 fully equal
```

Level 1 matters more than one more tally: it is a different planet with a
different geometry extent, a different light bank and a different
runtime-set population, and it agrees on every byte. The two fixes below are
therefore not level-0 coincidences.

What this establishes:

- **The tfrag locator and the pointer model are right.** 833 of 833 headers
  match on invariant bytes and 833 of 833 `data` pointers are relocated
  exactly `table + disc offset`, so the block found in RAM is the level's and
  the geometry read out of it is the geometry the game drew.
- **The light bank framing is right.** The section found in RAM is
  byte-identical to the disc's: offset at `+0x04`, an `i32` set count, twelve
  bytes of pad, then 0x40-byte sets. The previous reading -- offset at
  `+0x00`, count at `+0x04` -- was wrong, and this is what settled it.
- **The lighting magnitudes match exactly**: every one of the 162,400 colour
  bytes the game wrote equals ours. This was the open item in
  `docs/roadmap.md` and `docs/plan/level-assets.md` ("Lighting magnitudes are
  unverified: they need comparison against reference frames").

## Two bugs the comparison found

Both produced plausible output and neither was visible in a screenshot.

**1. The normal's z was not negated.** `vertex_normal_bits` negated x and y
and left z positive, while the renderer's `NormalTable::normal` negates all
three. The result: `dA = -0.707` where the game had `+0.707`, so
`max(dA, dA·wA)` picked the wrong branch and every vertex lit near its base
colour -- dark, but not *wrong* dark. Level 0 agreed on 95.88% of colour
bytes with the bug and 100.00% without it.

**2. A record with no light set drew transparent black.** `select 0x000f`
names set 15; the level's directional bank has 5 sets, so the pass had no set
to light with and yielded `[0,0,0,0]`. The game writes the record's **base
colour**: 3,130 of level 0's 40,600 vertices do this. The vertex is unlit, not
invisible.

## Finding the next thing

- **Compare another level.** All three inputs available are level 0; another
  level needs a savestate taken in it (F1 in-game, or `gc-trace
  pine-savestate SLOT`). The harness takes `--level N` already.
- **Point lights.** The runtime-set cluster is set 15 and above; those slots
  are filled by gameplay, so they cannot be verified from a level-load image.
  Comparing them needs a frame with point lights in it.
- **Blended sets.** Both levels' vertices are all single-set (`select &
  0xff00 == 0`), so the blend path -- the only path `rsqrt` touches -- is
  untested against the game.
- **Distil, then test.** A comparison is not a regression. When a fact from
  here becomes load-bearing, distil it into a numbers-only fixture under
  `tools/trace/tests/` and test against that; no test may read a savestate, a
  dump or a live emulator.

## The test setup

PCSX2 is installed and working on this machine, so every comparison above can
be re-run rather than being a one-off observation.

| What | Where |
|---|---|
| Emulator | `~/Desktop/pcsx2/pcsx2-v2.8.2-linux-appimage-x64-Qt.AppImage` — the directory name is **lowercase** on disk |
| Game disc | `~/Desktop/pcsx2/games/Ratchet & Clank - Going Commando.iso`; verified target `SCUS_972.68` v1.01 NTSC ([`docs/disc-identification.md`](../disc-identification.md)) |
| Emulator config | `~/.config/PCSX2/` (`sstates`, `bios`, `memcards`, `inis`, `logs`) |
| BIOS | extracted into `~/.config/PCSX2/bios/` |
| Savestates | `~/.config/PCSX2/sstates/SCUS-97268 (38996035).NN.p2s`, `NN` = slot |
| Unpacked disc | `~/Documents/gc-extracted/`, all 27 levels |
| PINE socket | `$XDG_RUNTIME_DIR/pcsx2.sock` (`/run/user/1000/pcsx2.sock`), slot 28011 |

The emulator boots the disc with `-fastboot`; a level takes about a minute
from a cold start to reach. To capture a state manually, press F1 in-game (it
lands in the slot's `.p2s`); to capture from a script, `gc-trace
pine-savestate SLOT` asks the emulator to write one.

The disc integration tests need the image, not the emulator:

```sh
GC_DISC="$HOME/Desktop/pcsx2/games/Ratchet & Clank - Going Commando.iso" \
    cargo test --workspace
```

## Running it

```sh
cargo run -q -p gc-trace -- pine-info
cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 0 --pine
cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 1 \
    --state ~/.config/PCSX2/sstates/SCUS-97268\ \(38996035\).02.p2s
# the tie record relocation, one class and LOD at a time
cargo run -q -p gc-trace -- verify-tie-records ~/Documents/gc-extracted 0 0 1 \
    --state ~/.config/PCSX2/sstates/SCUS-97268\ \(38996035\).01.p2s
cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 0 \
    --state ~/.config/PCSX2/sstates/<name>.p2s
```

Exit status is 0 when every byte matches, 2 on mismatches, 1 on error, so it
can be run as a check. PINE must be enabled in the emulator (Settings →
Advanced); the socket is `$XDG_RUNTIME_DIR/pcsx2.sock` on Linux.

**Nothing here redistributes game data.** The disc, the BIOS and the
savestates are the user's own; the repository holds the tool and the numbers.

## Shrub packet headers are rewritten in RAM: the loader relocates them

Answering the "what is the tag byte" question needed an anchor, and searching
RAM from scratch was the wrong way to get one -- it finds coincidental matches
(`04 04 00 01` matches unrelated data at `0x1ac988`) and no way to tell them
apart. The savestates are the anchor: slot `.01` is level 0, and searching one
gives candidates that can be *checked* rather than guessed at.

**Range measured:** savestate slot 1,
`~/.config/PCSX2/sstates/SCUS-97268 (38996035).01.p2s`, records at `0x22a6b8`
onward, read with `gc-trace read --state`.

**Result: 40+ records beginning `04 04 00 01 00 00 00 00` at `0x30` spacing**,
which is the packet framing identified on disc, and the loader has rewritten
the second half of each header:

    disc:  04 04 00 01 | 00 00 00 00 | 00 00 00 05 | 00 80 0e 6c
    RAM:   04 04 00 01 | 00 00 00 00 | 13 00 00 30 | 70 6a 93 00

Bytes 0-7 are unchanged. Bytes 8-11 and 12-15 are not:

- on disc, bytes 12-13 are the constant `00 80` and byte 15 the constant `6c`;
- in RAM, bytes 12-15 hold what is plainly a pointer -- `0x00936a70`,
  `0x009376f0`, `0x00937850`, `0x00939c50` in consecutive records, all inside
  EE RAM, and ascending across the table exactly as a walk of consecutive
  objects would be.

Byte 11 becomes `0x30` in every record seen, against a constant `0x05` on
disc, and bytes 8-9 hold a varying small value (`0x13`, `0x16`, `0x24`).

**What this means.** The packet table at `+0x40` is the thing the loader walks:
it holds span-relative offsets on disc and absolute RAM addresses after
loading, which is what the previous pass found and what these 40+ records show
directly. Any header field read past byte 8 on disc is describing something the
loader replaces.

**Consequence for the tag bytes.** The values at record bytes 7 and 15 measured
on disc are *not* stable identifiers -- at least byte 15 sits in a field the
loader overwrites with a pointer. The four-value and two-value vocabularies are
real (0x30, 0x6d, 0x80 and 0 appear against 1636 records of otherwise-uniform
bytes), but byte 15 being a pointer in RAM means its disc values are being read
as data by something, not stored as a tag.

**Not settled:** whether byte 7's vocabulary is also rewritten, and whether
`0x6d` marks a payload that the loader expands. That needs the same records
compared one by one against their disc spans, which is the next step and is
now cheap because the anchor exists.

## Following the pointers: packet payloads differ by type

The header pointers found above are live, so they can be dereferenced. Two
consecutive records from the level 0 table at `0x22a6b8` were followed.

**Range measured:** savestate slot 1, reading `0x00936a70` (from the record at
`0x22a6b8`) and `0x009376f0` (from the record at `0x22a6e8`), 64 bytes each.

**Result: the two payloads have nothing in common.**

`0x00936a70` -- fixed-point vertex data:

    8d 8c 80 80  89 87 7c 80  8d 8c 80 80  89 87 7c 80
    4e 4e 4c 80  4e 4e 4c 80  4e 4e 4c 80  4e 4e 4c 80   (repeating)

Every high byte is `0x80` or `0x7c`, and the pattern repeats on a 8-byte
boundary. Read as `s16` these are large negatives in a narrow band around
-32500. That is consistent with Q16.15 fixed-point rather than `f32`, and with
a stride of 4 `s16` per vertex.

`0x009376f0` -- a tagged sub-list, not vertex data:

    00 00 00 30  cd 00 00 00  cd 00 00 00  cd 00 00 00
    cd 00 00 00  01 00 00 05  22 c1 01 6e  00 00 00 00

Structurally different: small integers, a repeating `0xcd`, and `0x30` tags in
the high byte. Nothing here is position data.

**What this supports.** Two records 0x30 apart in the same table hold payloads
of entirely different kinds. That is direct evidence for the type reading of
the header: the table is a tagged collection where the tag selects what the
payload is, not a uniform array of one thing.

It also rules out the alternative I could not previously exclude -- that the
16-byte tiling was a single uniform record format and the varying bytes were
all fields of it. Two adjacent records with unrelated payloads is not that.

**Still open.** The mapping from tag value to payload kind. The `0x30` that
appears in record bytes 8-11 in RAM also appears at the head of the second
payload, which is suggestive but is one coincidence so far and should not be
written up as a link until it is seen in more than one record.

## The tag values map onto three payload kinds

Following the pointers out of the level 0 table, all nine that carry the `0x30`
tag at byte 11 and an EE pointer at 12..15 were dereferenced and read 32 bytes
each. They fall into exactly three classes, three payloads each, with no
overlap.

**A -- Q16.15 fixed-point vertex data** (3 of 9)

    00936a70  8d 8c 80 80 89 87 7c 80 8d 8c 80 80 89 87 7c 80
              4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80
    0093cde0  8a 8a 7f 80 92 90 84 80 8a 8a 7f 80 92 90 84 80
              4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80
    009cf4f0  6e 6f 68 80 6e 6f 68 80 6e 6f 68 80 6e 6f 68 80
              54 54 51 80 54 54 51 80 54 54 51 80 54 54 51 80

Every high byte is `0x80` or `0x7c`, each payload repeats on an 8-byte
boundary, and the values differ between payloads while the `4e 4e 4c 80` tail
is shared by two of them.

**B -- a `0x30` sub-list with a repeated count** (3 of 9)

    009376f0  00 00 00 30 cd 00 00 00 cd 00 00 00 cd 00 00 00
              cd 00 00 00 01 00 00 05 22 c1 01 6e 00 00 00 00
    00998880  00 00 00 30 c0 00 00 00 c0 00 00 00 c0 00 00 00
              c0 00 00 00 01 00 00 05 11 c1 01 6e 00 00 00 00
    009d2da0  00 00 00 30 c8 00 00 00 c8 00 00 00 c8 00 00 00
              c8 00 00 00 01 00 00 05 1d c1 01 6e 00 00 00 00

Structurally identical apart from the repeated word: `0xcd`, `0xc0`, `0xc8`.
All three carry `01 00 00 05` at offset 16 and a `.. c1 01 6e` at 20.

**C -- an index/attribute list** (3 of 9)

    00939700  00 c0 05 6d 3e 00 00 00 01 00 00 00 12 00 00 00
              18 00 c0 00 fe 00 fe 00 ff 00 ff 00 11 01 17 01
    009c1540  00 c0 05 6d 36 00 00 00 01 00 00 00 08 00 00 00
              13 00 93 00 c9 00 c9 00 ca 00 ca 00 d2 00 d5 00

Both open `00 c0 05 6d`, then a count, then a run of small ascending `u16`.

**The mapping.** The tag vocabulary measured on disc -- `0x00`, `0x30`,
`0x6d`, `0x80` at record byte 7 -- is not decorative:

| tag | payload |
|---|---|
| `0x30` | class B, the `00 00 00 30` sub-list |
| `0x6d` | class C, the `00 c0 05 6d` attribute list |
| `0x00` | class A, the fixed-point vertex data |

`0x30` heads the class B payloads and `0x6d` heads the class C payloads, which
is the link flagged as a single coincidence two entries back. It is now seen in
three payloads each, so it is a rule rather than an accident.

`0x80` did not appear among these nine. Its meaning is still open -- it is the
fourth value in the vocabulary and there is no sample of it here.

**A correction.** The record spacing in this table is not uniformly `0x30`. The
`find` output shows `0x30`, `0x10`, `0x30`, `0x30` between consecutive packet
headers, and only 9 of the 5,632 bytes scanned carry both the `0x30` tag and an
EE pointer. The earlier "records at 0x30 spacing" was inferred from a truncated
list of hits and does not hold.

## Correction: the record tag does not select the payload

The previous entry mapped the disc tag vocabulary onto the three payload
classes. A wider scan says that is wrong.

**Range measured:** savestate slot 1, RAM `0x22a000..0x24037f` -- 91,008 bytes,
20 times the span scanned before.

**Result: 19 packet headers, and all 19 carry byte 11 = `0x30`.** There are no
`0x6d` or `0x80` tags anywhere in this range. Reading six of their payloads:

    rec 0022a6b8  tag 30 -> 00936a70  8d 8c 80 80 89 87 7c 80 ...   (fixed point)
    rec 0022a6e8  tag 30 -> 009376f0  00 00 00 30 cd 00 00 00 ...   (sub-list)
    rec 0022a6f8  tag 30 -> 00937850  92 90 84 80 94 94 86 80 ...   (fixed point)
    rec 0022a728  tag 30 -> 00939c50  00 00 00 30 c0 00 00 00 ...   (sub-list)
    rec 0022a738  tag 30 -> 00939e90  4c 4c 4b 80 4c 4c 4b 80 ...   (fixed point)
    rec 0022a768  tag 30 -> 0093cc70  00 00 00 30 78 00 00 00 ...   (sub-list)

Records of identical tag alternate between two payload kinds. **The tag does
not select the payload.**

**What went wrong.** The previous entry took the `0x30` and `0x6d` that *head
the payloads* to be the disc-side tag vocabulary, and matched them to the
payload classes by that correspondence. The disc vocabulary measured at record
byte 7 -- `0x00`, `0x30`, `0x6d`, `0x80` -- and the RAM record byte 11 were
treated as the same field. They are not: the payload magics are a property of
the payload, and record byte 11 is a different thing that happens to be
constantly `0x30` here.

Three payloads each made the coincidence look like a rule. It was a
correspondence between two vocabularies that overlap in two values.

**What is actually established about the RAM header**, per position:

| byte | value |
|---|---|
| 0-3 | `04 04 00 01`, constant |
| 4-7 | zero, constant |
| 8 | a **count**, `0x0b` to `0x29` across the 19 records |
| 9-10 | zero, constant |
| 11 | `0x30`, constant on all 19 in this range |
| 12-15 | an EE RAM pointer |

Byte 8 is the finding that survives. It is a small count that varies per
record, and it is the one field here that is doing arithmetic rather than
tagging.

**Still open:** what `0x30` at byte 11 means, what discriminates the payload
kinds given that the record tag does not, and the meaning of `0x80`, which has
no sample in 91 KB of RAM.

## The table interleaves two record magics

Scanning the same 91 KB for other four-byte record magics beside `04 04 00 01`
turns up a second one at the same density: **`02 01 00 01`, 10 occurrences**
against `04 04 00 01`'s 19.

**Range measured:** savestate slot 1, RAM `0x22a000..0x24037f`.

**Result: the two magics interleave.** Every `02 01 00 01` sits between two
`04 04 00 01` records:

    0x22a6b8  04 04 00 01   13 00 00 30  70 6a 93 00
    0x22a6c8  02 01 00 01   19 c0 4c 6e  00 00 10 00  00 00 00 00
    0x22a6e8  04 04 00 01   16 00 00 30  f0 76 93 00
    0x22a6f8  04 04 00 01   16 00 00 30  50 78 93 00
    0x22a708  02 01 00 01   1e c0 58 6e  00 00 10 00  00 00 00 00
    0x22a728  04 04 00 01   24 00 00 30  50 9c 93 00
    0x22a738  04 04 00 01   16 00 00 30  50 9e 93 00
    0x22a748  02 01 00 01   19 c0 54 6e  00 00 10 00  00 00 00 00

The `02 01 00 01` records are 16 bytes like the others but carry **no pointer
** -- bytes 12-15 are zero in all ten. Their fields are a count at 4-5
(`0x14`, `0x19`, `0x1e`), a constant `0xc0` at 6, a varying byte at 7
(`0x30`, `0x4c`, `0x54`, `0x58`), `0x6e` at 8-9, and `10 00` at 10-11.

**The `0x6e` link.** These records carry `0x6e` at bytes 8-9, and the sub-list
payloads found earlier carry `.. c1 01 6e` at offset 20-23. That is a second
correspondence between this record type and the sub-list payloads, independent
of the one retracted in the previous entry -- and this one is a shared *value*
rather than a shared *position*.

**What this does not settle.** It still does not say what discriminates the two
payload kinds among the `04 04 00 01` records, because those all carry the same
magic, the same `0x30` at byte 11 and differ only in the count at byte 8. The
`02 01 00 01` records sit between them rather than replacing them, so they are
probably a companion structure, not the missing discriminator.

Still open: what distinguishes a fixed-point payload from a sub-list payload
when the records carrying them look identical in bytes 0-3 and 11. The most
likely place to look is the count at byte 8 against the payload's own length.

## The count at byte 8 sizes the sub-list payload: exactly 16 bytes per record

The previous entry left one test: compare the count at byte 8 against the
payload's own extent. Pointer spacing gives the extent, since the packets are
laid out in ascending order in EE RAM.

**Range measured:** savestate slot 1, the 17 packets in RAM
`0x22a000..0x24037f`, each with its record address, byte-8 count and pointer,
sorted by pointer so the next packet is the next object in memory.

**Result: eight packets sit exactly `count * 16` bytes after their predecessor
-- an integer ratio of 16.00 in every one, with no rounding.**

    record      count      ptr   gap   gap/count
    0x22a6e8       22  009376f0    352    16.00
    0x22a728       36  00939c50    576    16.00
    0x22a768       23  0093cc70    368    16.00
    0x22a7a8       36  00998880    576    16.00
    0x22a7e8       25  009c19c0    400    16.00
    0x22a828       24  009cf370    384    16.00
    0x22a868       41  009d2da0    656    16.00
    0x22a8a8       21  009f5aa0    336    16.00

Eight exact hits at a ratio that has to be an integer is not sampling luck. The
count at byte 8 is the **record count**, and these payloads are 16 bytes per
record.

**And these are exactly the sub-list payloads** -- `0x009376f0`, `0x00939c50`,
`0x0093cc70`, `0x00998880`, `0x009d2da0` were all read directly in earlier
passes and all open `00 00 00 30` followed by a repeated word.

**What the other nine are.** Their ratios are 168, 419, 559, 31288, 7985,
3458, 764, 6452 -- no shared factor, and far too large to be a record count
times anything. So either their payload is not followed immediately by the next
packet in the pointer order (other data intervenes), or they are not 16 bytes
per record. Both readings are open; what is ruled out is that they follow the
same rule.

**This resolves byte 8.** It was measured as "a count, 0x0b to 0x29" two entries
ago and is now shown to size a payload exactly, in eight independent cases,
within a single level. Whether it also sizes the fixed-point payloads is the
open half.

Still open: why the fixed-point packets do not follow the same rule, and what
the 16 bytes per sub-list record are -- the repeated word (`0xcd`, `0xc0`,
`0xc8`, `0x78`) differs per packet, so the records are not identical.

## Inside a sub-list payload: tagged blocks and a byte permutation table

With the extent known (`count * 16`), a whole sub-list payload can be read.
This is the packet at `0x009376f0`, count 22, 352 bytes, in full.

**Range measured:** savestate slot 1, `0x009376f0`..`0x0093784f`, 352 bytes.

**Result: the payload is a sequence of blocks, each opening `00 00 00 30`,**
carrying a repeating word and then a short trailer:

    00 00 00 30 | cd 00 00 00 cd 00 00 00 cd 00 00 00 cd 00 00 00
                01 00 00 05 | 22 c1 01 6e | 00 00 00 00
    00 00 00 30 | 00 00 00 45 00 00 00 45 1d 00 00 00 1d 00 00 00 1f
                81 01 6d 00 | 00 00 00 00 | 00 00 a4 00
    00 00 00 30 | bc c0 03 00 23 17 03 00 e5 dd 00 00 00 00 00 00
                02 01 00 01 | c1 80 01 69 | 00 00 00 00

The repeating word differs per block -- `0xcd`, `0x45`, `0x03` -- and the
trailer's second word varies in its high bytes: `22 c1 01 6e`, `81 01 6d 00`,
`c1 80 01 69`. Every block seen ends `... 6e` or `69` or `6d` at byte 23, which
is the same `6e` the `02 01 00 01` records carried at bytes 8-9.

**The payload contains a byte permutation table.** At `0x009377c4`:

    00 01 02 03 3e 3f 53 54 40 41 42 43 4e 4f 50 51 04 05 06 07 08 ...

This is not a position or vertex list -- it is a run of ascending byte values
with a handful (`3e 3f`, `53 54`, `40 41 42 43`, `4e 4f 50 51`) lifted out of
order ahead of the sequential run. Read as a table, it maps each of the 0..0x39
symbols to a slot. That is the shape of a **code-length or symbol-ordering
table**, and it is the first thing in this format that is self-evidently a table
rather than data to be interpreted.

**What this does not say.** Whether the table is a Huffman-style code, an index
reorder for the vertex data that follows, or a palette permutation. The
following bytes continue the sequence to `39` and then `23 c1 01 6e`, after
which the next block opens.

**Corroboration worth noting.** The fixed-point payload `0x00937850` begins
immediately after this one ends at `0x00937850` exactly -- the 22-record
identity from the previous entry made the boundary exact, and the next packet
starts on a payload boundary with nothing in between. The two payload kinds sit
adjacent in RAM, which is consistent with them being two records of the same
blob rather than two unrelated objects.

## The sub-list payload has a fixed five-block shape

All eight sub-list payloads from level 0 were read to their exact extent
(`count * 16`) and compared block by block. The structure is the same in every
one.

**Range measured:** savestate slot 1, the eight packets at `0x009376f0`,
`0x00939c50`, `0x0093cc70`, `0x00998880`, `0x009c19c0`, `0x009cf370`,
`0x009d2da0` and `0x009f5aa0`, counts 21 to 41, each truncated to exactly
`count * 16` bytes.

**Result: every payload contains exactly five blocks tagged `00 00 00 30`, and
the first three are always at offsets 0, 32 and 64.**

| payload | count | blocks at |
|---|---|---|
| `009376f0` | 22 | 0, 32, 64, 184, 304 |
| `00939c50` | 36 | 0, 32, 64, 268, 400 |
| `0093cc70` | 23 | 0, 32, 64, 200, 288 |
| `00998880` | 36 | 0, 32, 64, 272, 404 |
| `009c19c0` | 25 | 0, 32, 64, 196, 300 |
| `009cf370` | 24 | 0, 32, 64, 204, 316 |
| `009d2da0` | 41 | 0, 32, 64, 308, 448 |
| `009f5aa0` | 21 | 0, 32, 64, 176, 280 |

Blocks 4 and 5 move with the payload; the first three do not. Eight payloads,
eight different block-4/5 positions, and 0, 32, 64 in every case.

**Per-block fields, constant across all eight:**

| block | byte 4 | other |
|---|---|---|
| 0 | varies | `0x78`-`0xcd` |
| 1 | **0** in 8/8 | trailer `00 00 00 00` in 8/8 |
| 2 | varies independently | bytes 16-20 `00000000` in 8/8 |
| 3 | **equals block 0 in 8/8** | |
| 4 | **0** in 8/8 | |

The block-0/block-3 equality is exact in all eight:

    009376f0  0xcd  0x00  0xbc  0xcd  0x00
    00939c50  0xc0  0x00  0xbf  0xc0  0x00
    0093cc70  0x78  0x00  0x30  0x78  0x00
    00998880  0xc0  0x00  0xc9  0xc0  0x00
    009c19c0  0x93  0x00  0x1f  0x93  0x00
    009cf370  0xb5  0x00  0x0d  0xb5  0x00
    009d2da0  0xc8  0x00  0x65  0xc8  0x00
    009f5aa0  0x9b  0x00  0x25  0x9b  0x00

Block 2's word is unrelated to block 0's in every case, so the equality is not
every block reading a shared field.

**Correction to the previous entry.** The "byte permutation table" reported
there does not generalise. It appears in `0x009376f0` at offset 212, but a
scan for it across all eight payloads finds no run of more than a few bytes
outside that one. What looked like a table is better described as one payload's
block content. The `00 00 00 30` block structure, the fixed 0/32/64 offsets and
the block-0/block-3 equality are the findings that survive eight-way
comparison; the permutation table was not among them.

**Not established.** What the five blocks are, why blocks 0 and 3 carry the
same word, and what moves blocks 4 and 5 -- their offsets are not monotone in
the count (`count` 21 gives 280 and `count` 36 gives 400 and 404), so something
inside the earlier blocks sets them.

## Partitioned across two levels: the sub-list shape is universal, the payload kinds are not mixed

The five-block result was measured on level 0 only. Rule 3a asks for the same
measurement per partition, so it was repeated on level 1's savestate before
being written up as a property.

**Range measured:** savestate slot 2 (level 1), RAM `0x22a000..0x28f52f`,
415,024 bytes, then six of its pointer-bearing records read to `count * 16`.

**The six sampled pointers split 3/3 between the two payload kinds:**

| pointer | count | first four bytes | classification |
|---|---|---|---|
| `00949430` | 41 | `00 00 00 30` | sub-list |
| `009496c0` | 13 | `69 b0 bf 80` | fixed-point |
| `00957970` | 36 | `00 00 00 30` | sub-list |
| `00957bb0` | 10 | `58 7e 8d 80` | fixed-point |
| `0095b740` | 12 | `37 49 41 80` | fixed-point |
| `0095d6d0` | 57 | `00 00 00 30` | sub-list |

Taking all six as sub-list gives 3/6 with five blocks. Taking only the three
that actually open `00 00 00 30` gives **3/3**, and the first three blocks land
at 0, 32, 64 in all three:

    00949430  41  blocks at 0, 32, 64, 308, 428
    00957970  36  blocks at 0, 32, 64, 268, 376
    0095d6d0  57  blocks at 0, 32, 64, 400, 572

The three fixed-point payloads have **zero** `00 00 00 30` blocks, as
expected. So the earlier 3/6 was my sampling, not a counterexample: the six
pointers were never all the same kind of object.

**Combined: 11 of 11 sub-list payloads across both levels have exactly five
blocks, the first three at offsets 0, 32, 64.**

**The fixed-point payloads have an exact signature.** Byte 3 of every 4-byte
group is `0x80` in 100% of cases -- 52/52, 40/40 and 48/48 over the first 512
bytes of the three level 1 samples, and the same holds across all nine level 0
payloads. That is a clean one-bit-per-group marker and is what separates the
two payload kinds without needing the record tag, which the previous entries
established does not select them.

**Level 1's record tags are not level 0's.** Scanning the same 64 KB for
`04 04 00 01` headers gives 18, against 19 on level 0, plus 6 `02 01 00 01`
against 10. And byte 11 is no longer constant: `0x30` on 10, `0x00` on 5,
`0x20` on 2 and `0xbc` on 1. Level 0 had `0x30` on all 19.

That corrects an earlier statement. "Byte 11 is `0x30`, constant" was a
level-0 partition result written up as universal. Across both levels it takes
four values. The constancy was never a property of the format.

**Fixed-point payload entropy**, all nine level 0 payloads read 4,096 bytes:
5.51 to 5.89 bits per byte, against 8.0 for uniform. Structured rather than
random, and `mean` of the `s16` reading varies per payload from -82 to -907,
which is consistent with vertex positions relative to a per-payload origin.

**Not established.** What the five blocks are, and how the fixed-point data
decodes to vertices -- the payload entropy is far below random, so it is
compressed or permuted rather than stored plainly, and the permutation table
seen once in `0x009376f0` remains unexplained.

## The loader produces the RAM forms: the disc payload is neither of them

Everything above about five-block sub-lists and fixed-point payloads was read
out of the running game. Checking the same packets on disc settles what the
disc actually stores.

**Range measured:** level 0, all 29 packets named by the `+0x40` tables of the 6
bounded spans, read at their span-relative offsets; 1,636 whole 16-byte records
after the 16-byte header.

**Result: neither RAM form exists on disc.**

| | disc | RAM |
|---|---|---|
| `00 00 00 30` per sub-list packet | **exactly 1** | exactly 5 |
| sub-list blocks at 0, 32, 64 | **0 of 10** | 11 of 11 |
| fixed-point byte 3 == `0x80` | **0 of 19** | 100% |

The 29 packets split 10 sub-list / 19 fixed-point by whether they contain
`00 00 00 30` at all, and not one of the ten has the block structure, and not
one of the nineteen has the fixed-point marker.

**What the disc body actually is.** A compact 16-byte record stream:

    +000: 04 04 00 01 | 00 00 00 00 | 00 00 00 05 | 00 80 0e 6c
    +010: 02 00 00 00 | 05 00 00 00 | 32 00 00 00 | 0e 00 00 00
    +020: 09 00 00 00 | 00 40 3e 30 | 12 04 00 00 | 05 00 00 00
    +030: 09 00 00 00 | 00 40 3e 30 | 12 04 00 00 | 21 00 00 00

Small `u32` fields and a repeating `00 40 3e 30 12 04 00 00`. Per position over
all 1,636 records the same two outliers hold as on the earlier disc pass: byte 7
takes 4 distinct values and byte 15 takes 2, against 101 to 153 for every other
position.

**And byte 7 is per-record, not per-packet.** The first record of all 29
packets has byte 7 = `0x00`. So the four-value vocabulary is a field that varies
within a packet body, not a tag on the packet -- which is why grouping packets
by it never separated the two kinds.

**Consequence for Phase 2.** The packet payload is encoded on disc and expanded
at load, so the disc parser can bound it and check its signature but cannot read
it as either RAM form. Decoding it means reversing the expansion, not
re-describing what the loader produced. That is a larger job than the framing
work so far, and it is the honest boundary of what is currently known.

## Collision: a 64-byte record stride in RAM

Searching the savestates for the collision grid dimensions as a `u32` pair --
level 0's `11 x 33` and level 1's `24 x 16` -- puts level 1's at several RAM
addresses and level 0's nowhere. So the grid dimensions are **not held in RAM as
adjacent `u32`s**, and the collision payload is reshaped on load, as the shrub
packet tables are.

A nearby region of level 1's RAM does show structure worth recording. At
`0x12e048c` and `0x12e04cc`, 64 bytes apart:

    28 01 00 00 | 92 ff 00 00 | 04 00 a0 41 | ...
       296        -110        20.0f

Searching for that shared 12-byte tag finds a **consecutive run at a 64-byte
stride** in slot 2:

    0x12df390
    0x12df3d0   +0x40
    0x12df410   +0x40

plus scattered singles elsewhere. Slot 1 has the tag at four addresses that are
*not* on a 64-byte stride, so the two levels lay it out differently.

**What this is.** A 64-byte record carrying two small signed values and a float
that reads exactly 20.0. That is the shape of a per-cell entry with coordinates
and a radius or half-extent, which is what the collision grid would hold.

**What this is not.** It is not decoded. Three consecutive records is not enough
to establish a stride, the tag matches only a few records rather than a whole
grid, and `20.0` may be coincidence. Level 1's grid is 384 cells; nothing here
has been shown to count to 384.

**Method note.** Savestates are compressed on disk, so a byte scan of the file
finds nothing. All RAM search must go through `gc-trace find --state`, which
decompresses first -- scanning the file directly reports zero hits for a
sequence that demonstrably exists in memory.

### The 64-byte stride was not the collision grid

A 415 KB RAM dump of level 1 around the tag above, scanned locally for 64-byte
strides sharing a header, finds the longest run at **27 records, not 384** --
and those records are:

    u32 [1065353216, 0, 0, 0, 0, 1065353216, 0, 0]
    f32 [1.0,       0, 0, 0, 0, 1.0,       0, 0]

Identity matrices, **exactly 27 of them** -- one per level. So it is a per-level
transform table, not the collision grid, and the `20.0` reported above was a
coincidence inside a short run.

**The collision lead in this file is disproved, not merely unproven.** Recorded
here so it is not picked up again. Level 1's grid is 384 cells and nothing found
so far counts to it.

## The chunk container: what the loader does to it

This is what RAM was needed for. Nothing about the loader's transformation is
recoverable from the disc.

**Anchor.** A 32-byte probe from inside a chunk record in level 0's `core_data`
was searched in the level-0 savestate. It hits at `0x18538a8`, and the record's
first bytes hit at `0x185387e` -- exactly `chunk_base + record_offset`, with no
rescaling. So the chunk payload is copied **verbatim**; it is not expanded or
repacked the way the collision payload and the shrub packet tables are.

**Result 1 -- the offset table is rewritten to absolute EE addresses.**
Disc, at chunk `0x100b302`:

    offsets[0..3] = f000057c f00006b0 f00007e4 f0000914

RAM, in a table at `0x185331c`:

    0xf185387c  0xf18539b0  0xf1853ae4  0xf1853c14

The tagged form `0xf0000000 | low24` becomes `0xf1850000 | low24`: the tag is
replaced by the chunk's load base and the low 24 bits are untouched. The
consecutive deltas agree exactly (`0x134, 0x134, 0x130`, which is why the
disc-side `stride` is approximate rather than constant).

**Result 2 -- every record gains a constant 2-byte prefix, and the table points
at the prefix.** RAM at each record:

    0x185387c  02 00 00 00 38 d3 4c 38 ...
    0x18539b0  02 00 10 00 2f bb 4c 38 ...
    0x1853ae4  02 00 20 00 20 8f 4c 38 ...
    0x1853c14  02 00 30 00 14 6f 4c 38 ...

Walked all **90** records of the chunk from the RAM dump:

    2-byte prefix                          : 02 00 on 90 of 90
    disc index bytes preserved at +2/+3    : 90 of 90
    byte at +2, distinct values            : 0, 16, 32, ... 240 (16 values)

**Result 3 -- the disc-side index finding is confirmed against live data.** The
bytes at `+2` take exactly the 16 values `ResolvedChunk::index_rule` predicts for
`step = 16, phase = 0`, stepping 0x00, 0x10, 0x20, ... That is the mixed-radix
record index, measured on 2,430 records from the disc and now corroborated on the
running game's own copy of one chunk.

**What this means for the format.** On disc, a chunk is
`[16-byte header][10 zero bytes][tagged offset table][records]`. In RAM it is
`[16-byte header][10 zero bytes][absolute-pointer table][records each prefixed
with 02 00]`. The `02 00` is unexplained -- it may be a record type or an
alignment the loader needs -- but it is constant, so it carries no per-record
information either.

## The collision payload is verbatim in RAM -- and the earlier "not in RAM" was wrong

**This overturns the premise the collision work was blocked on.**

Searching the level-0 savestate for a probe from collision payload `+1024` hits at
`0xac6c00`; a probe from `+64` hits at `0xac6840`. The difference is 0x3C0 = 960,
which is exactly 1024 - 64. So the payload sits at a fixed relative address --
RAM `0xac6800` for the payload whose disc address is `0x27e800` -- and reading
that address back gives byte-for-byte the disc's own bytes:

    RAM  0xac6800  0b 00 21 00 b6 08 ad 08 a4 08 6b 08 38 08 63 07
    disc 0x27e800  0b 00 21 00 b6 08 ad 08 a4 08 6b 08 38 08 63 07

**Nothing is reshaped -- except the last 36 bytes.** The test that now asserts
this compares the *whole* payload rather than the 4 KiB head the original
measurement used, and the finer comparison gives a more precise answer:

    payload                903,248 bytes at RAM 0xac6800
    first difference       +0xdc82c, i.e. 903,212 bytes in
    bytes that differ      6  (0.00%)

So **903,212 of 903,248 bytes are verbatim and 6 are not**, all of them in the
final 36. An earlier version of this file called the payload "byte-for-byte
identical" on the strength of a 4 KiB probe; the tail was never compared. Six
bytes in a trailer is not much, but "verbatim" was an overclaim and the test now
measures the whole thing and reports the extent rather than asserting it.

### The correction

The earlier note here reads: "level 1's `24 x 16` grid dimensions appear in
memory, level 0's `11 x 33` appear nowhere, so disc-side decoding has to work
against the reshaped form."

**Level 0's dimensions are in RAM, at `0xac6800`** -- `0b 00` and `21 00` are the
two leading bytes of the payload, 11 and 33.

The search that missed them looked for the pattern `0b 00 00 00 21 00 00 00`,
i.e. the two dimensions as a pair of `u32`s. They are stored as two `u16`s, so
that pattern never occurs. The negative result was an artifact of the search
pattern, not evidence about the data -- and it was then used as a premise for
the next several rounds of disc-side work.

Two lessons, both already written down and both re-learned:

- **a negative result from a byte search bounds the search, not the format.**
  Getting the pattern right is part of the measurement;
- the control for this one is trivial and was not run: read the predicted RAM
  address and compare it to the disc, which takes one command and would have
  shown the verbatim copy immediately.

## tfrag: two relocated pointer fields per table record, and morphed vertices behind them

Checking the tfrag record framing against RAM. The tfrags block header is
`table_offset 0x40, tfrag_count 833, lod_distance 32.0`, and that 8-byte prefix
hits once in the level-0 savestate: **RAM `0x848000`**. So the block is at a
known place, and disc offset 0 maps to RAM `0x848000`.

**Result 1 -- exactly two fields per record are rewritten.** Comparing all 833
64-byte table records, field by field:

    records compared                833
    u32 fields differing            1,665  = 833 x 2, and no others
    their offsets                   +0x10 and +0x38, on every record
    relocation delta                8,683,584 = 0x848000, on 833 of them

So `+0x10` and `+0x38` are **block-relative pointers**, and the loader adds the
block's RAM base to each. Every other byte of all 833 records is verbatim. The
remaining six fields differ by 0xF80000 and 0x1F00000 rather than 0x848000 --
three records pointing into a different buffer, recorded rather than averaged.

**Correction:** both fields were already named. `+0x10` is
`TfragHeader::data` -- the byte offset of the tfrag's data block, relative to the
table -- and `+0x38` is `TfragHeader::cube_ofs`. The RAM comparison does not name
them; what it adds is that both are **pointers**, relocated by adding the block's
RAM base, rather than plain offsets.

**Result 2 -- behind the pointers, about 8.6% of bytes are rewritten.** Following
record 0's `+0x10` to disc `0xd040` / RAM `0x855040` and comparing 16 KiB:

    identical bytes                  14,977 of 16,384
    differing                         1,407  = 8.6%

The differences are spread across many field positions rather than concentrated,
which is what morphed **positions** look like: the topology, UVs, counts and
command lists stay verbatim while a fraction of the position bytes move. 8.6% is
about half of a 16-byte vertex's 6 position bytes, i.e. consistent with roughly
half the vertices in this patch being mid-morph at the saved camera distance.

### How much of a patch actually changes, and where

The whole tfrags block was read from RAM -- 4,379,200 bytes at `0x848000` -- and
every one of the 833 patches compared against its disc copy, grouped by whether
it *can* morph at all.

    patches with NO morphing entries   334    differing bytes: max 10.59%, median 7.80%
    patches WITH morphing entries     499    differing bytes: max 10.49%, median 8.12%
    patches 100% identical to disc      0/334 and 0/499

**The two groups are indistinguishable.** Morphing is **not** what changes the
bytes. This is the disproof of the claim above that the 8.6% was "morphed
positions with topology and UVs left verbatim" -- if it were, the 334 patches with
no morphing entries would be untouched, and none of them is.

Nor is it a relocation: the differing bytes fall in short structured runs
(257-263, 272-277, 305 on patch 0) whose `u32` deltas have no common value.

The remaining explanation, consistent with everything measured, is that **the
loader regenerates each patch's VU packet stream for the LOD tier it selected** --
which happens to every patch, morphing or not. That also settles the verification
route: this memory image shows the LOD-selected form, not the morph weight, so the
weight cannot be read out of it at any camera distance.

Patch 1 (13 morphing entries of 83) compared over 4 KiB, disc `0xdf60` against RAM
`0x855f60`:

    bytes differing        354 of 4,096  = 8.6%
    contiguous runs        120
    longest run            8 bytes
    extent                 +0xc1 through the end of the window

83 positions at 6 bytes each is 498 bytes, and only 13 of 83 vertices morph, so
**if morphing rewrote position fields alone** the differing bytes would be a
13/83 slice of a 498-byte array -- a couple of dozen bytes in one contiguous
region. Instead 354 bytes differ, spread over 120 short runs across the whole
window.

So the loader is not patching positions in place. It **regenerates the VU packet
stream** for the current morph state, which on this hardware is what morph
actually is -- VU1 GIF packets, not a CPU-side vertex array.

**Consequence for verification.** The plan of "map RAM vertices to disc indices and
solve for the weight" does not work against this memory image: the positions are
inside regenerated packets, so there is no stable per-vertex correspondence to
read a weight out of. Verifying the morph weight against the original needs the
**camera distance at the savestate** so the weight can be computed from the
derivation and compared with the rendered result, rather than read back out of
RAM.

Along the way, two population figures worth having: **833 of 833 patches parse**,
carrying **43,613** vertex-info entries, and **499 of 833 patches have at least
one morphing entry** -- patch 0, the first record, is base-only with none.