Going-Commando/docs/plan/pcsx2-verification.md

# PCSX2 verification (`gc-trace`)

Status as of **2026-10-11**. The tfrag lighting magnitudes are verified; this
is the file that records how, so the next comparison starts from the
harness instead of from scratch.

Tool: `tools/trace` (`gc-trace`). It never ships, no shipped crate depends on
it, and it reads the game's own memory out of a running emulator, a
savestate, or a raw dump.

## What exists now

```text
gc-trace pine-info                       # protocol handshake
gc-trace dump-ee   <src> [--out FILE]     # 32 MiB EE RAM image
gc-trace find      <src> --bytes HEX      # content search (the locators use it too)
gc-trace read      <src> --addr HEX --len N
gc-trace info      <src>                  # savestate metadata
gc-trace compare-tfrag-light <unpacked-dir> [--level N] <src>
```

`<src>` is `--pine [slot]`, `--state FILE` / `--state latest`, or `--ee FILE`.
It may sit anywhere on the command line.

The comparison:

1. **Locates the tfrags block in RAM by content** -- the key is tfrag 0's
   bounding sphere, sixteen bytes unique in 32 MiB. A candidate is accepted
   only when the block header matches, **every** header matches the disc on
   the invariant bytes (everything but the relocated `data` pointer and the
   runtime occlusion index), and every `data` pointer is the relocated form
   `table + disc offset`. Anything less is a plausible wrong read.
2. **Locates the light bank the same way**, by its exact section bytes.
3. **Runs the pass** over the same disc data with
   `gc_engine::tfrag_light::light_tfrag`.
4. **Diffs per channel, per vertex**, bucketed by vertex class, with the first
   few mismatching vertices' *terms* (`gc_engine::tfrag_light::
   light_vertex_terms`) printed so a mismatch can be attributed.

## Result: levels 0 and 1, verified 2026-10-11

Level 0 (Veldin), three inputs -- the live emulator over PINE and two PCSX2
savestates from that level, all agreeing byte for byte:

```text
level0: tfrags block at 0x848000, table at 0x848040, 833 tfrags
  headers matching the disc on invariant bytes: 833/833
  data pointers relocated as table + disc offset: 833/833
  light bank in RAM at 0x50ab20, byte-identical to the disc's

vertices compared: 40600 (162400 colour bytes)
  equal:              162400 (100.00%)
  runtime-set: 3130 vertices, 3130 fully equal
  single-set: 37470 vertices, 37470 fully equal
```

Level 1, from savestate slot 2 and cross-checked against the live emulator:

```text
level1: tfrags block at 0x84c000, table at 0x84c040, 401 tfrags
  headers matching the disc on invariant bytes: 401/401
  data pointers relocated as table + disc offset: 401/401
  light bank in RAM at 0x50f4a0, byte-identical to the disc's

vertices compared: 26470 (105880 colour bytes)
  equal:              105880 (100.00%)
  runtime-set: 129 vertices, 129 fully equal
  single-set: 26341 vertices, 26341 fully equal
```

Level 1 matters more than one more tally: it is a different planet with a
different geometry extent, a different light bank and a different
runtime-set population, and it agrees on every byte. The two fixes below are
therefore not level-0 coincidences.

What this establishes:

- **The tfrag locator and the pointer model are right.** 833 of 833 headers
  match on invariant bytes and 833 of 833 `data` pointers are relocated
  exactly `table + disc offset`, so the block found in RAM is the level's and
  the geometry read out of it is the geometry the game drew.
- **The light bank framing is right.** The section found in RAM is
  byte-identical to the disc's: offset at `+0x04`, an `i32` set count, twelve
  bytes of pad, then 0x40-byte sets. The previous reading -- offset at
  `+0x00`, count at `+0x04` -- was wrong, and this is what settled it.
- **The lighting magnitudes match exactly**: every one of the 162,400 colour
  bytes the game wrote equals ours. This was the open item in
  `docs/roadmap.md` and `docs/plan/level-assets.md` ("Lighting magnitudes are
  unverified: they need comparison against reference frames").

## Two bugs the comparison found

Both produced plausible output and neither was visible in a screenshot.

**1. The normal's z was not negated.** `vertex_normal_bits` negated x and y
and left z positive, while the renderer's `NormalTable::normal` negates all
three. The result: `dA = -0.707` where the game had `+0.707`, so
`max(dA, dA·wA)` picked the wrong branch and every vertex lit near its base
colour -- dark, but not *wrong* dark. Level 0 agreed on 95.88% of colour
bytes with the bug and 100.00% without it.

**2. A record with no light set drew transparent black.** `select 0x000f`
names set 15; the level's directional bank has 5 sets, so the pass had no set
to light with and yielded `[0,0,0,0]`. The game writes the record's **base
colour**: 3,130 of level 0's 40,600 vertices do this. The vertex is unlit, not
invisible.

## Finding the next thing

- **Compare another level.** All three inputs available are level 0; another
  level needs a savestate taken in it (F1 in-game, or `gc-trace
  pine-savestate SLOT`). The harness takes `--level N` already.
- **Point lights.** The runtime-set cluster is set 15 and above; those slots
  are filled by gameplay, so they cannot be verified from a level-load image.
  Comparing them needs a frame with point lights in it.
- **Blended sets.** Both levels' vertices are all single-set (`select &
  0xff00 == 0`), so the blend path -- the only path `rsqrt` touches -- is
  untested against the game.
- **Distil, then test.** A comparison is not a regression. When a fact from
  here becomes load-bearing, distil it into a numbers-only fixture under
  `tools/trace/tests/` and test against that; no test may read a savestate, a
  dump or a live emulator.

## The test setup

PCSX2 is installed and working on this machine, so every comparison above can
be re-run rather than being a one-off observation.

| What | Where |
|---|---|
| Emulator | `~/Desktop/pcsx2/pcsx2-v2.8.2-linux-appimage-x64-Qt.AppImage` — the directory name is **lowercase** on disk |
| Game disc | `~/Desktop/pcsx2/games/Ratchet & Clank - Going Commando.iso`; verified target `SCUS_972.68` v1.01 NTSC ([`docs/disc-identification.md`](../disc-identification.md)) |
| Emulator config | `~/.config/PCSX2/` (`sstates`, `bios`, `memcards`, `inis`, `logs`) |
| BIOS | extracted into `~/.config/PCSX2/bios/` |
| Savestates | `~/.config/PCSX2/sstates/SCUS-97268 (38996035).NN.p2s`, `NN` = slot |
| Unpacked disc | `~/Documents/gc-extracted/`, all 27 levels |
| PINE socket | `$XDG_RUNTIME_DIR/pcsx2.sock` (`/run/user/1000/pcsx2.sock`), slot 28011 |

The emulator boots the disc with `-fastboot`; a level takes about a minute
from a cold start to reach. To capture a state manually, press F1 in-game (it
lands in the slot's `.p2s`); to capture from a script, `gc-trace
pine-savestate SLOT` asks the emulator to write one.

The disc integration tests need the image, not the emulator:

```sh
GC_DISC="$HOME/Desktop/pcsx2/games/Ratchet & Clank - Going Commando.iso" \
    cargo test --workspace
```

## Running it

```sh
cargo run -q -p gc-trace -- pine-info
cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 0 --pine
cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 0 \
    --state ~/.config/PCSX2/sstates/<name>.p2s
```

Exit status is 0 when every byte matches, 2 on mismatches, 1 on error, so it
can be run as a check. PINE must be enabled in the emulator (Settings →
Advanced); the socket is `$XDG_RUNTIME_DIR/pcsx2.sock` on Linux.

**Nothing here redistributes game data.** The disc, the BIOS and the
savestates are the user's own; the repository holds the tool and the numbers.