Verify the tfrag lighting magnitudes against the running game
The harness that compares the port against the game's own memory existed but had never been compiled: tools/trace did not build (27 errors, all in the argument plumbing), and the tfrag_light.rs light-bank correction it motivated was unverified. This finishes it and runs it, and the run finds two real bugs. Result, on level 0 against three inputs -- the live emulator over PINE and two PCSX2 savestates from the same level, agreeing byte for byte: headers matching the disc on invariant bytes: 833/833 data pointers relocated as table + disc offset: 833/833 light bank in RAM at 0x50ab20, byte-identical to the disc's vertices compared: 40600 (162400 colour bytes) equal: 162400 (100.00%) The locator finds the level's tfrags block in RAM by tfrag 0's bounding sphere and accepts a candidate only when every header matches the disc on the bytes the game leaves alone and every data pointer is the relocated form, so "the block was found" is itself a verification rather than an assumption. The bank is searched in RAM by its exact section bytes, which is what settles its framing: offset at +0x04, an i32 set count, twelve bytes of pad, then the sets. The previous reading (offset at +0x00, count at +0x04) was wrong. Two bugs the comparison found, both of which produced plausible output: 1. vertex_normal_bits negated x and y and left z positive, while NormalTable::normal negates all three. dA came out -0.707 where the game had +0.707, so max(dA, dA*wA) took the wrong branch and every vertex lit near its base colour: dark, but not wrong-dark. 95.88% agreement with the bug, 100.00% without. 2. A record whose select names a set the bank lacks drew transparent black. The game writes the record's base colour; 3,130 of level 0's 40,600 vertices do this (select 0x000f, set 15 -- a runtime slot the level's directional bank does not have). The vertex is unlit, not invisible. Both are now regression tests in the engine. The comparison prints the pass's per-vertex terms (light_vertex_terms) for the first mismatches, because a colour mismatch alone cannot be attributed and the comparison is worthless without attribution -- that is how the second bug was identified from a three-line read rather than guessed at. Docs: docs/plan/pcsx2-verification.md records the harness, the result, the two bugs and what to compare next; roadmap and level-assets.md stop carrying lighting magnitudes as an open item. Findings reach tests only as numbers -- no test reads a savestate, a dump or a live emulator.
This commit is contained in:
@@ -0,0 +1,119 @@
|
||||
# PCSX2 verification (`gc-trace`)
|
||||
|
||||
Status as of **2026-10-11**. The tfrag lighting magnitudes are verified; this
|
||||
is the file that records how, so the next comparison starts from the
|
||||
harness instead of from scratch.
|
||||
|
||||
Tool: `tools/trace` (`gc-trace`). It never ships, no shipped crate depends on
|
||||
it, and it reads the game's own memory out of a running emulator, a
|
||||
savestate, or a raw dump.
|
||||
|
||||
## What exists now
|
||||
|
||||
```text
|
||||
gc-trace pine-info # protocol handshake
|
||||
gc-trace dump-ee <src> [--out FILE] # 32 MiB EE RAM image
|
||||
gc-trace find <src> --bytes HEX # content search (the locators use it too)
|
||||
gc-trace read <src> --addr HEX --len N
|
||||
gc-trace info <src> # savestate metadata
|
||||
gc-trace compare-tfrag-light <unpacked-dir> [--level N] <src>
|
||||
```
|
||||
|
||||
`<src>` is `--pine [slot]`, `--state FILE` / `--state latest`, or `--ee FILE`.
|
||||
It may sit anywhere on the command line.
|
||||
|
||||
The comparison:
|
||||
|
||||
1. **Locates the tfrags block in RAM by content** -- the key is tfrag 0's
|
||||
bounding sphere, sixteen bytes unique in 32 MiB. A candidate is accepted
|
||||
only when the block header matches, **every** header matches the disc on
|
||||
the invariant bytes (everything but the relocated `data` pointer and the
|
||||
runtime occlusion index), and every `data` pointer is the relocated form
|
||||
`table + disc offset`. Anything less is a plausible wrong read.
|
||||
2. **Locates the light bank the same way**, by its exact section bytes.
|
||||
3. **Runs the pass** over the same disc data with
|
||||
`gc_engine::tfrag_light::light_tfrag`.
|
||||
4. **Diffs per channel, per vertex**, bucketed by vertex class, with the first
|
||||
few mismatching vertices' *terms* (`gc_engine::tfrag_light::
|
||||
light_vertex_terms`) printed so a mismatch can be attributed.
|
||||
|
||||
## Result: level 0 (Veldin), verified 2026-10-11
|
||||
|
||||
Inputs: the live emulator over PINE, and two PCSX2 savestates from the same
|
||||
level. All three agree byte for byte.
|
||||
|
||||
```text
|
||||
level0: tfrags block at 0x848000, table at 0x848040, 833 tfrags
|
||||
headers matching the disc on invariant bytes: 833/833
|
||||
data pointers relocated as table + disc offset: 833/833
|
||||
light bank in RAM at 0x50ab20, byte-identical to the disc's
|
||||
|
||||
vertices compared: 40600 (162400 colour bytes)
|
||||
equal: 162400 (100.00%)
|
||||
runtime-set: 3130 vertices, 3130 fully equal
|
||||
single-set: 37470 vertices, 37470 fully equal
|
||||
```
|
||||
|
||||
What this establishes:
|
||||
|
||||
- **The tfrag locator and the pointer model are right.** 833 of 833 headers
|
||||
match on invariant bytes and 833 of 833 `data` pointers are relocated
|
||||
exactly `table + disc offset`, so the block found in RAM is the level's and
|
||||
the geometry read out of it is the geometry the game drew.
|
||||
- **The light bank framing is right.** The section found in RAM is
|
||||
byte-identical to the disc's: offset at `+0x04`, an `i32` set count, twelve
|
||||
bytes of pad, then 0x40-byte sets. The previous reading -- offset at
|
||||
`+0x00`, count at `+0x04` -- was wrong, and this is what settled it.
|
||||
- **The lighting magnitudes match exactly**: every one of the 162,400 colour
|
||||
bytes the game wrote equals ours. This was the open item in
|
||||
`docs/roadmap.md` and `docs/plan/level-assets.md` ("Lighting magnitudes are
|
||||
unverified: they need comparison against reference frames").
|
||||
|
||||
## Two bugs the comparison found
|
||||
|
||||
Both produced plausible output and neither was visible in a screenshot.
|
||||
|
||||
**1. The normal's z was not negated.** `vertex_normal_bits` negated x and y
|
||||
and left z positive, while the renderer's `NormalTable::normal` negates all
|
||||
three. The result: `dA = -0.707` where the game had `+0.707`, so
|
||||
`max(dA, dA·wA)` picked the wrong branch and every vertex lit near its base
|
||||
colour -- dark, but not *wrong* dark. Level 0 agreed on 95.88% of colour
|
||||
bytes with the bug and 100.00% without it.
|
||||
|
||||
**2. A record with no light set drew transparent black.** `select 0x000f`
|
||||
names set 15; the level's directional bank has 5 sets, so the pass had no set
|
||||
to light with and yielded `[0,0,0,0]`. The game writes the record's **base
|
||||
colour**: 3,130 of level 0's 40,600 vertices do this. The vertex is unlit, not
|
||||
invisible.
|
||||
|
||||
## Finding the next thing
|
||||
|
||||
- **Compare another level.** All three inputs available are level 0; another
|
||||
level needs a savestate taken in it (F1 in-game, or `gc-trace
|
||||
pine-savestate SLOT`). The harness takes `--level N` already.
|
||||
- **Point lights.** The runtime-set cluster is set 15 and above; those slots
|
||||
are filled by gameplay, so they cannot be verified from a level-load image.
|
||||
Comparing them needs a frame with point lights in it.
|
||||
- **Blended sets.** Level 0's vertices are all single-set (`select & 0xff00 ==
|
||||
0`), so the blend path -- the only path `rsqrt` touches -- is untested
|
||||
against the game.
|
||||
- **Distil, then test.** A comparison is not a regression. When a fact from
|
||||
here becomes load-bearing, distil it into a numbers-only fixture under
|
||||
`tools/trace/tests/` and test against that; no test may read a savestate, a
|
||||
dump or a live emulator.
|
||||
|
||||
## Running it
|
||||
|
||||
```sh
|
||||
cargo run -q -p gc-trace -- pine-info
|
||||
cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 0 --pine
|
||||
cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 0 \
|
||||
--state ~/.config/PCSX2/sstates/<name>.p2s
|
||||
```
|
||||
|
||||
Exit status is 0 when every byte matches, 2 on mismatches, 1 on error, so it
|
||||
can be run as a check. PINE must be enabled in the emulator (Settings →
|
||||
Advanced); the socket is `$XDG_RUNTIME_DIR/pcsx2.sock` on Linux.
|
||||
|
||||
**Nothing here redistributes game data.** The disc, the BIOS and the
|
||||
savestates are the user's own; the repository holds the tool and the numbers.
|
||||
Reference in New Issue
Block a user