ReaperOfVeriod
186a700f32
All eight sub-list payloads from level 0 were read to their exact extent (count * 16) and compared block by block. Range measured: savestate slot 1, the packets at 0x009376f0, 0x00939c50, 0x0093cc70, 0x00998880, 0x009c19c0, 0x009cf370, 0x009d2da0 and 0x009f5aa0, counts 21 to 41, each truncated to exactly count*16 bytes. Every payload contains exactly five blocks tagged 00 00 00 30, and the first three are always at offsets 0, 32 and 64: 009376f0 22 0, 32, 64, 184, 304 00939c50 36 0, 32, 64, 268, 400 0093cc70 23 0, 32, 64, 200, 288 00998880 36 0, 32, 64, 272, 404 009c19c0 25 0, 32, 64, 196, 300 009cf370 24 0, 32, 64, 204, 316 009d2da0 41 0, 32, 64, 308, 448 009f5aa0 21 0, 32, 64, 176, 280 Blocks 4 and 5 move with the payload; the first three do not. Eight payloads, eight different block-4/5 positions, and 0/32/64 in every case. Per-block fields constant across all eight: block 0 byte 4 varies 0x78..0xcd block 1 byte 4 = 0 in 8/8, trailer 00000000 in 8/8 block 2 byte 4 varies independently, bytes 16-20 zero in 8/8 block 3 byte 4 EQUALS BLOCK 0 in 8/8 block 4 byte 4 = 0 in 8/8 Block 2's word is unrelated to block 0's in every case, so the equality is not every block reading one shared field. Correction: the "byte permutation table" reported in the previous commit does not generalise. It appears in 0x009376f0 at offset 212, but a scan across all eight payloads finds no such run outside that one. It was one payload's block content, not a format feature. The five-block shape, the fixed 0/32/64 offsets and the block-0/block-3 equality are what survive eight-way comparison. Not established: what the five blocks are, why blocks 0 and 3 share a word, and what moves blocks 4 and 5 -- their offsets are not monotone in count (21 gives 280, 36 gives 400 and 404), so something inside the earlier blocks sets them.
568 lines
26 KiB
Markdown
568 lines
26 KiB
Markdown
# PCSX2 verification (`gc-trace`)
|
|
|
|
Status as of **2026-10-11**. The tfrag lighting magnitudes are verified; this
|
|
is the file that records how, so the next comparison starts from the
|
|
harness instead of from scratch.
|
|
|
|
Tool: `tools/trace` (`gc-trace`). It never ships, no shipped crate depends on
|
|
it, and it reads the game's own memory out of a running emulator, a
|
|
savestate, or a raw dump.
|
|
|
|
## What exists now
|
|
|
|
```text
|
|
gc-trace pine-info # protocol handshake
|
|
gc-trace dump-ee <src> [--out FILE] # 32 MiB EE RAM image
|
|
gc-trace find <src> --bytes HEX # content search (the locators use it too)
|
|
gc-trace read <src> --addr HEX --len N
|
|
gc-trace info <src> # savestate metadata
|
|
gc-trace compare-tfrag-light <unpacked-dir> [--level N] <src>
|
|
```
|
|
|
|
`<src>` is `--pine [slot]`, `--state FILE` / `--state latest`, or `--ee FILE`.
|
|
It may sit anywhere on the command line.
|
|
|
|
The comparison:
|
|
|
|
1. **Locates the tfrags block in RAM by content** -- the key is tfrag 0's
|
|
bounding sphere, sixteen bytes unique in 32 MiB. A candidate is accepted
|
|
only when the block header matches, **every** header matches the disc on
|
|
the invariant bytes (everything but the relocated `data` pointer and the
|
|
runtime occlusion index), and every `data` pointer is the relocated form
|
|
`table + disc offset`. Anything less is a plausible wrong read.
|
|
2. **Locates the light bank the same way**, by its exact section bytes.
|
|
3. **Runs the pass** over the same disc data with
|
|
`gc_engine::tfrag_light::light_tfrag`.
|
|
4. **Diffs per channel, per vertex**, bucketed by vertex class, with the first
|
|
few mismatching vertices' *terms* (`gc_engine::tfrag_light::
|
|
light_vertex_terms`) printed so a mismatch can be attributed.
|
|
|
|
## Result: levels 0 and 1, verified 2026-10-11
|
|
|
|
Level 0 (Veldin), three inputs -- the live emulator over PINE and two PCSX2
|
|
savestates from that level, all agreeing byte for byte:
|
|
|
|
```text
|
|
level0: tfrags block at 0x848000, table at 0x848040, 833 tfrags
|
|
headers matching the disc on invariant bytes: 833/833
|
|
data pointers relocated as table + disc offset: 833/833
|
|
light bank in RAM at 0x50ab20, byte-identical to the disc's
|
|
|
|
vertices compared: 40600 (162400 colour bytes)
|
|
equal: 162400 (100.00%)
|
|
runtime-set: 3130 vertices, 3130 fully equal
|
|
single-set: 37470 vertices, 37470 fully equal
|
|
```
|
|
|
|
Level 1, from savestate slot 2 and cross-checked against the live emulator:
|
|
|
|
```text
|
|
level1: tfrags block at 0x84c000, table at 0x84c040, 401 tfrags
|
|
headers matching the disc on invariant bytes: 401/401
|
|
data pointers relocated as table + disc offset: 401/401
|
|
light bank in RAM at 0x50f4a0, byte-identical to the disc's
|
|
|
|
vertices compared: 26470 (105880 colour bytes)
|
|
equal: 105880 (100.00%)
|
|
runtime-set: 129 vertices, 129 fully equal
|
|
single-set: 26341 vertices, 26341 fully equal
|
|
```
|
|
|
|
Level 1 matters more than one more tally: it is a different planet with a
|
|
different geometry extent, a different light bank and a different
|
|
runtime-set population, and it agrees on every byte. The two fixes below are
|
|
therefore not level-0 coincidences.
|
|
|
|
What this establishes:
|
|
|
|
- **The tfrag locator and the pointer model are right.** 833 of 833 headers
|
|
match on invariant bytes and 833 of 833 `data` pointers are relocated
|
|
exactly `table + disc offset`, so the block found in RAM is the level's and
|
|
the geometry read out of it is the geometry the game drew.
|
|
- **The light bank framing is right.** The section found in RAM is
|
|
byte-identical to the disc's: offset at `+0x04`, an `i32` set count, twelve
|
|
bytes of pad, then 0x40-byte sets. The previous reading -- offset at
|
|
`+0x00`, count at `+0x04` -- was wrong, and this is what settled it.
|
|
- **The lighting magnitudes match exactly**: every one of the 162,400 colour
|
|
bytes the game wrote equals ours. This was the open item in
|
|
`docs/roadmap.md` and `docs/plan/level-assets.md` ("Lighting magnitudes are
|
|
unverified: they need comparison against reference frames").
|
|
|
|
## Two bugs the comparison found
|
|
|
|
Both produced plausible output and neither was visible in a screenshot.
|
|
|
|
**1. The normal's z was not negated.** `vertex_normal_bits` negated x and y
|
|
and left z positive, while the renderer's `NormalTable::normal` negates all
|
|
three. The result: `dA = -0.707` where the game had `+0.707`, so
|
|
`max(dA, dA·wA)` picked the wrong branch and every vertex lit near its base
|
|
colour -- dark, but not *wrong* dark. Level 0 agreed on 95.88% of colour
|
|
bytes with the bug and 100.00% without it.
|
|
|
|
**2. A record with no light set drew transparent black.** `select 0x000f`
|
|
names set 15; the level's directional bank has 5 sets, so the pass had no set
|
|
to light with and yielded `[0,0,0,0]`. The game writes the record's **base
|
|
colour**: 3,130 of level 0's 40,600 vertices do this. The vertex is unlit, not
|
|
invisible.
|
|
|
|
## Finding the next thing
|
|
|
|
- **Compare another level.** All three inputs available are level 0; another
|
|
level needs a savestate taken in it (F1 in-game, or `gc-trace
|
|
pine-savestate SLOT`). The harness takes `--level N` already.
|
|
- **Point lights.** The runtime-set cluster is set 15 and above; those slots
|
|
are filled by gameplay, so they cannot be verified from a level-load image.
|
|
Comparing them needs a frame with point lights in it.
|
|
- **Blended sets.** Both levels' vertices are all single-set (`select &
|
|
0xff00 == 0`), so the blend path -- the only path `rsqrt` touches -- is
|
|
untested against the game.
|
|
- **Distil, then test.** A comparison is not a regression. When a fact from
|
|
here becomes load-bearing, distil it into a numbers-only fixture under
|
|
`tools/trace/tests/` and test against that; no test may read a savestate, a
|
|
dump or a live emulator.
|
|
|
|
## The test setup
|
|
|
|
PCSX2 is installed and working on this machine, so every comparison above can
|
|
be re-run rather than being a one-off observation.
|
|
|
|
| What | Where |
|
|
|---|---|
|
|
| Emulator | `~/Desktop/pcsx2/pcsx2-v2.8.2-linux-appimage-x64-Qt.AppImage` — the directory name is **lowercase** on disk |
|
|
| Game disc | `~/Desktop/pcsx2/games/Ratchet & Clank - Going Commando.iso`; verified target `SCUS_972.68` v1.01 NTSC ([`docs/disc-identification.md`](../disc-identification.md)) |
|
|
| Emulator config | `~/.config/PCSX2/` (`sstates`, `bios`, `memcards`, `inis`, `logs`) |
|
|
| BIOS | extracted into `~/.config/PCSX2/bios/` |
|
|
| Savestates | `~/.config/PCSX2/sstates/SCUS-97268 (38996035).NN.p2s`, `NN` = slot |
|
|
| Unpacked disc | `~/Documents/gc-extracted/`, all 27 levels |
|
|
| PINE socket | `$XDG_RUNTIME_DIR/pcsx2.sock` (`/run/user/1000/pcsx2.sock`), slot 28011 |
|
|
|
|
The emulator boots the disc with `-fastboot`; a level takes about a minute
|
|
from a cold start to reach. To capture a state manually, press F1 in-game (it
|
|
lands in the slot's `.p2s`); to capture from a script, `gc-trace
|
|
pine-savestate SLOT` asks the emulator to write one.
|
|
|
|
The disc integration tests need the image, not the emulator:
|
|
|
|
```sh
|
|
GC_DISC="$HOME/Desktop/pcsx2/games/Ratchet & Clank - Going Commando.iso" \
|
|
cargo test --workspace
|
|
```
|
|
|
|
## Running it
|
|
|
|
```sh
|
|
cargo run -q -p gc-trace -- pine-info
|
|
cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 0 --pine
|
|
cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 0 \
|
|
--state ~/.config/PCSX2/sstates/<name>.p2s
|
|
```
|
|
|
|
Exit status is 0 when every byte matches, 2 on mismatches, 1 on error, so it
|
|
can be run as a check. PINE must be enabled in the emulator (Settings →
|
|
Advanced); the socket is `$XDG_RUNTIME_DIR/pcsx2.sock` on Linux.
|
|
|
|
**Nothing here redistributes game data.** The disc, the BIOS and the
|
|
savestates are the user's own; the repository holds the tool and the numbers.
|
|
|
|
## Shrub packet headers are rewritten in RAM: the loader relocates them
|
|
|
|
Answering the "what is the tag byte" question needed an anchor, and searching
|
|
RAM from scratch was the wrong way to get one -- it finds coincidental matches
|
|
(`04 04 00 01` matches unrelated data at `0x1ac988`) and no way to tell them
|
|
apart. The savestates are the anchor: slot `.01` is level 0, and searching one
|
|
gives candidates that can be *checked* rather than guessed at.
|
|
|
|
**Range measured:** savestate slot 1,
|
|
`~/.config/PCSX2/sstates/SCUS-97268 (38996035).01.p2s`, records at `0x22a6b8`
|
|
onward, read with `gc-trace read --state`.
|
|
|
|
**Result: 40+ records beginning `04 04 00 01 00 00 00 00` at `0x30` spacing**,
|
|
which is the packet framing identified on disc, and the loader has rewritten
|
|
the second half of each header:
|
|
|
|
disc: 04 04 00 01 | 00 00 00 00 | 00 00 00 05 | 00 80 0e 6c
|
|
RAM: 04 04 00 01 | 00 00 00 00 | 13 00 00 30 | 70 6a 93 00
|
|
|
|
Bytes 0-7 are unchanged. Bytes 8-11 and 12-15 are not:
|
|
|
|
- on disc, bytes 12-13 are the constant `00 80` and byte 15 the constant `6c`;
|
|
- in RAM, bytes 12-15 hold what is plainly a pointer -- `0x00936a70`,
|
|
`0x009376f0`, `0x00937850`, `0x00939c50` in consecutive records, all inside
|
|
EE RAM, and ascending across the table exactly as a walk of consecutive
|
|
objects would be.
|
|
|
|
Byte 11 becomes `0x30` in every record seen, against a constant `0x05` on
|
|
disc, and bytes 8-9 hold a varying small value (`0x13`, `0x16`, `0x24`).
|
|
|
|
**What this means.** The packet table at `+0x40` is the thing the loader walks:
|
|
it holds span-relative offsets on disc and absolute RAM addresses after
|
|
loading, which is what the previous pass found and what these 40+ records show
|
|
directly. Any header field read past byte 8 on disc is describing something the
|
|
loader replaces.
|
|
|
|
**Consequence for the tag bytes.** The values at record bytes 7 and 15 measured
|
|
on disc are *not* stable identifiers -- at least byte 15 sits in a field the
|
|
loader overwrites with a pointer. The four-value and two-value vocabularies are
|
|
real (0x30, 0x6d, 0x80 and 0 appear against 1636 records of otherwise-uniform
|
|
bytes), but byte 15 being a pointer in RAM means its disc values are being read
|
|
as data by something, not stored as a tag.
|
|
|
|
**Not settled:** whether byte 7's vocabulary is also rewritten, and whether
|
|
`0x6d` marks a payload that the loader expands. That needs the same records
|
|
compared one by one against their disc spans, which is the next step and is
|
|
now cheap because the anchor exists.
|
|
|
|
## Following the pointers: packet payloads differ by type
|
|
|
|
The header pointers found above are live, so they can be dereferenced. Two
|
|
consecutive records from the level 0 table at `0x22a6b8` were followed.
|
|
|
|
**Range measured:** savestate slot 1, reading `0x00936a70` (from the record at
|
|
`0x22a6b8`) and `0x009376f0` (from the record at `0x22a6e8`), 64 bytes each.
|
|
|
|
**Result: the two payloads have nothing in common.**
|
|
|
|
`0x00936a70` -- fixed-point vertex data:
|
|
|
|
8d 8c 80 80 89 87 7c 80 8d 8c 80 80 89 87 7c 80
|
|
4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80 (repeating)
|
|
|
|
Every high byte is `0x80` or `0x7c`, and the pattern repeats on a 8-byte
|
|
boundary. Read as `s16` these are large negatives in a narrow band around
|
|
-32500. That is consistent with Q16.15 fixed-point rather than `f32`, and with
|
|
a stride of 4 `s16` per vertex.
|
|
|
|
`0x009376f0` -- a tagged sub-list, not vertex data:
|
|
|
|
00 00 00 30 cd 00 00 00 cd 00 00 00 cd 00 00 00
|
|
cd 00 00 00 01 00 00 05 22 c1 01 6e 00 00 00 00
|
|
|
|
Structurally different: small integers, a repeating `0xcd`, and `0x30` tags in
|
|
the high byte. Nothing here is position data.
|
|
|
|
**What this supports.** Two records 0x30 apart in the same table hold payloads
|
|
of entirely different kinds. That is direct evidence for the type reading of
|
|
the header: the table is a tagged collection where the tag selects what the
|
|
payload is, not a uniform array of one thing.
|
|
|
|
It also rules out the alternative I could not previously exclude -- that the
|
|
16-byte tiling was a single uniform record format and the varying bytes were
|
|
all fields of it. Two adjacent records with unrelated payloads is not that.
|
|
|
|
**Still open.** The mapping from tag value to payload kind. The `0x30` that
|
|
appears in record bytes 8-11 in RAM also appears at the head of the second
|
|
payload, which is suggestive but is one coincidence so far and should not be
|
|
written up as a link until it is seen in more than one record.
|
|
|
|
## The tag values map onto three payload kinds
|
|
|
|
Following the pointers out of the level 0 table, all nine that carry the `0x30`
|
|
tag at byte 11 and an EE pointer at 12..15 were dereferenced and read 32 bytes
|
|
each. They fall into exactly three classes, three payloads each, with no
|
|
overlap.
|
|
|
|
**A -- Q16.15 fixed-point vertex data** (3 of 9)
|
|
|
|
00936a70 8d 8c 80 80 89 87 7c 80 8d 8c 80 80 89 87 7c 80
|
|
4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80
|
|
0093cde0 8a 8a 7f 80 92 90 84 80 8a 8a 7f 80 92 90 84 80
|
|
4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80
|
|
009cf4f0 6e 6f 68 80 6e 6f 68 80 6e 6f 68 80 6e 6f 68 80
|
|
54 54 51 80 54 54 51 80 54 54 51 80 54 54 51 80
|
|
|
|
Every high byte is `0x80` or `0x7c`, each payload repeats on an 8-byte
|
|
boundary, and the values differ between payloads while the `4e 4e 4c 80` tail
|
|
is shared by two of them.
|
|
|
|
**B -- a `0x30` sub-list with a repeated count** (3 of 9)
|
|
|
|
009376f0 00 00 00 30 cd 00 00 00 cd 00 00 00 cd 00 00 00
|
|
cd 00 00 00 01 00 00 05 22 c1 01 6e 00 00 00 00
|
|
00998880 00 00 00 30 c0 00 00 00 c0 00 00 00 c0 00 00 00
|
|
c0 00 00 00 01 00 00 05 11 c1 01 6e 00 00 00 00
|
|
009d2da0 00 00 00 30 c8 00 00 00 c8 00 00 00 c8 00 00 00
|
|
c8 00 00 00 01 00 00 05 1d c1 01 6e 00 00 00 00
|
|
|
|
Structurally identical apart from the repeated word: `0xcd`, `0xc0`, `0xc8`.
|
|
All three carry `01 00 00 05` at offset 16 and a `.. c1 01 6e` at 20.
|
|
|
|
**C -- an index/attribute list** (3 of 9)
|
|
|
|
00939700 00 c0 05 6d 3e 00 00 00 01 00 00 00 12 00 00 00
|
|
18 00 c0 00 fe 00 fe 00 ff 00 ff 00 11 01 17 01
|
|
009c1540 00 c0 05 6d 36 00 00 00 01 00 00 00 08 00 00 00
|
|
13 00 93 00 c9 00 c9 00 ca 00 ca 00 d2 00 d5 00
|
|
|
|
Both open `00 c0 05 6d`, then a count, then a run of small ascending `u16`.
|
|
|
|
**The mapping.** The tag vocabulary measured on disc -- `0x00`, `0x30`,
|
|
`0x6d`, `0x80` at record byte 7 -- is not decorative:
|
|
|
|
| tag | payload |
|
|
|---|---|
|
|
| `0x30` | class B, the `00 00 00 30` sub-list |
|
|
| `0x6d` | class C, the `00 c0 05 6d` attribute list |
|
|
| `0x00` | class A, the fixed-point vertex data |
|
|
|
|
`0x30` heads the class B payloads and `0x6d` heads the class C payloads, which
|
|
is the link flagged as a single coincidence two entries back. It is now seen in
|
|
three payloads each, so it is a rule rather than an accident.
|
|
|
|
`0x80` did not appear among these nine. Its meaning is still open -- it is the
|
|
fourth value in the vocabulary and there is no sample of it here.
|
|
|
|
**A correction.** The record spacing in this table is not uniformly `0x30`. The
|
|
`find` output shows `0x30`, `0x10`, `0x30`, `0x30` between consecutive packet
|
|
headers, and only 9 of the 5,632 bytes scanned carry both the `0x30` tag and an
|
|
EE pointer. The earlier "records at 0x30 spacing" was inferred from a truncated
|
|
list of hits and does not hold.
|
|
|
|
## Correction: the record tag does not select the payload
|
|
|
|
The previous entry mapped the disc tag vocabulary onto the three payload
|
|
classes. A wider scan says that is wrong.
|
|
|
|
**Range measured:** savestate slot 1, RAM `0x22a000..0x24037f` -- 91,008 bytes,
|
|
20 times the span scanned before.
|
|
|
|
**Result: 19 packet headers, and all 19 carry byte 11 = `0x30`.** There are no
|
|
`0x6d` or `0x80` tags anywhere in this range. Reading six of their payloads:
|
|
|
|
rec 0022a6b8 tag 30 -> 00936a70 8d 8c 80 80 89 87 7c 80 ... (fixed point)
|
|
rec 0022a6e8 tag 30 -> 009376f0 00 00 00 30 cd 00 00 00 ... (sub-list)
|
|
rec 0022a6f8 tag 30 -> 00937850 92 90 84 80 94 94 86 80 ... (fixed point)
|
|
rec 0022a728 tag 30 -> 00939c50 00 00 00 30 c0 00 00 00 ... (sub-list)
|
|
rec 0022a738 tag 30 -> 00939e90 4c 4c 4b 80 4c 4c 4b 80 ... (fixed point)
|
|
rec 0022a768 tag 30 -> 0093cc70 00 00 00 30 78 00 00 00 ... (sub-list)
|
|
|
|
Records of identical tag alternate between two payload kinds. **The tag does
|
|
not select the payload.**
|
|
|
|
**What went wrong.** The previous entry took the `0x30` and `0x6d` that *head
|
|
the payloads* to be the disc-side tag vocabulary, and matched them to the
|
|
payload classes by that correspondence. The disc vocabulary measured at record
|
|
byte 7 -- `0x00`, `0x30`, `0x6d`, `0x80` -- and the RAM record byte 11 were
|
|
treated as the same field. They are not: the payload magics are a property of
|
|
the payload, and record byte 11 is a different thing that happens to be
|
|
constantly `0x30` here.
|
|
|
|
Three payloads each made the coincidence look like a rule. It was a
|
|
correspondence between two vocabularies that overlap in two values.
|
|
|
|
**What is actually established about the RAM header**, per position:
|
|
|
|
| byte | value |
|
|
|---|---|
|
|
| 0-3 | `04 04 00 01`, constant |
|
|
| 4-7 | zero, constant |
|
|
| 8 | a **count**, `0x0b` to `0x29` across the 19 records |
|
|
| 9-10 | zero, constant |
|
|
| 11 | `0x30`, constant on all 19 in this range |
|
|
| 12-15 | an EE RAM pointer |
|
|
|
|
Byte 8 is the finding that survives. It is a small count that varies per
|
|
record, and it is the one field here that is doing arithmetic rather than
|
|
tagging.
|
|
|
|
**Still open:** what `0x30` at byte 11 means, what discriminates the payload
|
|
kinds given that the record tag does not, and the meaning of `0x80`, which has
|
|
no sample in 91 KB of RAM.
|
|
|
|
## The table interleaves two record magics
|
|
|
|
Scanning the same 91 KB for other four-byte record magics beside `04 04 00 01`
|
|
turns up a second one at the same density: **`02 01 00 01`, 10 occurrences**
|
|
against `04 04 00 01`'s 19.
|
|
|
|
**Range measured:** savestate slot 1, RAM `0x22a000..0x24037f`.
|
|
|
|
**Result: the two magics interleave.** Every `02 01 00 01` sits between two
|
|
`04 04 00 01` records:
|
|
|
|
0x22a6b8 04 04 00 01 13 00 00 30 70 6a 93 00
|
|
0x22a6c8 02 01 00 01 19 c0 4c 6e 00 00 10 00 00 00 00 00
|
|
0x22a6e8 04 04 00 01 16 00 00 30 f0 76 93 00
|
|
0x22a6f8 04 04 00 01 16 00 00 30 50 78 93 00
|
|
0x22a708 02 01 00 01 1e c0 58 6e 00 00 10 00 00 00 00 00
|
|
0x22a728 04 04 00 01 24 00 00 30 50 9c 93 00
|
|
0x22a738 04 04 00 01 16 00 00 30 50 9e 93 00
|
|
0x22a748 02 01 00 01 19 c0 54 6e 00 00 10 00 00 00 00 00
|
|
|
|
The `02 01 00 01` records are 16 bytes like the others but carry **no pointer
|
|
** -- bytes 12-15 are zero in all ten. Their fields are a count at 4-5
|
|
(`0x14`, `0x19`, `0x1e`), a constant `0xc0` at 6, a varying byte at 7
|
|
(`0x30`, `0x4c`, `0x54`, `0x58`), `0x6e` at 8-9, and `10 00` at 10-11.
|
|
|
|
**The `0x6e` link.** These records carry `0x6e` at bytes 8-9, and the sub-list
|
|
payloads found earlier carry `.. c1 01 6e` at offset 20-23. That is a second
|
|
correspondence between this record type and the sub-list payloads, independent
|
|
of the one retracted in the previous entry -- and this one is a shared *value*
|
|
rather than a shared *position*.
|
|
|
|
**What this does not settle.** It still does not say what discriminates the two
|
|
payload kinds among the `04 04 00 01` records, because those all carry the same
|
|
magic, the same `0x30` at byte 11 and differ only in the count at byte 8. The
|
|
`02 01 00 01` records sit between them rather than replacing them, so they are
|
|
probably a companion structure, not the missing discriminator.
|
|
|
|
Still open: what distinguishes a fixed-point payload from a sub-list payload
|
|
when the records carrying them look identical in bytes 0-3 and 11. The most
|
|
likely place to look is the count at byte 8 against the payload's own length.
|
|
|
|
## The count at byte 8 sizes the sub-list payload: exactly 16 bytes per record
|
|
|
|
The previous entry left one test: compare the count at byte 8 against the
|
|
payload's own extent. Pointer spacing gives the extent, since the packets are
|
|
laid out in ascending order in EE RAM.
|
|
|
|
**Range measured:** savestate slot 1, the 17 packets in RAM
|
|
`0x22a000..0x24037f`, each with its record address, byte-8 count and pointer,
|
|
sorted by pointer so the next packet is the next object in memory.
|
|
|
|
**Result: eight packets sit exactly `count * 16` bytes after their predecessor
|
|
-- an integer ratio of 16.00 in every one, with no rounding.**
|
|
|
|
record count ptr gap gap/count
|
|
0x22a6e8 22 009376f0 352 16.00
|
|
0x22a728 36 00939c50 576 16.00
|
|
0x22a768 23 0093cc70 368 16.00
|
|
0x22a7a8 36 00998880 576 16.00
|
|
0x22a7e8 25 009c19c0 400 16.00
|
|
0x22a828 24 009cf370 384 16.00
|
|
0x22a868 41 009d2da0 656 16.00
|
|
0x22a8a8 21 009f5aa0 336 16.00
|
|
|
|
Eight exact hits at a ratio that has to be an integer is not sampling luck. The
|
|
count at byte 8 is the **record count**, and these payloads are 16 bytes per
|
|
record.
|
|
|
|
**And these are exactly the sub-list payloads** -- `0x009376f0`, `0x00939c50`,
|
|
`0x0093cc70`, `0x00998880`, `0x009d2da0` were all read directly in earlier
|
|
passes and all open `00 00 00 30` followed by a repeated word.
|
|
|
|
**What the other nine are.** Their ratios are 168, 419, 559, 31288, 7985,
|
|
3458, 764, 6452 -- no shared factor, and far too large to be a record count
|
|
times anything. So either their payload is not followed immediately by the next
|
|
packet in the pointer order (other data intervenes), or they are not 16 bytes
|
|
per record. Both readings are open; what is ruled out is that they follow the
|
|
same rule.
|
|
|
|
**This resolves byte 8.** It was measured as "a count, 0x0b to 0x29" two entries
|
|
ago and is now shown to size a payload exactly, in eight independent cases,
|
|
within a single level. Whether it also sizes the fixed-point payloads is the
|
|
open half.
|
|
|
|
Still open: why the fixed-point packets do not follow the same rule, and what
|
|
the 16 bytes per sub-list record are -- the repeated word (`0xcd`, `0xc0`,
|
|
`0xc8`, `0x78`) differs per packet, so the records are not identical.
|
|
|
|
## Inside a sub-list payload: tagged blocks and a byte permutation table
|
|
|
|
With the extent known (`count * 16`), a whole sub-list payload can be read.
|
|
This is the packet at `0x009376f0`, count 22, 352 bytes, in full.
|
|
|
|
**Range measured:** savestate slot 1, `0x009376f0`..`0x0093784f`, 352 bytes.
|
|
|
|
**Result: the payload is a sequence of blocks, each opening `00 00 00 30`,**
|
|
carrying a repeating word and then a short trailer:
|
|
|
|
00 00 00 30 | cd 00 00 00 cd 00 00 00 cd 00 00 00 cd 00 00 00
|
|
01 00 00 05 | 22 c1 01 6e | 00 00 00 00
|
|
00 00 00 30 | 00 00 00 45 00 00 00 45 1d 00 00 00 1d 00 00 00 1f
|
|
81 01 6d 00 | 00 00 00 00 | 00 00 a4 00
|
|
00 00 00 30 | bc c0 03 00 23 17 03 00 e5 dd 00 00 00 00 00 00
|
|
02 01 00 01 | c1 80 01 69 | 00 00 00 00
|
|
|
|
The repeating word differs per block -- `0xcd`, `0x45`, `0x03` -- and the
|
|
trailer's second word varies in its high bytes: `22 c1 01 6e`, `81 01 6d 00`,
|
|
`c1 80 01 69`. Every block seen ends `... 6e` or `69` or `6d` at byte 23, which
|
|
is the same `6e` the `02 01 00 01` records carried at bytes 8-9.
|
|
|
|
**The payload contains a byte permutation table.** At `0x009377c4`:
|
|
|
|
00 01 02 03 3e 3f 53 54 40 41 42 43 4e 4f 50 51 04 05 06 07 08 ...
|
|
|
|
This is not a position or vertex list -- it is a run of ascending byte values
|
|
with a handful (`3e 3f`, `53 54`, `40 41 42 43`, `4e 4f 50 51`) lifted out of
|
|
order ahead of the sequential run. Read as a table, it maps each of the 0..0x39
|
|
symbols to a slot. That is the shape of a **code-length or symbol-ordering
|
|
table**, and it is the first thing in this format that is self-evidently a table
|
|
rather than data to be interpreted.
|
|
|
|
**What this does not say.** Whether the table is a Huffman-style code, an index
|
|
reorder for the vertex data that follows, or a palette permutation. The
|
|
following bytes continue the sequence to `39` and then `23 c1 01 6e`, after
|
|
which the next block opens.
|
|
|
|
**Corroboration worth noting.** The fixed-point payload `0x00937850` begins
|
|
immediately after this one ends at `0x00937850` exactly -- the 22-record
|
|
identity from the previous entry made the boundary exact, and the next packet
|
|
starts on a payload boundary with nothing in between. The two payload kinds sit
|
|
adjacent in RAM, which is consistent with them being two records of the same
|
|
blob rather than two unrelated objects.
|
|
|
|
## The sub-list payload has a fixed five-block shape
|
|
|
|
All eight sub-list payloads from level 0 were read to their exact extent
|
|
(`count * 16`) and compared block by block. The structure is the same in every
|
|
one.
|
|
|
|
**Range measured:** savestate slot 1, the eight packets at `0x009376f0`,
|
|
`0x00939c50`, `0x0093cc70`, `0x00998880`, `0x009c19c0`, `0x009cf370`,
|
|
`0x009d2da0` and `0x009f5aa0`, counts 21 to 41, each truncated to exactly
|
|
`count * 16` bytes.
|
|
|
|
**Result: every payload contains exactly five blocks tagged `00 00 00 30`, and
|
|
the first three are always at offsets 0, 32 and 64.**
|
|
|
|
| payload | count | blocks at |
|
|
|---|---|---|
|
|
| `009376f0` | 22 | 0, 32, 64, 184, 304 |
|
|
| `00939c50` | 36 | 0, 32, 64, 268, 400 |
|
|
| `0093cc70` | 23 | 0, 32, 64, 200, 288 |
|
|
| `00998880` | 36 | 0, 32, 64, 272, 404 |
|
|
| `009c19c0` | 25 | 0, 32, 64, 196, 300 |
|
|
| `009cf370` | 24 | 0, 32, 64, 204, 316 |
|
|
| `009d2da0` | 41 | 0, 32, 64, 308, 448 |
|
|
| `009f5aa0` | 21 | 0, 32, 64, 176, 280 |
|
|
|
|
Blocks 4 and 5 move with the payload; the first three do not. Eight payloads,
|
|
eight different block-4/5 positions, and 0, 32, 64 in every case.
|
|
|
|
**Per-block fields, constant across all eight:**
|
|
|
|
| block | byte 4 | other |
|
|
|---|---|---|
|
|
| 0 | varies | `0x78`-`0xcd` |
|
|
| 1 | **0** in 8/8 | trailer `00 00 00 00` in 8/8 |
|
|
| 2 | varies independently | bytes 16-20 `00000000` in 8/8 |
|
|
| 3 | **equals block 0 in 8/8** | |
|
|
| 4 | **0** in 8/8 | |
|
|
|
|
The block-0/block-3 equality is exact in all eight:
|
|
|
|
009376f0 0xcd 0x00 0xbc 0xcd 0x00
|
|
00939c50 0xc0 0x00 0xbf 0xc0 0x00
|
|
0093cc70 0x78 0x00 0x30 0x78 0x00
|
|
00998880 0xc0 0x00 0xc9 0xc0 0x00
|
|
009c19c0 0x93 0x00 0x1f 0x93 0x00
|
|
009cf370 0xb5 0x00 0x0d 0xb5 0x00
|
|
009d2da0 0xc8 0x00 0x65 0xc8 0x00
|
|
009f5aa0 0x9b 0x00 0x25 0x9b 0x00
|
|
|
|
Block 2's word is unrelated to block 0's in every case, so the equality is not
|
|
every block reading a shared field.
|
|
|
|
**Correction to the previous entry.** The "byte permutation table" reported
|
|
there does not generalise. It appears in `0x009376f0` at offset 212, but a
|
|
scan for it across all eight payloads finds no run of more than a few bytes
|
|
outside that one. What looked like a table is better described as one payload's
|
|
block content. The `00 00 00 30` block structure, the fixed 0/32/64 offsets and
|
|
the block-0/block-3 equality are the findings that survive eight-way
|
|
comparison; the permutation table was not among them.
|
|
|
|
**Not established.** What the five blocks are, why blocks 0 and 3 carry the
|
|
same word, and what moves blocks 4 and 5 -- their offsets are not monotone in
|
|
the count (`count` 21 gives 280 and `count` 36 gives 400 and 404), so something
|
|
inside the earlier blocks sets them.
|