186a700f329c2b766f4076281543ba4c75c7cd80
11
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ReaperOfVeriod
|
186a700f32 |
Establish the sub-list payload as a fixed five-block shape
All eight sub-list payloads from level 0 were read to their exact extent (count * 16) and compared block by block. Range measured: savestate slot 1, the packets at 0x009376f0, 0x00939c50, 0x0093cc70, 0x00998880, 0x009c19c0, 0x009cf370, 0x009d2da0 and 0x009f5aa0, counts 21 to 41, each truncated to exactly count*16 bytes. Every payload contains exactly five blocks tagged 00 00 00 30, and the first three are always at offsets 0, 32 and 64: 009376f0 22 0, 32, 64, 184, 304 00939c50 36 0, 32, 64, 268, 400 0093cc70 23 0, 32, 64, 200, 288 00998880 36 0, 32, 64, 272, 404 009c19c0 25 0, 32, 64, 196, 300 009cf370 24 0, 32, 64, 204, 316 009d2da0 41 0, 32, 64, 308, 448 009f5aa0 21 0, 32, 64, 176, 280 Blocks 4 and 5 move with the payload; the first three do not. Eight payloads, eight different block-4/5 positions, and 0/32/64 in every case. Per-block fields constant across all eight: block 0 byte 4 varies 0x78..0xcd block 1 byte 4 = 0 in 8/8, trailer 00000000 in 8/8 block 2 byte 4 varies independently, bytes 16-20 zero in 8/8 block 3 byte 4 EQUALS BLOCK 0 in 8/8 block 4 byte 4 = 0 in 8/8 Block 2's word is unrelated to block 0's in every case, so the equality is not every block reading one shared field. Correction: the "byte permutation table" reported in the previous commit does not generalise. It appears in 0x009376f0 at offset 212, but a scan across all eight payloads finds no such run outside that one. It was one payload's block content, not a format feature. The five-block shape, the fixed 0/32/64 offsets and the block-0/block-3 equality are what survive eight-way comparison. Not established: what the five blocks are, why blocks 0 and 3 share a word, and what moves blocks 4 and 5 -- their offsets are not monotone in count (21 gives 280, 36 gives 400 and 404), so something inside the earlier blocks sets them. |
||
|
ReaperOfVeriod
|
3fa64b7003 |
Read a full sub-list payload: tagged blocks and a byte permutation table
With the extent known from the previous entry (count * 16), a whole sub-list
payload can be read. This is the packet at 0x009376f0, count 22, 352 bytes.
Range measured: savestate slot 1, 0x009376f0..0x0093784f.
The payload is a sequence of blocks, each opening 00 00 00 30, carrying a
repeating word and a short trailer:
00 00 00 30 | cd 00 00 00 cd 00 00 00 cd 00 00 00 cd 00 00 00
01 00 00 05 | 22 c1 01 6e | 00 00 00 00
00 00 00 30 | 00 00 00 45 00 00 00 45 1d 00 00 00 1d 00 00 00 1f
81 01 6d 00 | 00 00 00 00 | 00 00 a4 00
00 00 00 30 | bc c0 03 00 23 17 03 00 e5 dd 00 00 00 00 00 00
02 01 00 01 | c1 80 01 69 | 00 00 00 00
The repeating word differs per block (0xcd, 0x45, 0x03) and the trailer's
second word varies in its high bytes (22 c1 01 6e, 81 01 6d 00, c1 80 01 69).
Every block ends 6e, 69 or 6d at byte 23 -- the same 6e the 02 01 00 01 records
carried at bytes 8-9.
The payload contains a byte permutation table at 0x009377c4:
00 01 02 03 3e 3f 53 54 40 41 42 43 4e 4f 50 51 04 05 06 07 08 ...
Ascending byte values with a handful (3e 3f, 53 54, 40 41 42 43, 4e 4f 50 51)
lifted out of order ahead of the sequential run. Read as a table it maps each
of 0x00..0x39 to a slot -- the shape of a code-length or symbol-ordering table,
and the first thing in this format that is self-evidently a table rather than
data to be interpreted.
Not said: whether it is a Huffman-style code, an index reorder for the vertex
data that follows, or a palette permutation. The sequence continues to 39 then
23 c1 01 6e, after which the next block opens.
Corroboration: the fixed-point payload 0x00937850 begins exactly where this one
ends. The 22-record identity made the boundary exact and the next packet starts
on a payload boundary with nothing in between, so the two payload kinds sit
adjacent in RAM -- consistent with them being two records of the same blob.
|
||
|
ReaperOfVeriod
|
f1cf6cd164 |
Show the byte-8 count sizes sub-list payloads at exactly 16 bytes each
The previous entry left one test: compare the count at byte 8 against the payload extent. Pointer spacing gives the extent, since the packets lie in ascending order in EE RAM. Range measured: savestate slot 1, the 17 packets in RAM 0x22a000..0x24037f, each with its record address, byte-8 count and pointer, sorted by pointer. Result: eight packets sit exactly count*16 bytes after their predecessor -- a ratio of 16.00 in every one, no rounding. 0x22a6e8 count 22 ptr 009376f0 gap 352 16.00 0x22a728 count 36 ptr 00939c50 gap 576 16.00 0x22a768 count 23 ptr 0093cc70 gap 368 16.00 0x22a7a8 count 36 ptr 00998880 gap 576 16.00 0x22a7e8 count 25 ptr 009c19c0 gap 400 16.00 0x22a828 count 24 ptr 009cf370 gap 384 16.00 0x22a868 count 41 ptr 009d2da0 gap 656 16.00 0x22a8a8 count 21 ptr 009f5aa0 gap 336 16.00 Eight exact hits at a ratio that must be an integer is not luck. The count at byte 8 is the record count, and these payloads are 16 bytes per record. These are exactly the sub-list payloads -- 009376f0, 00939c50, 0093cc70, 00998880 and 009d2da0 were read directly in earlier passes and all open 00 00 00 30 followed by a repeated word. The other nine have ratios of 168, 419, 559, 31288, 7985, 3458, 764 and 6452 -- no shared factor, far too large to be a record count times anything. Either their payload is not followed immediately by the next packet in pointer order, or they are not 16 bytes per record. Ruled out either way: they do not follow the same rule. This resolves byte 8, measured two entries ago as "a count, 0x0b to 0x29" and now shown to size a payload exactly in eight independent cases within one level. Whether it also sizes the fixed-point payloads is the open half. Still open: why the fixed-point packets differ, and what the 16 bytes per sub-list record are -- the repeated word (0xcd, 0xc0, 0xc8, 0x78) differs per packet, so the records are not identical to each other. |
||
|
ReaperOfVeriod
|
db76319600 |
Find a second packet record magic interleaved with the first
Range measured: savestate slot 1, RAM 0x22a000..0x24037f. Result: a second four-byte record magic appears at comparable density beside 04 04 00 01 -- 02 01 00 01, ten occurrences against nineteen. The two interleave: every 02 01 00 01 sits between two 04 04 00 01 records. 0x22a6b8 04 04 00 01 13 00 00 30 70 6a 93 00 0x22a6c8 02 01 00 01 19 c0 4c 6e 00 00 10 00 00 00 00 00 0x22a6e8 04 04 00 01 16 00 00 30 f0 76 93 00 0x22a6f8 04 04 00 01 16 00 00 30 50 78 93 00 0x22a708 02 01 00 01 1e c0 58 6e 00 00 10 00 00 00 00 00 0x22a728 04 04 00 01 24 00 00 30 50 9c 93 00 The 02 01 00 01 records are 16 bytes like the others but carry no pointer -- bytes 12-15 are zero in all ten. Their fields are a count at 4-5 (0x14, 0x19, 0x1e), a constant 0xc0 at 6, a varying byte at 7 (0x30, 0x4c, 0x54, 0x58), 0x6e at 8-9, and 10 00 at 10-11. The 0x6e at bytes 8-9 matches the ".. c1 01 6e" the sub-list payloads carry at offset 20-23. That is a second correspondence, and unlike the one retracted in the previous commit it is a shared value rather than a shared position. Not settled: this still does not say what discriminates the two payload kinds among the 04 04 00 01 records, since those share the same magic and the same 0x30 at byte 11 and differ only in the count at byte 8. The 02 01 00 01 records sit between them rather than replacing them, so they are a companion structure rather than the missing discriminator. Next: compare the count at byte 8 against the payload's own length. |
||
|
ReaperOfVeriod
|
2d7de9f13e |
Correct the packet tag mapping: the tag does not select the payload
The previous commit mapped the disc tag vocabulary onto the three payload classes. A wider scan says that is wrong. Range measured: savestate slot 1, RAM 0x22a000..0x24037f -- 91008 bytes, 20 times the span scanned before. Result: 19 packet headers, and all 19 carry byte 11 = 0x30. There are no 0x6d or 0x80 tags anywhere in this range. Reading six payloads: rec 0022a6b8 tag 30 -> 8d 8c 80 80 89 87 7c 80 ... fixed point rec 0022a6e8 tag 30 -> 00 00 00 30 cd 00 00 00 ... sub-list rec 0022a6f8 tag 30 -> 92 90 84 80 94 94 86 80 ... fixed point rec 0022a728 tag 30 -> 00 00 00 30 c0 00 00 00 ... sub-list rec 0022a738 tag 30 -> 4c 4c 4b 80 4c 4c 4b 80 ... fixed point rec 0022a768 tag 30 -> 00 00 00 30 78 00 00 00 ... sub-list Records of identical tag alternate between two payload kinds. The tag does not select the payload. What went wrong: the previous entry took the 0x30 and 0x6d that head the payloads to be the disc-side tag vocabulary, and matched them to payload classes by that correspondence. The disc vocabulary at record byte 7 (0x00, 0x30, 0x6d, 0x80) and RAM record byte 11 are different fields that merely overlap in two values. Three payloads each made the coincidence look like a rule. What is actually established about the RAM header: 0-3 04 04 00 01, constant 4-7 zero, constant 8 a count, 0x0b to 0x29 across the 19 records 9-10 zero, constant 11 0x30, constant on all 19 in this range 12-15 an EE RAM pointer Byte 8 is the finding that survives: a small count varying per record, and the one field here doing arithmetic rather than tagging. Still open: what 0x30 at byte 11 means, what discriminates payload kinds given the tag does not, and 0x80, which has no sample in 91 KB of RAM. |
||
|
ReaperOfVeriod
|
0b1793d6af |
Map shrub packet tags onto three payload kinds
Dereferenced all nine pointers in the level 0 table that carry the 0x30 tag at byte 11 and an EE pointer at 12..15, reading 32 bytes each. They fall into exactly three classes, three payloads each, with no overlap. A, Q16.15 fixed-point vertex data (3 of 9): every high byte 0x80 or 0x7c, repeating on an 8-byte boundary, values differing per payload while the 4e 4e 4c 80 tail is shared by two. 00936a70 8d 8c 80 80 89 87 7c 80 ... 4e 4e 4c 80 0093cde0 8a 8a 7f 80 92 90 84 80 ... 4e 4e 4c 80 009cf4f0 6e 6f 68 80 6e 6f 68 80 ... 54 54 51 80 B, a 0x30 sub-list with a repeated word (3 of 9), structurally identical apart from that word (0xcd, 0xc0, 0xc8), all carrying 01 00 00 05 at offset 16 and a ".. c1 01 6e" at 20. 009376f0 00 00 00 30 cd 00 00 00 cd 00 00 00 ... 00998880 00 00 00 30 c0 00 00 00 c0 00 00 00 ... 009d2da0 00 00 00 30 c8 00 00 00 c8 00 00 00 ... C, an index/attribute list (3 of 9), both opening 00 c0 05 6d, then a count, then a run of small ascending u16. 00939700 00 c0 05 6d 3e 00 00 00 01 00 00 00 ... 009c1540 00 c0 05 6d 36 00 00 00 01 00 00 00 ... So the tag vocabulary measured on disc at record byte 7 -- 0x00, 0x30, 0x6d, 0x80 -- selects the payload: 0x30 class B, the 00 00 00 30 sub-list 0x6d class C, the 00 c0 05 6d attribute list 0x00 class A, the fixed-point vertex data 0x30 heads the class B payloads and 0x6d heads the class C payloads. That was flagged as a single coincidence two commits back; three payloads each makes it a rule. 0x80 has no sample among these nine and its meaning stays open. Correction: the record spacing in this table is not uniformly 0x30. The find output shows 0x30, 0x10, 0x30 between consecutive packet headers, and only 9 of the 5632 bytes scanned carry both the 0x30 tag and an EE pointer. The earlier "0x30 spacing" was inferred from a truncated hit list. |
||
|
ReaperOfVeriod
|
ad70e4c4be |
Follow shrub packet pointers: adjacent records hold unrelated payloads
The header pointers found in the previous commit are live, so they can be dereferenced. Two records 0x30 apart in the level 0 table at 0x22a6b8 were followed. Range measured: savestate slot 1, reading 0x00936a70 (from the record at 0x22a6b8) and 0x009376f0 (from the record at 0x22a6e8), 64 bytes each. The two payloads have nothing in common. 0x00936a70 is fixed-point vertex data: 8d 8c 80 80 89 87 7c 80 8d 8c 80 80 89 87 7c 80 4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80 4e 4e 4c 80 repeating Every high byte is 0x80 or 0x7c and the pattern repeats on an 8-byte boundary. As s16 these are large negatives in a narrow band around -32500, consistent with Q16.15 fixed point rather than f32, and with a stride of 4 s16 per vertex. 0x009376f0 is a tagged sub-list, not vertex data: 00 00 00 30 cd 00 00 00 cd 00 00 00 cd 00 00 00 cd 00 00 00 01 00 00 05 22 c1 01 6e 00 00 00 00 Structurally different: small integers, a repeating 0xcd, 0x30 tags in the high byte. Nothing here is position data. This supports the type reading of the header: the table is a tagged collection where the tag selects the payload kind, not a uniform array. It also rules out the alternative that the 16-byte tiling was one uniform record format with varying fields -- two adjacent records with unrelated payloads is not that. Still open: the mapping from tag value to payload kind. The 0x30 appearing in record bytes 8-11 in RAM also heads the second payload, which is suggestive but is one coincidence so far and is deliberately not written up as a link. |
||
|
ReaperOfVeriod
|
3854e52019 |
Show the loader rewrites shrub packet headers into RAM pointers
Answering the "what is the tag byte" question needed an anchor. Searching live RAM from scratch was the wrong way to get one: 04 04 00 01 matches unrelated data at 0x1ac988 and there is no way to tell a real hit from a coincidental one. The savestates are the anchor -- slot 1 is level 0, and searching one gives candidates that can be checked rather than guessed at. Range measured: savestate slot 1, ~/.config/PCSX2/sstates/SCUS-97268 (38996035).01.p2s, records at 0x22a6b8 onward, read with gc-trace read --state. Result: 40+ records beginning 04 04 00 01 00 00 00 00 at 0x30 spacing -- the packet framing identified on disc -- with the second half of each header rewritten: disc: 04 04 00 01 | 00 00 00 00 | 00 00 00 05 | 00 80 0e 6c RAM: 04 04 00 01 | 00 00 00 00 | 13 00 00 30 | 70 6a 93 00 Bytes 0-7 unchanged; bytes 8-15 are not. On disc bytes 12-13 are the constant 00 80 and byte 15 the constant 6c; in RAM bytes 12-15 hold a pointer -- 0x00936a70, 0x009376f0, 0x00937850, 0x00939c50 in consecutive records, all inside EE RAM and ascending across the table as a walk of consecutive objects would be. Byte 11 becomes 0x30 in every record seen, against a constant 0x05 on disc, and bytes 8-9 hold a varying small value (0x13, 0x16, 0x24). So the packet table at +0x40 is what the loader walks: span-relative on disc, absolute in RAM. This was inferred from one class before; these 40+ records show it directly. Consequence for the tag bytes: the vocabularies at record bytes 7 and 15 are real, but byte 15 sits in a field the loader overwrites with a pointer. Its disc values are read as data by something rather than stored as a tag. Not settled: whether byte 7's vocabulary is also rewritten, and whether 0x6d marks payload the loader expands. That needs these records compared one by one against their disc spans, which is cheap now that the anchor exists. |
||
|
ReaperOfVeriod
|
c83f5104d2 |
Document the PCSX2 test setup
PCSX2 was set up and working but existed only in this machine's filesystem and in people's heads, so every comparison looked like a one-off observation rather than something re-runnable. The setup section in docs/plan/pcsx2-verification.md now records where the emulator, the BIOS, the disc, the savestates and the PINE socket are, how a state is captured, and which command the disc integration tests need -- and AGENTS.md and README.md point at it from their orientation sections. One operational detail worth spelling out: the emulator's directory is lowercase (pcsx2), which is not what people type. |
||
|
ReaperOfVeriod
|
bbdb51e2f9 |
Verify level 1's tfrag lighting against the game
Savestate slot 2 is the second planet, so the comparison runs on a level with a different geometry extent, light bank and runtime-set population: level1: 401/401 headers, 401/401 pointers, bank byte-identical vertices compared: 26470 (105880 colour bytes) -- equal: 100.00% The live emulator agrees with the savestate byte for byte. Level 1 is a different planet, so the two fixes in the previous commit are not level-0 coincidences: the normal's z sign and the unlit-record base colour both hold across 268,280 colour bytes on two levels. |
||
|
ReaperOfVeriod
|
69a8398f6e |
Verify the tfrag lighting magnitudes against the running game
The harness that compares the port against the game's own memory existed but had never been compiled: tools/trace did not build (27 errors, all in the argument plumbing), and the tfrag_light.rs light-bank correction it motivated was unverified. This finishes it and runs it, and the run finds two real bugs. Result, on level 0 against three inputs -- the live emulator over PINE and two PCSX2 savestates from the same level, agreeing byte for byte: headers matching the disc on invariant bytes: 833/833 data pointers relocated as table + disc offset: 833/833 light bank in RAM at 0x50ab20, byte-identical to the disc's vertices compared: 40600 (162400 colour bytes) equal: 162400 (100.00%) The locator finds the level's tfrags block in RAM by tfrag 0's bounding sphere and accepts a candidate only when every header matches the disc on the bytes the game leaves alone and every data pointer is the relocated form, so "the block was found" is itself a verification rather than an assumption. The bank is searched in RAM by its exact section bytes, which is what settles its framing: offset at +0x04, an i32 set count, twelve bytes of pad, then the sets. The previous reading (offset at +0x00, count at +0x04) was wrong. Two bugs the comparison found, both of which produced plausible output: 1. vertex_normal_bits negated x and y and left z positive, while NormalTable::normal negates all three. dA came out -0.707 where the game had +0.707, so max(dA, dA*wA) took the wrong branch and every vertex lit near its base colour: dark, but not wrong-dark. 95.88% agreement with the bug, 100.00% without. 2. A record whose select names a set the bank lacks drew transparent black. The game writes the record's base colour; 3,130 of level 0's 40,600 vertices do this (select 0x000f, set 15 -- a runtime slot the level's directional bank does not have). The vertex is unlit, not invisible. Both are now regression tests in the engine. The comparison prints the pass's per-vertex terms (light_vertex_terms) for the first mismatches, because a colour mismatch alone cannot be attributed and the comparison is worthless without attribution -- that is how the second bug was identified from a three-line read rather than guessed at. Docs: docs/plan/pcsx2-verification.md records the harness, the result, the two bugs and what to compare next; roadmap and level-assets.md stop carrying lighting magnitudes as an open item. Findings reach tests only as numbers -- no test reads a savestate, a dump or a live emulator. |