e6eecb871f9f5422139bf1ccc88b65c109c91d9e
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ReaperOfVeriod
|
e6eecb871f |
Decode the tie LOD block: it is a table of 16-byte records
The tie block's word at +0x00 was read as an element count, and the reading failed in a way worth recording: the payload length divided by it is a whole number on 0 of 922 blocks, at any record size and any header up to 512 bytes. That is the correct measurement of the wrong hypothesis -- the word never described the payload. Range measured: every LOD block of level 1 whose word is a non-zero multiple of 16, plus the two read by hand in full. Method: locate each block's record table in PCSX2's RAM by its twelve field bytes, which are the only distinctive 12-byte run in a block, then compare every record against its disc copy. Each block has several copies in RAM, and the one that matches is the loaded one -- checking the first hit is what made the first attempt report 0 of 16. Result: on 16 of 16 blocks, every record's leading word has become table_address + word, and the other twelve bytes are byte-identical to the disc. So a block is [16-byte header][record table][vertex payload], the records are 16 bytes -- a leading u32 pointer plus twelve bytes of fields -- and the header's word at +0x00 is the byte offset from the block start to the payload, that is, the length of the table. That is what the multiple-of-16 was: the table holds 16-byte records. It is never zero because there is always at least the first record, and its 42 distinct values are table lengths. Each record's pointer is relative to the table start, relocated at load to table + word. TieBlockHeader::data_offset and TieBlockHeader::records carry it, bound by the offset, with TieRecord for the twelve field bytes, and the disc test asserts the table is bounded on every block. What the records' fields name and what the payload holds are still open -- the payload is known to be variable-length, float-shaped, not LZ77 and not VIF1. |
||
|
ReaperOfVeriod
|
291153a0d9 |
Find the tie LOD block terminator; packets == 0 means no block at that LOD
parse_tie_block walks to a 0xf6 terminator that every real block carries, on
2177 of 2177 across all 27 levels, followed by zero padding. This is what makes
the blocks parseable: packets[1] bounds one on only 461 of 1864 classes, so the
terminator is the only general way to find one.
The terminator was worth a control, because 0xf6 could have been byte-frequency
coincidence. It occurs in payloads at 0.83x the uniform rate -- slightly LESS
often than chance -- while the last non-zero byte is 0xf6 on 909 of 922 blocks in
the first subset, against 17 of 922 for the same blocks cut at a random point. So
it is a terminator, not a common byte.
packets == 0 means the class has NO block at that LOD: of 3,728 (class, LOD)
slots, 2177 point at a block and 1551 are exactly zero.
Corrected header, over all 2177:
+0x00 count multiple of 16 on all; 42 distinct; never zero
+0x02 zero on all
+0x04 exactly four values: 0x501 (1069), 0xa02 (720), 0xf03 (350),
0x1404 (38)
+0x06 16 distinct, never zero
+0x08 180 distinct, never zero
+0x0a 61 distinct, never zero
+0x0c 125 distinct, never zero, clustered 0x043f..0x0443
+0x0e 11 distinct, zero on 2162 -- the only non-universal field
+0x0c clusters around 1087-1091, close enough to a block length to be worth
stating, but it is NOT the length: the difference against the true payload length
scatters from -12 to -172.
A correction to the previous commit. It reported 3,728 blocks all terminating and
packed_texture taking 842 distinct values, and on that basis retracted the
four-value claim. Both numbers were wrong the same way: the scan did not skip
packets == 0, so it computed class_base + 0 and walked forward from the CLASS
BODY, finding a terminator there by accident. That contaminated 1551 of the 3728.
With the zeros excluded the slot takes exactly four values on all 2177 blocks.
The four-value claim was right; my retraction of it was wrong; and the retraction
was made on a number from the same contaminated set. A retraction is not
automatically better than the claim it retracts -- it needs its own bounds.
Still open: the payload's records. Established this round: blocks are
self-delimiting, absent LODs are marked by a zero packets word, and the header is
fully characterised. Not established: any record boundary inside the payload,
which stays variable-length.
|
||
|
ReaperOfVeriod
|
02342de31b |
Decode the tie LOD block header; establish the payload is variable-length
TieBlockHeader parses the block's own 16 bytes, measured over the 922 blocks where "the next packets offset ends this block" holds, across all 27 levels. A real-disc test asserts every figure. +0x00 count multiple of 16 on 922/922; 16..2096; 72 distinct disc-wide +0x02 zero on 922/922 +0x04 ONE OF FOUR VALUES DISC-WIDE: 0x501, 0xa02, 0xf03, 0x1404 +0x06 15 distinct, 0x0206..0x0512 +0x08 135 distinct, 0x0b05..0x4ee4 +0x0a 45 distinct +0x0c 94 distinct, 0x010e..0x0553 +0x0e zero on 914/922 +0x04 is packed rather than a magnitude: four values across the whole disc with the two bytes moving independently, low 1-4 and high 5/10/15/20 in lockstep. That it is a texture index paired with a scale is a guess; that it is two fields is not. The count is not a record count -- block_size minus the header is divisible by it on 0 of 922, for any record size and any header up to 512 bytes. On the payload, a control was needed and changed the answer. Reading blocks as 3 x f32 triples, 53-64% of reads land within +/-5000 of origin, against 16% for a uniform-random control of identical length. So the content is float-shaped and genuinely not random. But the rate is flat across every start offset and every stride tried, 53-65% throughout. A statistic that cannot separate the right stride from the wrong one cannot identify a stride -- rule 4's exact case. Flatness plus a count that does not tile means the records are variable length. That killed two readings. A VIF1 command walk consumes 0 of 88 blocks exactly, so it is not a VIF1 CPU-format stream. wad::decompress rejects the block at every start offset, so it is not WAD LZ77. Scanning every (start, stride) pair from 0x10 to 0x7f for byte positions with a small distinct-value vocabulary finds none: every position takes all 256 values at every alignment. For float payload that is expected, so the scan found nothing rather than confirming anything -- worth saying, because "no fields found" here is uninformative, not evidence against fields. Still open: what the payload holds. Established: variable-length, float-shaped, non-random, not LZ77, not VIF1. Not established: any record boundary. |
||
|
ReaperOfVeriod
|
855e640f39 |
Establish the 16-byte tie record on measurable classes; packets[0] is a constant
The previous commit reported "3580 non-empty LOD blocks compute as divisible by 16" and then said the figure should not be trusted, because it came from the "next packets offset ends this block" rule. That rule is unsound: packets is strictly increasing on only 461 of 1864 classes. On the other 1403 the two offsets are unordered and cannot both be block starts into one ascending region. Restricting to the 461 classes where the rule does hold, 922 of 922 consecutive block sizes are a multiple of 16. That is the evidence for a 16-byte record, and it is now scoped to the subset where it can be checked rather than claimed over all 1864. The other 1403 are unmeasured, not measured-and-different. packets[0] is a constant, not an offset: it reads 256 on 1864 of 1864 classes, while packets[1] and packets[2] vary widely and land at different addresses in core_data on every class. Read as a block start, 256 points just past the 0x48-byte header -- which is how it was being used, and exactly how a field gets mistaken for a pointer. It is now TIE_PACKETS0 and nothing depends on it. Still not decoded: what the 16 bytes hold. i16[0] at +4 takes 8969 distinct values on level 0 and is not the two-valued field a 12-unit sample suggested; the word at +0 is not a run offset. The test pins all four numbers -- packets[0] constant, the 922/922 divisibility, the 461/1864 ordering, and the 0/1864 monotonic disproof -- and asserts that the ordering is *not* universal, so a future change there fails loudly instead of quietly invalidating the record size. |
||
|
ReaperOfVeriod
|
60e3f129d8 |
Resolve both tie indirections on all 1,864 classes, and disprove a sample artifact
resolve_tie_class walks the chain, which is two steps and both are easy to get
wrong:
1. The tie class table lives in core_index.bin, not core_data.bin. On level 0
the tables chain as moby at 192, 238 x 32 bytes later tie at 7808, 74 x 32
later shrub at 10176, which closes. Reading the same table out of
core_data.bin produces plausible garbage rather than an error.
2. ClassEntry::offset_in_asset_wad addresses core_data.bin, despite the field
name and despite core_data.wad sitting alongside it.
Verified: all 1864 tie classes resolve, o_class agreeing on every one, and all
5592 LOD block offsets land inside core_data.
Disproved, and this is the part worth reading: the word at +0 of a 16-byte unit
inside an LOD block. On the first twelve units of level 0's class 0 it rises
strictly -- 288, 1520, 2848, 3968, 5088, 6096, 7312, 8528, 9792, 10976, 12016,
13072 -- which reads exactly like a linked list, and the accompanying i16s looked
quantised, two distinct values in i16[0] and three in i16[1].
Across the population it is strictly increasing on 0 of 1864 blocks, and i16[0]
takes 8969 distinct values on level 0 alone rather than two. The sample was
twelve units out of 1,724,334. Both the monotonic reading and the quantisation
were artifacts of that sample.
That is the pooled-claim error AGENTS.md rule 3a exists to prevent, committed
almost verbatim: the first twelve units looked clean, and I was one step from
writing "u32[0] is a linked list of runs" into a format spec. The test now
asserts the disproof, so it cannot creep back.
Not established: the block's record size. 3580 non-empty LOD blocks compute as
divisible by 16, but some packets offsets run backwards, so the "next offset
delimits this block" end is wrong for them and that figure should not be trusted.
The tie layout stays partial: indirections and header done, records not.
|
||
|
ReaperOfVeriod
|
79025b22d6 |
Decode the tie class header prefix that RAC1 and RAC2 agree on
Tie is the easy one of the three geometry subsystems -- instanced static props, no skeleton -- so it is the right place to start. RAC1's class header is 0x80 bytes. RAC2 agrees with it over the first 0x48 and diverges after. From 0x48 the RAC1 layout does not describe RAC2 data: lod_info at 0x50 reads back as noise, packet_count and texture_count are zero on every class examined, and the normals offset at 0x0c points 117,788,434 bytes into a 16.9 MB lump. Only the agreed-on prefix is decoded. What makes it trustworthy: * o_class at 0x44 equals the core index's class number for all 1,864 tie classes across all 27 levels. A field landing on the right number that often is not a coincidence, and it pins the header base. * The LOD distances are ordered 0 < near < mid < far for 1,784 of 1,864. The 80 exceptions are real, unexplained, and are reported rather than asserted away. Level 0's first tie class has distances 12.0 / 25.0 / 40.0, a plausible prop LOD band, and packet offsets 256, 20,592 and 26,048. The rest is deliberately not guessed at. A field order that produces plausible output and is wrong is the specific failure this project keeps recording, three times now: the gameplay extent, the tfrag record framing and the class tables were all "reasonable" and all wrong. |