Most of the chunk container is level-independent: 65 of 74 signatures

Comparing every chunk record against the same record index in the other 26
levels, across all 74 signatures:

    65 of 74   every record byte-identical in all 27 levels
     9 of 74   level-specific
     0 ragged  no signature appeared with different record counts per level

The nine that vary:
    [5d 44 13] [75 42 ef] [7f 44 ee] [82 c4 2f] [96 c3 37]
    [a2 44 a5] [c6 c3 ef] [df c3 ea] [f3 c4 b9]

This reframes the container. 65 of 74 chunk types are static tables the game ships
unchanged in every level -- constants and lookups, not level content. The largest
by record count are all in that group: [38 c4 7e] stride 328 with 90 records per
level, [a9 c4 d8] stride 308 with 90, [3d c4 9e] stride 368 with 77.

The consequence is about method, and it is worth being blunt about. For a
level-specific type, comparing levels is the strongest lever available: a field
that moves with the level is identifiable by correlating it with something known.
For the other 65 there is nothing to compare -- one copy of the data, repeated 27
times -- so they must be decoded structurally, from what the bytes are. A future
decoder that reaches for per-level comparison as its first tool will find it
useless on the majority of the container.

[75 42 ef] is both level-specific AND the one signature with three strides. A
type that is both variable-length and level-varying is the most informative one
here, and is where per-type field decoding should start.

How this was found, including the wrong turn:

The first look at [53 c4 6c d7 9a 43] (stride 108) showed 45 of 108 byte positions
varying WITHIN a chunk and 810 records all distinct -- which reads like per-record
data. Counting distinct values per u32 slot across all 810 records contradicted
that: 29 to 30 distinct values across 27 levels, i.e. about one per level.

Both readings were wrong because they measure different things. The 45 positions
vary between records within a level; the u32 counts were low because the same 30
values recur in every level. Comparing record i across levels settled it: all 30
records are byte-identical in all 27 levels, and zero byte positions vary across
levels.

Had the u32 distinct counts been the only measurement, the conclusion would have
been "one value per level" -- the opposite of the truth, describing a per-level
parameter table that does not exist. The within-chunk and across-level views had
to be taken together to get a true one.

Asserted in most_chunk_signatures_are_level_independent. 242 tests, clippy clean.
This commit is contained in:
2026-10-13 07:43:44 +02:00
parent 59dba3e58d
commit af62032ddc
3 changed files with 145 additions and 1 deletions
+51
View File
@@ -2336,3 +2336,54 @@ look like a layout and would otherwise be rediscovered and over-read.
Neither of these decodes any record's *fields*. What is now settled is the
shape the decoder has to handle: uniform stride per type, 4-byte aligned, with
one named exception carrying three strides.
## Most of the chunk container is level-independent
Comparing every chunk record against the same record index in the other 26
levels, across all 74 signatures:
65 of 74 signatures: every record byte-identical in all 27 levels
9 of 74 signatures: level-specific
0 ragged -- no signature appeared with different record counts per level
The nine that vary:
[5d 44 13] [75 42 ef] [7f 44 ee] [82 c4 2f] [96 c3 37]
[a2 44 a5] [c6 c3 ef] [df c3 ea] [f3 c4 b9]
This reframes the whole container. **65 of 74 chunk types are static tables the
game ships unchanged in every level** -- constants and lookups, not level
content. The largest by record count are all in that group: `[38 c4 7e]` at
stride 328 with 90 records per level, `[a9 c4 d8]` at 308 with 90, `[3d c4 9e]`
at 368 with 77. Each contributes the same bytes to all 27 levels.
The practical consequence is about method. For a level-specific type, comparing
levels is a powerful lever: a field that changes with the level is identifiable
by correlating it with something known. For the other 65 there is nothing to
compare -- there is one copy of the data, repeated 27 times -- so they have to be
decoded structurally, from what the bytes *are*, not from how they vary. Any
future decoder that reaches for per-level comparison as its first tool will find
it useless on the majority of the container.
`[75 42 ef]` appears in the level-specific list, and it is also the one signature
with three strides. A type that is both variable-length and level-varying is the
most informative one in the container, and is where per-type field decoding should
start.
### How this was found, and the near-miss before it
The first look at one signature (`[53 c4 6c d7 9a 43]`, stride 108) showed 45 of
108 byte positions varying *within* a chunk, and 810 records all distinct --
which reads like per-record data. Counting distinct values per `u32` slot across
all 810 records contradicted that immediately: 29 to 30 distinct values across 27
levels, i.e. roughly one per level.
Both readings were wrong, and the resolution is that they measure different
things. The 45 positions vary *between records within a level*. The u32 distinct
counts were low because the same 30 values recur in every level. Comparing record
*i* across levels settled it: **all 30 records are byte-identical in all 27
levels**, and zero byte positions vary across levels.
Had the u32 distinct counts been the only measurement taken, the conclusion would
have been "one value per level" -- which is the opposite of the truth, and would
have described a per-level parameter table that does not exist.