The shrub region's end was the last unlocated bound in core_data. It was not
in the core index, and unlike moby->tie it had no successor boundary to chain
against, so the last shrub class in every level ran out to the lump's end and
the aggregates over it (the "3.5M records / 28MB of shrub body") were
artifacts of that oversized span.
Range measured: for each level, from the last shrub class's geometry offset to
the constant trailer. Nothing outside that range is read.
Method: scan forward from the last shrub class for the pair (a) the constant
two bytes f6 7c sitting immediately in front of (b) a chunk signature -- a
16-byte header ending 0xff, ten zero bytes, then four or more 0xf0-tagged
offsets. The two halves are each weak alone, and the halves are measured
separately: the marker pair occurs 25 to 103 times in a 5 to 17 MB lump, so
matching it alone picks dozens of wrong offsets; the chunk shape matches
random data often enough to need four tagged offsets before it is believed.
Result: on all 27 levels the marker sits exactly two bytes before the first
chunk header, and the two parts of the signature are never found apart. No
chunk signature occurs inside any last shrub body, on any level. The three
levels without shrub (10, 25, 26) carry the same marker before the container
that closes tie instead, which is what makes the locator a property of the
region rather than a shrub rule.
Consequences:
- All 358 shrub class bodies are bounded and are whole 8-byte records. That
was not assertable while the last class in each level ran to the lump's end,
and it is the check the boundary earns its keep for.
- 9,600 records in total, against the 8,938 previously measurable in
successor-bounded spans; 662 of them sit in the 24 last-class bodies. The
shrub region totals 2,033,776 bytes across the 24 levels that have one.
- The chunk container is 25,775,850 bytes across 27 levels (1,236 chunks,
22,562 records). The container measurements were previously taken over "the
900,000 bytes before the trailer", a window that did not align to its real
extent: on level 0 it is 690,302 bytes, and 27 of 1,123 chunks were
truncated at that invented edge. chunk_containers_resolve_on_every_level now
reads the region from core_layout instead of measuring backwards from the
end of the lump.
core_layout takes core_data now: the container's start is the one bound that
cannot be derived from the index. It gains the chunks region, so every byte of
level-specific core_data belongs to a named region. Where the signature is not
found the end stays None -- the region is left visible rather than clamped to
the tail.
Still open: the marker's meaning (constant across 27 levels, at exactly this
position, unnamed) and the shrub body payload's encoding.
Range measured: all 27 levels, 1123 chunks, over the 900000 bytes before the
constant trailer.
Method: compare header byte 14 against the number of offsets actually resolved
for that chunk.
Result: byte 14 equals the record count on 1123 of 1123 chunks, zero
mismatches. This is the first genuinely decoded field in this container rather
than a measured correlation.
The parser now truncates the offset table to header byte 14, so a stray
tagged word in a chunk's record data cannot silently extend the table and put
the count at odds with its own header.
Two more header bytes are near-constant and read as ASCII: byte 13 takes only
0x45 (1076 chunks) and 0x46 (47); byte 9 takes 0x44 (26), 0x45 (1084) and
0x46 (13). Those are D, E and F -- a three-letter tag at two byte positions
with different distributions. Recorded as suggestive, not established: the
other 14 header bytes are high-entropy and read as ASCII at no shift tried, so
the D/E/F may be incidental. Worth testing against a known type table.
Still open: the 14 opaque header bytes, whether bytes 9 and 13 are type tags,
which record fields are index/length/offset, and whether a chunk is moby_anim.
gc_formats::chunks parses the container that fills the region after the last
shrub class body and before the constant trailer:
+0x00 16 bytes, last byte 0xff (opaque, differs per chunk)
+0x10 10 zero bytes
+0x1a N tagged offsets, top byte 0xf0
Each tagged offset is 0xF0000000 | low24, and a record is read at
chunk_start + low. Resolving that way gives records sharing a field layout,
which is what distinguishes the container from a coincidental byte pattern.
Fields are not named. The 16-byte header is opaque, the 0xf0 tag is
unexplained, and it is not established that a chunk is moby_anim -- a
49 x 368-byte table inside this region is byte-identical on all 27 levels,
which is evidence against that identification. What the parser provides is
chunk and record extraction, which is what the analysis established.
The gated test measures, over all 27 levels: 1123 chunks, 19546 records, 50
distinct strides, 12.4% of byte positions constant across records (per-chunk
range 2.8% to 75%). Chance gives a constant position probability of 1/256^n.
Three things the test got wrong first, recorded because they are the traps:
- A per-chunk threshold of 20% constant positions. Individual chunks run from
2.8% to 75%; the aggregate is the invariant, not the chunk. Asserting 20%
per chunk was asserting a property the format does not have.
- Requiring every offset to resolve. 27 chunks truncate at the window edge,
because the 900000-byte window does not align to chunk boundaries. Counted
now rather than asserted away.
- read_u32 in the chunk signature check was big-endian while Buf::u32 is
little-endian. The tag lives in the high byte of a little-endian word, so
the mismatch made every level report zero chunks while parse_chunk -- which
goes through Buf -- was correct. Worth remembering that the two paths can
disagree on a format this young.
Tests: 190 -> 191.
ReaperOfVeriod