The shrub region's end was the last unlocated bound in core_data. It was not
in the core index, and unlike moby->tie it had no successor boundary to chain
against, so the last shrub class in every level ran out to the lump's end and
the aggregates over it (the "3.5M records / 28MB of shrub body") were
artifacts of that oversized span.
Range measured: for each level, from the last shrub class's geometry offset to
the constant trailer. Nothing outside that range is read.
Method: scan forward from the last shrub class for the pair (a) the constant
two bytes f6 7c sitting immediately in front of (b) a chunk signature -- a
16-byte header ending 0xff, ten zero bytes, then four or more 0xf0-tagged
offsets. The two halves are each weak alone, and the halves are measured
separately: the marker pair occurs 25 to 103 times in a 5 to 17 MB lump, so
matching it alone picks dozens of wrong offsets; the chunk shape matches
random data often enough to need four tagged offsets before it is believed.
Result: on all 27 levels the marker sits exactly two bytes before the first
chunk header, and the two parts of the signature are never found apart. No
chunk signature occurs inside any last shrub body, on any level. The three
levels without shrub (10, 25, 26) carry the same marker before the container
that closes tie instead, which is what makes the locator a property of the
region rather than a shrub rule.
Consequences:
- All 358 shrub class bodies are bounded and are whole 8-byte records. That
was not assertable while the last class in each level ran to the lump's end,
and it is the check the boundary earns its keep for.
- 9,600 records in total, against the 8,938 previously measurable in
successor-bounded spans; 662 of them sit in the 24 last-class bodies. The
shrub region totals 2,033,776 bytes across the 24 levels that have one.
- The chunk container is 25,775,850 bytes across 27 levels (1,236 chunks,
22,562 records). The container measurements were previously taken over "the
900,000 bytes before the trailer", a window that did not align to its real
extent: on level 0 it is 690,302 bytes, and 27 of 1,123 chunks were
truncated at that invented edge. chunk_containers_resolve_on_every_level now
reads the region from core_layout instead of measuring backwards from the
end of the lump.
core_layout takes core_data now: the container's start is the one bound that
cannot be derived from the index. It gains the chunks region, so every byte of
level-specific core_data belongs to a named region. Where the signature is not
found the end stays None -- the region is left visible rather than clamped to
the tail.
Still open: the marker's meaning (constant across 27 levels, at exactly this
position, unnamed) and the shrub body payload's encoding.
Comparing all 27 levels byte-for-byte from the end: the last 231,952 bytes
(0x38A10) of every level's core_data are identical. Level-specific content
therefore ends at len - 0x38A10, which is a real bound for the final region
and is not core_data.len().
This closes shrub. Across the 24 levels that have a shrub region the bound
gives 24,442,928 bytes total, and the disc test asserts on every level that
the region ends strictly inside the lump rather than at its end.
What the trailer is remains unestablished -- fixed data, or padding from a
fixed-size decompression buffer -- but it is constant, which is all a bound
needs.
The data between the last shrub class and the trailer is high entropy: 183 to
256 distinct byte values per 4 KB block, mean 254, where texture data sits
under 200. It is real content of unknown type and is the most likely home for
moby_anim, now that the range between collision and moby has been ruled out
as texture.
A lump shorter than the trailer now yields end: None rather than a backwards
bound, covered by a test.
Printing the region table next to the header fields makes the gap visible
without having to remember to consult core_layout, and it paid for itself
immediately.
On level 0 the collision region comes out 6216960 bytes, from 2615232 to
the first moby class at 8832192. Collision's own header declares a payload
of 903,248. So roughly 6.2 MB sits between the collision header and the
first moby class, and almost none of it is collision.
The successor rule is correct -- a region does end where the next one
begins -- but "the next" is not yet known there. collision..moby is
really collision + <something unnamed> .. moby. The name is wrong, not
the bound.
Candidates for the space, rough order of likelihood: moby and tie model
animation data, which is what moby_anim would need, then volumes, then
per-class runtime structures. Tfrag texture texels are already placed and
sit earlier in the lump.
Worth resolving before starting moby_anim, since that format is the most
likely tenant. Until then, any probe reading between the collision header
and the first moby class is reading something the region table does not
describe.
Reading these formats by unpacking raw offsets out of core_index.bin has
cost real time. The index interleaves single-word region offsets with
count/offset pairs, so miscounting one shifts every field after it: that
produced a page of implausible collision headers on all 27 levels, sent
me chasing a tie-breakoff that turned out not to exist, and nearly had me
"fix" a stride bug in code that was already right.
probe dumps a level's core header and all three class tables through
LevelCoreHeader, so a field name printed is the field name the parser
uses. It also reports how many entries carry geometry inside core_data,
which is what pins the class table strides rather than guessing at them:
shrub reads 7/7 sane at 0x30 against 4/7 read at 0x20.
Reading a structure by hand in a shell still works; this just stops it
being the default, and stops the offset arithmetic being re-derived every
time a new region is investigated.
ReaperOfVeriod