The "3 of 21 blocks not texture-like" figure from the last commit was an
artifact of sampling every 256 KB and missing everything between. Re-scanned
at 4 KB granularity there are about 100 such blocks, and looking at them
rather than only counting them, they are smooth gradient ramps -- sky and
fog texture. A noisy gradient has a high distinct byte count without being
code, which is exactly the mistake the byte-count heuristic invites.
So collision..moby is entirely texture data. There is no separate moby_anim
region there, and two claims about this range have now been wrong: the
"6.2 MB of unknown data" and the "3 unexplained blocks". Both came from
sizes and counts taken without looking at what they described.
moby_anim is therefore still unlocated, but it is not here. The remaining
candidate is the unresolved tail after the last shrub class -- ~925 KB of
real, non-zero data on level 0 that nothing accounts for.
Recording the two corrections together rather than one at a time, because
the mechanism is the same in both: a heuristic count is not an
identification.
The previous commit claimed ~6.2 MB of unidentified data sat between the
collision header and the first moby class, and speculated it might be
moby_anim. That was wrong, and it came from quoting a size before bounding
the thing it measured.
Looking at the bytes rather than only the offsets: the range is dominated by
8-bit indexed texture data. Byte-value concentration distinguishes them
cleanly -- a texture block draws on a small palette, code and collision data
does not. 18 of 21 sampled 4 KB blocks fall under 200 distinct byte values,
against 249 for tfrags and 209 for collision.
The real layout on level 0:
collision 2615232 .. 3518544 (header plus its declared 903,248 payload)
textures 3518848 .. ~8832192
moby 8832192 ..
The texture table's data_offset values stop at 5326208, which is what made
the gap look genuine. Mip chains extend well past the last entry's offset,
so sizing a region from a table of first-level offsets understates the data
it points at.
Three of the 21 blocks are not texture-like, so something is mixed in that
is still unidentified. It is not 6.2 MB of moby_anim.
This is the third number this session that looked load-bearing and turned
out to rest on an unbounded measurement. The pattern is consistent enough
to be worth stating plainly: sizes derived from tables or offsets are not
evidence until the bytes underneath have been looked at.
Printing the region table next to the header fields makes the gap visible
without having to remember to consult core_layout, and it paid for itself
immediately.
On level 0 the collision region comes out 6216960 bytes, from 2615232 to
the first moby class at 8832192. Collision's own header declares a payload
of 903,248. So roughly 6.2 MB sits between the collision header and the
first moby class, and almost none of it is collision.
The successor rule is correct -- a region does end where the next one
begins -- but "the next" is not yet known there. collision..moby is
really collision + <something unnamed> .. moby. The name is wrong, not
the bound.
Candidates for the space, rough order of likelihood: moby and tie model
animation data, which is what moby_anim would need, then volumes, then
per-class runtime structures. Tfrag texture texels are already placed and
sit earlier in the lump.
Worth resolving before starting moby_anim, since that format is the most
likely tenant. Until then, any probe reading between the collision header
and the first moby class is reading something the region table does not
describe.
Region spans were being re-derived per format while probing, usually
running from one known offset to the next with the last span running to
core_data.len(). On level 0 that gave the final shrub class a 925,008-byte
span absorbing most of the lump's tail, and every aggregate computed over
it -- record counts, value distributions, "this looks random" -- was
silently wrong. Nothing errored, which is what made it survive two
commits and a correction.
core_layout computes the layout once: each region ends where the next one
begins, class regions start at the lowest non-zero geometry offset in their
table, and a region whose successor is unknown gets end: None rather than a
clamped guess. Zero class entries mean absent, not position zero -- the
same trap occlusion and collision each fell into separately.
The disc test asserts what the module is actually for: that the unknown is
visible. Shrub's end is unresolved on the 24 levels that have a shrub
region, and that is asserted rather than hidden.
It also found that not every level has every region -- level 10 has neither
occlusion nor shrub -- so the test checks ordering against the canonical
sequence instead of demanding an identical set.
Still open: where the shrub region ends. Texture texels cannot bound it;
they sit before moby, not after shrub. See docs/formats/core-data-regions.md.
ReaperOfVeriod