Going-Commando/docs/formats/core-data-regions.md

core_data region bounds

How a level's core_data lump divides up, and how each region is bounded.

The rule

A region's end is the next region's start, never core_data.len(). The last region in a layout is the exception, and when its successor is unknown the end is None rather than a guess.

This exists because region bounds were being re-derived per format during probing, usually by running from one known offset to the next with the last span running to the end of the lump. On level 0 that made the final shrub class a 925,008-byte span absorbing most of the lump's tail. Every aggregate computed over it -- record counts, value distributions, "this data looks random" -- was silently wrong, and nothing errored.

gc_formats::regions::core_layout computes these once. Callers use it rather than inventing their own.

Layout

region	start	end
tfrags	0	occlusion
occlusion	occlusion	sky
sky	sky	collision
collision	collision	first moby class
moby	first moby	first tie class
tie	first tie	first shrub class
shrub	first shrub	lump length − 0x38A10

Class regions start at the lowest non-zero geometry offset in their class table. A zero entry means absent, not position zero -- the same trap the occlusion and collision headers each fell into separately.

Not every level has every region: level 10 has no occlusion and no shrub.

The constant trailer

Comparing all 27 levels byte-for-byte from the end: the last 231,952 bytes (0x38A10) of every level's core_data are identical. Level-specific content therefore ends at len - 0x38A10, and the final region is bounded there rather than at core_data.len().

This is what closes shrub. Across the 24 levels that have a shrub region it gives 24,442,928 bytes total, and it is the same bound the disc test asserts on every level.

What the trailer is is not established. It may be fixed data, or padding from a fixed-size decompression buffer. Either way it is constant, which is all that is needed to bound the tail.

Open: what was in the region before it

The offset ending the shrub region is not in the core index.

Texture texels do not bound it. They sit before moby, not after shrub: on level 0 the texel range is 3539328..5326208 while the last shrub class starts at 16044288, leaving ~925 KB of real, non-zero data after the last shrub class that nothing yet accounts for.

core_layout reports end: None for shrub on the 24 levels that have one, and the disc test asserts it, so the gap stays visible rather than being papered over by clamping to the end of the lump.

Correction: it is mostly texture data, not an unnamed region

Adding the region table to gc-extract probe made the gap visible immediately, which is the point of printing it.

On level 0:

collision       2615232 .. 8832192       6216960 bytes

but collision's own header declares a payload of 903,248 bytes. So the range between the collision header and the first moby class is 6.2 MB, and almost none of it is collision.

The first conclusion drawn from this -- that ~6.2 MB of unidentified data sat between collision and moby, possibly moby_anim -- was wrong, and it was wrong because of the same failure that produced the other two bad numbers this session: quoting a figure before bounding it.

Checking the bytes rather than just the sizes: the range is dominated by 8-bit indexed texture data. Byte-value concentration (a texture block uses a small palette; code and collision data does not) puts 18 of 21 sampled 4 KB blocks under 200 distinct byte values, against 249 for tfrags and 209 for collision.

So the real layout is:

region	start	end
collision	2615232	3518544
textures	3518848	~8832192
moby	8832192	first tie class

The texture table's data_offset values only reach 5326208, which is why the gap looked real: mip chains extend well past the last entry's offset. A region sized from a table of first-level offsets understates the data it points at.

The "3 of 21 blocks not texture-like" figure in the previous revision was an artifact of sampling every 256 KB. Re-scanned at 4 KB granularity the whole range is texture data: ~100 blocks stand out, and looking at them they are smooth gradient ramps -- sky and fog texture, exactly what gradient texels look like. A noisy gradient has a high distinct byte count without being code.

So the range between collision and moby is entirely texture data, and there is no separate moby_anim region there. Combined with the correction above, two claims made about this range have now been wrong: the "6.2 MB of unknown data" and the "3 unexplained blocks".

Where moby_anim actually lives is therefore still open, but it is not here. The remaining candidate is the unresolved tail after the last shrub class, which on level 0 is ~925 KB of real, non-zero data that nothing yet accounts for.

Where shrub data ends, and the region after it

Range measured: for each level, the last shrub class's data_offset field (class + 0x2c) through to the start of the constant trailer. Nothing outside that range is read.

Method: the last shrub class's body is an array of 8-byte records whose final two bytes are 00 00. Scanning forward in 8-byte steps for the first record whose trailing pair differs locates the body's end. This is a search for a specific stated pattern, not a statistic, so it either finds the boundary or reports that it did not.

Result: on 24 of 24 levels the boundary lands exactly on an 8-byte record boundary. Body lengths are 192, 208, 224 or 240 bytes -- the same sizes seen when shrub classes were grouped and diffed earlier.

On level 0:

what	range	bytes
last shrub class data	16046800 .. 16047040	240
next region	16047040 .. 16737344	690,304
constant trailer	16737344 .. 16969296	231,952

So the 690,304 bytes between the last shrub class and the trailer are a distinct region, not shrub data. What they contain is not established.

Two things are worth noting about how this was found, given the rule in AGENTS.md: the earlier NaN-filler search found nothing at the start of this range and was about to conclude the body had zero length, which would have been wrong. The last shrub class is a real class with real data, and the NaN filler only appears in degenerate ones. Checking the actual byte pattern rather than concluding from a failed heuristic is what caught it.

First structure in the region after shrub

Range measured: level 0, 16047040 .. 16737344 (690,304 bytes), starting at the first byte after the last shrub class's 240-byte body. Nothing outside that range is read.

Method: reading the bytes at +0x20 as u32 shows ten consecutive entries whose low halves increase by a constant 0x94 (148), followed by a zero. Read as u32 the high half is 0xf000 for the first nine and 0x0a40 for the tenth, so the table is ten entries long and ends at +0x48 (absolute 16047112).

Result, stated plainly: a ten-entry table on a 148-byte stride. What the entries mean, what 148 refers to, and why there are ten, are all unestablished. Ten does not match any count already known for this level -- 238 moby classes, 74 tie, 7 shrub, 93 tfrag textures -- so this is not yet tied to any structure elsewhere in the lump.

Follow-up: the 148 stride is a coincidence

Treating the ten entries' low halves as offsets from the region start lands on blocks exactly 148 bytes apart, the first nine of which begin 08 00 followed by a 16-bit id stepping by 0x40. That is a convincing-looking structure and it does not survive being extended: walking the same 148-byte stride across the rest of the region, only 14 of 4657 blocks start with 08 00. The first nine agree and then the pattern breaks.

So the ten entries are not a pointer table. The constant stride held for eight consecutive values, which is far more than chance should produce and far less than a structure needs.

Recorded because the lead was committed as a lead and this is where it goes. Nothing else about the region has been identified: it is real, non-zero data of unknown type, bounded, and unopened beyond these ~150 bytes.

The region after shrub has a repeating structure at stride 368

Range measured: level 0, 16047040 .. 16737344, sampled as the first 65,536 bytes. Cross-checked on every level that has the region, each sampled the same way from its own region start.

Method: autocorrelation. For each candidate stride, count how often a 4-byte sequence at offset i repeats at i + stride, as a fraction of comparable positions. A random control of the same size is measured identically and reads flat at 0.0000 for every stride, so the statistic discriminates.

Result:

stride	region match rate
368	0.1110
340	0.0935
337–339	0.026–0.029 (background)
random control	0.0000

Stride 368 is the strongest peak. On level 0, 340 is a secondary peak well clear of the background; the two differ by 28, which is unexplained.

Cross-check: the peak stride was recomputed independently for every level that has the region, and all 24 peak at exactly 368. That is the part that makes this worth recording -- a single level's peak could be a property of that level's data, but 24 of 24 agreeing is a format invariant.

What a 368-byte record is remains unestablished. This is the same position the shrub body was in before differential analysis identified its records, and the same method is the obvious next step.

Differential analysis at stride 368: no constant substructure

Range measured: level 0, 16047040 .. 16737344. 1,875 whole 368-byte records from the region start; variation checked across the first 60.

Method: the same differential analysis that identified the shrub body -- compare records byte by byte and record which positions differ.

Result: all 368 positions vary. There is no constant field, no padding and no header inside a record. Unlike the shrub body, where differential analysis immediately separated a varying payload from a constant two-byte tail, this format has nothing constant to separate.

That does not contradict the autocorrelation peak. Checking which byte positions carry the 368-stride matches over 20,000 positions: 168 exact 4-byte repeats, distributed evenly at 0.84% across all four byte offsets. So whole 4-byte sequences recur at exactly 368 while no individual position is constant -- consistent with records containing entries that repeat at a fixed relative index, rather than with a fixed-layout record.

So the 368 stride stands and the records remain unidentified. The shrub method failed here not because it was the wrong tool but because it needs something constant to latch onto, and there is nothing.

Sky opens with a 256-byte-row image

Range measured: level 0, 2192576 .. 2615232 (the sky region, 422,656 bytes). Periodicity start and end located by sliding an 8 KB window in 256-byte steps through the region and measuring the 256-stride match rate.

Method: autocorrelation as above, then a periodicity scan. A random control of the same size reads flat at 0.0000 for every stride tested.

Result:

• Peak stride in the first 64 KB is 256 at a 0.772 match rate. Strides 255, 257 and 4 score nearly as high, which is what a true period looks like smeared by sampling.
• Across levels the peak is 256 on 21, 512 on 5 and 128 on 1. 512 and 128 are a harmonic and a sub-multiple, so the fundamental period is 256 bytes.

Correction to the control. The original comparison used uniform random bytes, which read flat at 0.0000 -- and made the 0.772 look like an enormous signal. It is not: 91.8% of the region's bytes are the same value (0x07), so a stride-256 comparison matches trivially wherever both positions are 0x07.

The right control is the same bytes shuffled, which preserves the value distribution and destroys only the ordering. That control reads 0.512 at stride 256, not 0.0000. The true excess is 0.772 − 0.512 = +0.26, a real signal and a far smaller one than the original framing suggested.

And the extent is wrong too. Re-measured against the matched control, the excess over the value distribution is:

offset	excess
+0 .. +49152	+0.38 .. +0.45
+53248	+0.26
+57344	+0.07
+61440	−0.13

So the 256-periodic data ends between +53248 and +57344 — somewhere in rows 205 to 224 — not at +60160 as first reported. The original figure came from a raw threshold with no distribution control, and 91.8% of one byte value drags that threshold wherever the dominant run ends.

The 256-byte period itself stands, cross-checked on 21 of 27 levels with the remaining levels showing its harmonic and sub-multiple. The image dimensions do not: an earlier "256x236" is wrong and should not be used.

So the sky region is not one thing. Its first ~60 KB is an image; the rest is whatever the layer offsets address. Both halves still need work, and the image is the tractable one.

Re-verified: the 368 stride, against a matched control

The sky autocorrelation was found to be inflated by a control that did not match the data, so the 368 result was re-checked the same way.

Range measured: each level's post-shrub region start, first 65,536 bytes.

Method: as before, but the control is the same bytes shuffled rather than uniform random bytes. Shuffling preserves the value distribution and destroys only the ordering, which is exactly what the control needs to do.

Result:

• Level 0's region is evenly distributed: 256 distinct byte values, most common only 15.6%. Uniform random was therefore the appropriate control here, and the shuffled control independently confirms it, reading 0.0000 at strides 340, 368 and 4.
• Level 0 peak remains 368 at 0.1110, against 0.0000 for the matched control.
• Recomputed across all levels with matched controls: 24 of 24 peak at exactly 368, every one with excess above 0.05.

So the 368-byte period stands, now with a control appropriate to its data. The contrast with sky is the point: the same statistic on the same kind of question gave a real answer in one region and an inflated one in the other, and the only thing that told them apart was whether the control matched the byte distribution.

Sky sub-objects: structure is consistent, stride is not

Range measured: every level's sky region, from the count field at +0x0c through the sub-object offsets at +0x20. Each sub-object measured over its first 16,384 bytes.

Method: as above, peak-stride autocorrelation per sub-object with a matched control (the same bytes shuffled, seeded per level and index).

Field note: the sub-object count is a u32 at +0x0c, not +0x0a. Reading +0x0a gives 327,680 on level 0, which is not a count.

Result, level 0: five sub-objects at offsets 365760, 388880, 396432, 406784 and 414032, of lengths 23120, 7552, 10352, 7248 and 1536. Each has internal structure well clear of its matched control.

But the strides do not repeat across levels. Recomputing per level:

sub-object	peak strides seen across levels
0	4 (x6), 8 (x13), 32, 288
1	16 (x3), 32 (x6), 288
2	8, 16 (x5), 256, 288
3	8, 16 (x6), 216
4	8, 16 (x3), 512 (x4)

So the earlier per-index stride table, read off level 0 alone, was over-fitted to one level. What is consistent is weaker and more honest: every sub-object sits above its matched control, and most peak at a small stride of 4, 8, 16 or 32, which suggests 16- or 32-bit records whose stride varies with content rather than being fixed.

Level 2's header remains genuinely different: its count field at +0x0c reads 196,617 and is unusable, while every other level yields a plausible count. Whatever makes it different is still unexplained.

Sky sub-object 0: sparse, with duplicated entries

Range measured: level 0, sky region +365760 .. +388880, 23,120 bytes, read as 2,890 records of 8.

Method: record composition, plus the adjacent-pair and zero-density statistics below. The matched-control autocorrelation puts the peak at stride 8 (0.217 real, 0.000 shuffled), so 8 is the unit to describe.

What it is made of:

• 1,185 distinct u32 values across 5,780 words.
• Mean 6.99 of 8 bytes are live -- the previous revision of this file said "zero", which is inverted and wrong. The live-byte histogram is 0:15, 2:1, 4:31, 5:238, 6:461, 7:1030, 8:1114, so most records are nearly full rather than sparse.
• 549 of 2,890 adjacent pairs (19%) are exact duplicates of each other. The most common 8-byte record, appearing 283 times, is d7d7d780 d7d7d780 -- a 4-byte value stored twice. Others follow the same shape: 00051980 00051980, 00082580 00082580.

Byte 7 is anomalous. It takes only 14 distinct values where bytes 0-6 take 142-204. Its common values are 0x00 (1378), 0xff (806) and 0x80 (592) -- the high bytes of signed 16-bit values, which is consistent with byte 7 being an MSB of something rather than ordinary data. Unexplained.

Not decoded. A signed-16-bit reading was tried and does not obviously help: the values span -32743..32633, close to the full range, with no per-position structure (533, 371, 401 and 187 distinct at positions 0-3). Recorded as a characterisation, with its range, so the next attempt starts from the composition rather than re-deriving it.

Sky sub-object 0 is a slowly-varying sequence, not independent records

Range measured: level 0, sky region +365760 .. +388880, 23,120 bytes.

Method: Pearson correlation between the byte sequence and itself at increasing lag, against a shuffled control. This measures whether consecutive records are related, which the position-wise distinct-value counts could not show.

Result:

lag (bytes)	correlation
1	+0.050
4	+0.560
8	+0.746
16	+0.666
32	+0.602
64	+0.432
128	+0.140
256	−0.044
512	+0.304
1024	+0.247
shuffled control (lag 1)	+0.003

The profile has a peak at lag 8, decays to a zero crossing near 256, and then a second, smaller peak at 512 before decaying again.

This is what settles the earlier "no constant substructure" reading. The records do not share fixed fields, but they are not independent: consecutive 8-byte records are strongly related, and that relation repeats at 512 bytes, i.e. every 64 records.

Slowly varying values, related within a record, repeating every 64 records, is the shape of a keyframe or lookup table -- which is what one would expect to find in sky data, where a colour or fog parameter is sampled rather than computed per vertex.

Still not decoded. Nothing here names the values or the sampling. But the structure is now characterised well enough that a decode attempt has a shape to aim at rather than a field to hunt for.

Sky sub-objects share one shape, across every level

Range measured: sub-objects of the sky region on levels 0, 1, 3, 7, 13 and 26 -- 25 sub-objects in total, each measured over its whole length.

Method: as above, correlation at lag against the byte sequence.

Result:

quantity	range over 25 sub-objects
lag 8	+0.53 .. +0.79
lag 64	+0.16 .. +0.63
lag 512	-0.13 .. +0.42 (not universal)
lag 1024	-0.05 .. +0.37 (not universal)

Lag 8 and lag 64 are positive on every sub-object of every level tested. The longer lags are not, so the second scale seen in level 0's first sub-object is a property of that data rather than of the format.

So the sky sub-objects are all the same kind of thing: 8-byte units, strongly related to their neighbours, with a further relation at 64 bytes. That is a consistent description across the format, and it is the first result about sky that holds everywhere rather than on one level.

Not decoded. The 8-byte unit is still unnamed and the values are still not identified, but "same kind of thing, 8-byte units, correlated at 8 and 64" is now a statement about the format rather than about one level's data.

Sound bank SBlk has a 40-byte period (27/27 levels)

Range measured: each level's SBlk region -- the descriptor table between sblk_offset and daw_offset, from the sound bank header rather than from core_data. 17-20 KB per level.

Method: peak-stride autocorrelation with a matched control (the same bytes shuffled, same seed, every level).

Result: the peak is at exactly 40 bytes on all 27 levels, every one with excess above 0.2. The peak is also sharply isolated on level 0 -- 0.454 excess at stride 40 against 0.207 to 0.278 for strides 36 to 44 -- so it is a period rather than a broad tendency.

That is the first structural result about the sound bank beyond the container framing already committed.

Record count: 487, 443, 428, 494 and 447 whole 40-byte records on levels 0, 1, 2, 13 and 26. The region is not an exact multiple of 40 on any level (19484 leaves 4 bytes over), so 40 is a period rather than a record size with no remainder.

Unexplained: which byte positions are constant varies by level -- level 0 has 7, 11, 31 and 35 constant; level 1 has 15, 19, 23, 35 and 39; level 2 has 7, 23, 27 and 31. A fixed 40-byte layout would have the same constant positions everywhere, so either the records have a variable tail or 40 is a period in something else that the records sit inside.

Also measured, for the record:

• DAW, the sample region, shows no stride above threshold against its control. Consistent with ADPCM nibble-packed audio, and consistent with the container framing being the only part worth decoding.
• gameplay.bin peaks at 16 bytes (excess 0.267 at strides 16, 32 and 64).
• gs_ram.bin has almost no stride structure (0.043 at 32) but a lag-8 correlation of +0.503, so it is smooth rather than tabular.

Tie class spans: 16-byte unit, 48-byte period (dominant, not universal)

Range measured: every tie class span on levels 0, 3, 6 and every third level after it -- 386 spans of 16 KB or more, bounded by consecutive class offsets.

Method: peak-stride autocorrelation with a matched control, over the first 32,768 bytes of each span.

Result:

peak stride	spans
48	285
16	58
64	29
96	11
24	2
32	1

48 bytes dominates but is not universal, and the weaker peak is 58 spans against 285 -- so unlike the sound bank's 40, this is a majority rather than a constant.

Every observed peak is a multiple of 16 (16, 24, 32, 48, 64, 96), which suggests a 16-byte base unit with a 48-byte period arising from how those units group, rather than 48 being the record itself.

This is weaker evidence than the sound bank's 40-byte result, which was sharply isolated and identical on all 27 levels. Here the peak is weaker (0.09-0.22 excess against sound bank's 0.45) and it moves between levels and spans. Recorded at the confidence it deserves.

Two independent measurements agree on moby's 16-byte vertex

The moby part table established vertex_offset + 16 * vertex_count == next_part on 43,467 of 43,467 parts, which fixes the vertex stride at 16. Autocorrelation over the spans reaches the same number without using that result at all.

Range measured: moby class spans of 16 KB or more on level 0 -- 60 spans, each bounded by consecutive class offsets, measured over the first 32,768 bytes.

Method: peak-stride autocorrelation against a matched control, searching strides 4 to 64.

Result:

peak stride	spans
16	43
8	8
48	4
32	3
24	2

16 dominates, and it is the same 16 the part table produces from a completely different argument -- a tiling identity over part records rather than a periodicity measurement over span contents.

Two unrelated methods agreeing is worth more than either alone, because the autocorrelation would be vulnerable to exactly the kinds of artifact this project has already been bitten by: a dominant byte value would inflate it, and a period that happened to divide the window would look the same as a real one. Neither applies here, and the tiling identity cannot be inflated at all.

This is the strongest form of evidence produced in this project: a structural invariant confirmed by two methods that do not share a failure mode.

Sound bank SBlk: a 12-byte marker that does not explain the 40-byte period

Range measured: level 0 SBlk, 24..19508 (19,484 bytes).

Method: positions of the 0x7f/0x47f marker words, and their spacing.

Result: those markers sit at 12-byte spacing on 144 of roughly 180 occurrences, with 24-byte and larger gaps accounting for the rest. So there is a 12-byte structure in the data.

But that contradicts the autocorrelation. If records were 12 bytes, stride 12 should be the peak. It is not:

stride	excess over shuffled
12	+0.237
20	+0.175
24	+0.237
40	+0.454
48	+0.239

40 is not a multiple of 12, so the two measurements disagree rather than one explaining the other. Recorded as an open discrepancy rather than resolved in favour of either: the marker spacing says 12, the period says 40, and 40 being twice as strong is the more surprising of the two.

Resolved: 40 is the record, the 12-byte markers are nested

A fine scan settles it in favour of 40.

stride	excess
40	+0.454
39	+0.266
41	+0.215
38	+0.214

The peak is sharp and isolated -- nearly twice its immediate neighbours. And the multiples confirm it: 80 scores +0.376, 120 +0.394, 160 +0.340, while the multiples of 12 (24, 48, 96, 144) all sit at 0.21-0.24.

So the record is 40 bytes, and the 0x7f/0x47f markers at 12-byte spacing are a nested structure inside those records rather than the record boundary. Their spacing is irregular -- 12, 24 and larger -- which is exactly what one would expect of a field that is present in some records and absent in others.

The earlier "they disagree" framing was wrong in one respect: the marker spacing never claimed to be the record size. Read as a claim about markers inside records rather than about the record itself, the two measurements were never in conflict.

SBlk record layout at 40 bytes

Range measured: level 0 SBlk, 24..19508, aligned at +0 — 487 whole records.

Method: for each of 40 candidate alignments, record the number of byte positions that are near-constant across records, and keep the best.

Result: alignment +0 is best, with 10 near-constant positions. Four are exactly constant and all zero: positions 7, 11, 31 and 35.

Distinct values per byte position across the 487 records runs from 1 to 155, with position 29 the most variable by a wide margin (155 against 21-48 elsewhere) and positions 2, 10, 27, 34 and 39 low (2 to 8).

The first record carries the SBlk magic and the DAW tag rather than descriptor data, so it is a header rather than an entry — which is worth accounting for before reading any field at a fixed offset.

Not decoded. The layout is established and the constant positions are known; the fields are not named, and which of the 40 bytes are an index, a length and an offset is still open.

SBlk: the field values do not read as record fields

Range measured: level 0 SBlk, 24..19508.

Method: read at 4-byte alignment and rank the most common u32 values.

Result: the common values are 0x0 (2261), 0x1 (369), 0x7f (168), 0x5a000000 (125), 0x1eff7f7f (125), 0x1dfcc (124), 0x19fc0 (93), 0x7c34240d (91), 0x7c3821c8 (91).

This does not look like a table of records with typed fields. A 40-byte record layout whose fields are an index, a length and an offset would not produce 0x1eff7f7f and 0x7c34240d a hundred times each, nor would those values survive a byte shift.

So either the 40-byte period is not a record boundary, or the region is an encoded stream -- something like varint or Huffman-coded data -- that happens to have a 40-byte statistical period.

The 0x7f values are suggestive: in a varint encoding a byte of 0x7f is the last value representable in seven bits, so it appears as a frequent saturated byte. That is consistent with a coded stream and inconsistent with a fixed-layout record.

Left open. The 40-byte period is solid (27/27 levels, sharply isolated, its multiples confirm it), and the alignment with four constant zero positions is solid. What sits in those bytes is not a record layout, and the two findings are in tension in a way this project has not otherwise seen: a confirmed period that is not a record boundary.

Correction: the 368 stride is real, but confined to one ~15 KB window

The claim earlier in this file that the post-shrub region has a 368-byte period throughout is wrong. Re-measured over the whole region, the autocorrelation curve is flat and near noise:

stride	excess over matched shuffled control
8	+0.027
12	+0.023
148	+0.009
340	+0.018
368	+0.018
632	+0.020

Nothing peaks at 368. The figure of 0.1110 recorded earlier does not reproduce under any sampling convention tried (step 1, 2 or 4 through the window; all give ~0.018).

Where the structure actually is. Scanning 8 KB windows at 1 KB steps for a 368-excess above 0.15 finds exactly one substantial region:

region+41984 .. region+57344   = 15,360 bytes   (abs 16089024..16104384)

plus a 4 KB patch at region+650240. Everything between and around is flat.

Within that window the 368 stride is unambiguous:

stride	excess
368	+0.315
736	+0.284
364	+0.024
372	+0.021
732	+0.027

The peak is sharp, 736 confirms it as a multiple, and the neighbours are at noise.

Record shape. 41 whole 368-byte records. Mean distinct values per position is 16 of 41, and nine positions are exactly constant and zero: 329 through 336, and 340. Twenty-five positions have 3 or fewer distinct values, clustered in 325-349 -- a tail region rather than scattered padding.

So the corrected statement is: a 15,360-byte window of the post-shrub region is a table of 41 records at stride 368, each with a roughly 43-byte tail whose fields are mostly zero. The surrounding 675 KB is not that table.

What went wrong before. The original measurement was taken over a window where 368 held and then reported as a property of the region. The 15 KB window holds 41/1841 of the region's records, and averaging that over the whole region flattens the peak to the noise floor. The re-verification pass checked the stride but not whether it applied uniformly, so it confirmed the number without testing the claim it was attached to.

The post-shrub region is a chunk container (explains the 368 stride)

Range measured: level 0 post-shrub, 16047040..16737344 (690,304 bytes).

Method: scan for the block shape 16-byte header ending 0xff followed by 10 zero bytes followed by a run of u32 whose top nibble is 0xF.

Result: 28 chunk headers on level 0, present on all 27 levels. Each chunk is:

+0x00  16 bytes, last byte 0xff     (opaque; differs per chunk)
+0x10  10 zero bytes
+0x1a  N tagged offsets, top nibble 0xF

and within a chunk the offsets step by a chunk-specific stride — 148, 340, 368, 288, 320, 236, 272, 384, 116, 160, 250, 580, 576 all occur on level 0. The stride is near-constant within a chunk (e.g. 148,148,148,148... then 340,340,336,340...) but differs between chunks.

This resolves two earlier results.

The "368-byte period" is one chunk's stride, not a property of the region. That is why the region-wide autocorrelation is flat: the region holds chunks with ~15 different strides averaging out to noise. The 15,360-byte window where 368 scored +0.315 is the third chunk, at region+30066, whose stride is 368.

The "ten-entry table" I earlier disconfirmed is real — chunks carry 5 to 30 tagged offsets, with 10 the most common on level 0. It was disconfirmed because I tested for a fixed stride of 148 across a whole region; the table exists, but 148 is one chunk's stride among many.

Two corrections this produces. Both earlier claims were half-right: real structure, wrong scope. I reported a local measurement as a regional one in both cases, which is the same mistake twice.

Not decoded. What the 16-byte header is, what the 0xF nibble tags, where the chunk data lives, and what the chunk means are all open. The container framing is established; its contents are not.

Chunk records resolve, and the layout is consistent across levels

Range measured: each level's post-shrub region, taken as the 900,000 bytes before the constant trailer.

Method: for each chunk, read the tagged offsets as 0xF0000000 | low24, take the stride as the first delta, load one record per offset at chunk_start + low, and count byte positions that are constant across that chunk's records.

Result: 1,096 chunks and roughly 13,000 records resolve on all 27 levels. Every level's chunks show 17-21% of byte positions constant across their own records — a consistent field layout rather than noise.

Level 0 in detail: 27 chunks resolve, 466 records, 20.8% constant. The stride-368 chunk (77 records) has 26 constant positions out of 368; the stride-340 chunk (77 records) has 25 out of 340. Constant positions cluster near the record end — 267-276 and following for stride 368, 258-268 for stride 340 — which is a footer or tail, not scattered padding.

The tag is uniform. Every tagged offset seen has top byte 0xF0, i.e. 0xF0000000 | low24, on all levels.

What this settles. The container framing from the previous entry is not a coincidence of byte patterns: the offsets genuinely address records, and those records share a field layout. The chunk-relative base is confirmed because resolving at chunk_start + low yields records with ~20% constant positions, which does not happen by chance.

Still not decoded. The 16-byte header (opaque, per-chunk), the meaning of the 0xF0 tag, which fields are index/length/offset, and whether a chunk is moby_anim or something else. A parser can extract chunks and records bounds-checked today; naming fields needs a domain argument this project has not yet found.

The chunk header: byte 14 is the record count

Range measured: all 27 levels, 1,123 chunks, over the 900,000 bytes before the constant trailer.

Method: compare header byte 14 against the number of offsets actually resolved for that chunk.

Result: byte 14 equals the record count on 1,123 of 1,123 chunks, zero mismatches. This is the first genuinely decoded field in this container rather than a measured correlation.

The parser now truncates the offset table to header byte 14, so a stray tagged word appearing in a chunk's record data cannot silently extend the table -- which would otherwise let the count disagree with its own header.

Two more header bytes are near-constant and look like ASCII:

• byte 13: only two values across 1,123 chunks -- 0x45 (1,076) and 0x46 (47).
• byte 9: three values -- 0x44 (26), 0x45 (1,084), 0x46 (13).

0x44, 0x45 and 0x46 are ASCII D, E and F. That three-letter tag appearing at two different byte positions with different distributions is suggestive of a type or class field, but it is not established: the remaining 14 header bytes are high-entropy and do not read as ASCII at any shift tried, so the ASCII in bytes 9 and 13 could be incidental.

Still open: the 14 opaque header bytes, whether bytes 9 and 13 are type tags, which record fields are index/length/offset, and whether a chunk is moby_anim. The D/E/F values are suggestive enough to be worth testing against a known type table, but that has not been done.

The header is a type ID, and the type determines the stride

Range measured: all 27 levels, 1,123 chunks.

Method: group chunks by their 16-byte header; for each distinct header, collect the set of strides observed across every level it appears on.

Result: only 64 distinct headers across 1,123 chunks. The header is a type identifier, not a per-chunk descriptor -- the same 16 bytes recur on every level.

63 of the 64 headers imply exactly one stride. So header to record layout is very nearly a function: the type determines the record size.

Resolved: the single exception is three types sharing a header.

Header 7542eff271b522a79e45348cca4509ff was reported with three strides, 76, 92 and 104. It is not one type with a variable stride, and not a misparse.

It appears three times per level, always with record count 9, always in the same order and always at the same offsets within the region. On level 0 and on every other level checked:

offset	stride	count	first offset	last offset
863842	92	9	928	1640
867554	76	9	912	1520
872866	104	9	1000	1832

The offsets are 3,712 apart each time and the strides differ, so these are three separate chunks that happen to share 16 header bytes.

Consequence for the type ID. 64 distinct headers for 66 distinct record layouts, not 64 for 64. The header is nearly a type ID but does not uniquely determine the layout: whatever distinguishes these three is not in the 16 bytes the header captures. Candidates not excluded: a distinguishing byte sits outside the 16 captured, or the layout varies within a type.

This does not change the earlier conclusion that 63 of 64 headers imply one stride -- that remains 63 of 64 -- but the mapping is not a bijection and should not be used as one.

The header does not encode the stride. All 90 two-byte windows of header[0..14] were tested: none maps to a unique stride across every group. So there is no simple two-byte field to read the record size from, and the stride must come from the offset table, as the parser already does, by differencing the first two tagged offsets.

That is the right implementation anyway, and this makes it more than a convenience: given three chunks sharing one header with three different strides, any dispatch that assumed "header implies size" would be wrong for 1,123 chunks' worth of type coverage on at least one layout. Reading the stride from the offsets is the only interpretation consistent with all the measurements.

The ASCII reading is confirmed as a type tag. Grouping by header bytes 9 and 13:

tag	chunks	distinct strides
DE	26	2 (184, 344)
EE	1050	46
EF	34	2 (268, 640)
FF	13	1 (520)

The rare tags are tightly coupled to their layouts -- FF is a single stride across all 13 of its chunks -- while EE is heterogeneous with 46. That pattern is what a type tag should look like, and it settles the earlier suggestion that the D/E/F bytes were incidental: they are not.

What this is good for. A chunk can now be dispatched on its header to a record layout, and 63 of 64 headers determine that layout's size without inspecting any record. That is a usable parse entry point, and it is the first point in this container where a field is doing real work rather than merely correlating.

Still open: what the 64 types mean, which record fields are index/length/offset, and whether any type is moby_anim.

Chunk records reference moby class IDs at 38x chance

Range measured: level 0, the 900,000 bytes before the trailer; 28 chunks, 475 records, 67,780 u16 slots. Reference set: the 238 distinct o_class values from level 0's moby class table, read through LevelCoreHeader.

Method: scan every u16 in every record; count how many are in the moby class set. Compare against the rate expected if hits were chance, at 238/65536 per slot.

Result: 9,493 hits against 246.1 expected -- 38.6x excess.

The hits are not spread. They concentrate in a contiguous run of u16 slots:

slot	hits		slot	hits
138	188		134	173
135	182		136	168
137	180		129	139
			130	131

Slots 129 through 140 -- twelve consecutive u16, that is bytes 258 through 280 of the record -- carry 105 to 188 hits each, and nothing else in the record comes close except a separate cluster at slot 38.

What this looks like. A contiguous twelve-value array in which most entries are moby class identifiers is what a model or instance definition listing the mobies it references would look like. Every chunk contributes in proportion to its record count: the stride-340 chunk with 77 records has 1,451 hits, the stride-368 chunk 1,384, the stride-148 chunk with 10 records has 254.

What this does not show. Enrichment is not identification. A 38x excess over chance is strong evidence that these values are moby class IDs rather than incidental small integers, but it does not establish that the containing chunk is moby_anim -- an instance table, a model list and an animation table would all reference class IDs. The format is still unnamed, and so is the meaning of slots 129-140.

Correction to the above: the slot offsets were a pooling artifact.

The entry above reported slots 129 through 140. Repeating the scan per chunk type shows that is wrong: every chunk type places its class array at a different offset.

type (header prefix)	top hit slots
aac4	129-134
82c4	104-109
4ec4	106-111
b8c4	120-125
9943	82-87
79c4	111-115
3245	24-29

Slots 129-140 were simply the type with the largest stride, dominating a scan that pooled all types together. The previous entry flagged exactly this as unverified, and the check confirmed the flag.

What survives the correction. In every one of the 26 types the top hits form a contiguous run of consecutive u16 slots. That is the real invariant, and it is per-type: each chunk type has its own class array at its own offset, of its own length.

So the corrected statement is: chunk records contain a contiguous array of moby class IDs, and where that array sits depends on the chunk's type header. This is consistent with each type being a distinct record layout, which is what the header-implies-one-stride result already indicated -- but the class array is a per-type field, not a common one at a fixed offset.

The moby reference itself is unaffected: 38.6x excess measured across all records stands, since it does not depend on which slots were counted.

Shrub body records on level 0: six varying bytes, two constant

Range measured: level 0, shrub class table 7 entries at core index offset 10176. Six classes are bounded by a successor; the seventh runs to the end of core_data and is excluded, because its span contains the whole post-shrub region and is not wholly records.

Method: bound each span by the next class's geometry offset, take the body from data_offset to the span end, and split into 8-byte records.

Result: 164 records across 6 bounded bodies, body lengths 192 (twice), 208 (twice), 224, 240.

Per-byte, over those 164 records:

byte	distinct	zero
0	74	60
1	58	65
2	76	43
3	62	48
4	72	35
5	59	38
6	1	164
7	1	164

Bytes 6 and 7 are zero in every record. Bytes 0 to 5 take 58 to 76 distinct values of a possible 164, with only 21-39% zero -- that is, uniformly varied rather than sparse.

Read as u32 the fields are not plausible: u32[0] ranges 0 to 4,294,871,316 and no field is monotone across the body. So the six bytes are not two packed 32-bit values.

What this settles. The six-varying-plus-two-constant shape holds on level 0 as a per-level partition rather than only in aggregate across the disc's 8,938 records, which is what rule 3a asks for. Two trailing zero bytes are padding or a flag pair, not data.

What it does not. Resolved by a follow-up measurement: the six bytes are not an f32 pair either, and not narrow u16s.

• f32 @0: 158 of 164 finite, but the range runs to about 1e35. Not positions or normals. The 6 non-finite values (3.7%) are far above the 0.05% that random bytes would produce, so the field is structured but is not an f32.
• f32 @4: all 164 finite, range 0 to 9.18e-5. This looks promising until you notice bytes 6 and 7 are always zero, which pins this value to a coarse grid and explains the tiny range. It is not independent evidence.
• u16 @0, @2, @4: distinct counts 85, 90 and 80, each spanning essentially the whole 0..65534 range. These are not small counts or indices.

So excluded are: two packed u32, an f32 pair, and narrow u16 fields. The six bytes remain uninterpreted.

Process note. The first attempt at this used ClassEntry and a hand-computed o_class offset instead of ShrubClassEntry, and every ShrubHeader::region call failed. The parser was not wrong; the caller was. The working call shape was already in the disc test.