Decode the chunk count field: header byte 14 is the record count

Range measured: all 27 levels, 1123 chunks, over the 900000 bytes before the
constant trailer.

Method: compare header byte 14 against the number of offsets actually resolved
for that chunk.

Result: byte 14 equals the record count on 1123 of 1123 chunks, zero
mismatches. This is the first genuinely decoded field in this container rather
than a measured correlation.

The parser now truncates the offset table to header byte 14, so a stray
tagged word in a chunk's record data cannot silently extend the table and put
the count at odds with its own header.

Two more header bytes are near-constant and read as ASCII: byte 13 takes only
0x45 (1076 chunks) and 0x46 (47); byte 9 takes 0x44 (26), 0x45 (1084) and
0x46 (13). Those are D, E and F -- a three-letter tag at two byte positions
with different distributions. Recorded as suggestive, not established: the
other 14 header bytes are high-entropy and read as ASCII at no shift tried, so
the D/E/F may be incidental. Worth testing against a known type table.

Still open: the 14 opaque header bytes, whether bytes 9 and 13 are type tags,
which record fields are index/length/offset, and whether a chunk is moby_anim.
This commit is contained in:
2026-10-11 04:26:44 +02:00
parent 681333535a
commit 10188416e2
3 changed files with 55 additions and 0 deletions
+9
View File
@@ -1956,6 +1956,15 @@ fn chunk_containers_resolve_on_every_level() {
chunk.chunk.offset
);
assert!(chunk.chunk.stride > 0, "level{n}: zero stride");
// The one decoded header field.
assert_eq!(
chunk.chunk.count as usize,
chunk.chunk.offsets.len(),
"level{n}: chunk at {} header count {} but {} offsets",
chunk.chunk.offset,
chunk.chunk.count,
chunk.chunk.offsets.len()
);
chunks += 1;
records += chunk.records.len();
strides.insert(chunk.chunk.stride);
+13
View File
@@ -55,6 +55,12 @@ pub struct Chunk {
pub header: [u8; CHUNK_HEADER_BYTES],
/// Offset of this chunk within the region it was parsed from.
pub offset: usize,
/// Record count, from header byte 14.
///
/// Measured: equals `offsets.len()` on **1,123 of 1,123** chunks across
/// all 27 levels, zero mismatches. This is the one header field that is
/// decoded rather than merely measured.
pub count: u8,
/// The low 24 bits of each tagged offset, in file order.
pub offsets: Vec<u32>,
/// Record stride, from the first two offsets. Zero if fewer than two.
@@ -124,6 +130,11 @@ pub fn parse_chunk(region: &[u8], offset: usize) -> Result<Option<Chunk>> {
offsets.push(v & 0x00ff_ffff);
}
// The offset table runs to the first untagged word, but the header carries
// its own count. Truncating to that count stops a stray tagged word in the
// record data from extending the table.
offsets.truncate(header[14] as usize);
let stride = if offsets.len() >= 2 {
(offsets[1] - offsets[0]) as usize
} else {
@@ -131,6 +142,7 @@ pub fn parse_chunk(region: &[u8], offset: usize) -> Result<Option<Chunk>> {
};
Ok(Some(Chunk {
count: header[14],
header,
offset,
offsets,
@@ -163,6 +175,7 @@ pub fn resolve<'r>(chunk: &Chunk, region: &'r [u8]) -> Result<ResolvedChunk<'r>>
}
Ok(ResolvedChunk {
chunk: Chunk {
count: chunk.count,
header: chunk.header,
offset: chunk.offset,
offsets: chunk.offsets.clone(),