Chunk records: the stride table, and the exception pinned by name

Walked every chunk on all 27 levels, grouped by six-byte header signature:

    1,506 chunks, 34,415 records, 74 distinct signatures
    55 distinct strides, 52..640, every one a multiple of 4
    73 of 74 signatures use exactly one stride

The exception is asserted rather than skipped, because "every type is uniform"
is precisely the claim that quietly stops being true:

    [75 42 ef f2 71 b5]   strides {76, 92, 104}

That is the variable-length type the module docs had referred to indirectly. It
is now pinned by signature and to those three strides exactly.

The 4-byte alignment is a result, not an artefact. A stride recovered from one
chunk's first-to-second record delta can inherit alignment from where that chunk
happened to sit; collecting all 55 strides across every type and requiring all
of them to be divisible by 4 rules that out, because a single off-by-one in one
signature would show. Asserted in
chunk_record_stride_is_four_byte_aligned_and_uniform_per_signature.

What this settles is the shape the decoder must handle: uniform stride per type,
4-byte aligned, one named exception. Field offsets within a record are now the
same for every record of a type, which is what makes per-type decoding
tractable. The fields themselves are still unnamed.

A layout that did NOT generalise, recorded so it is not rediscovered:

signature [53 c4 6c d7 9a 43], stride 108, 810 records, 63 of 108 byte positions
constant -- 48 bytes varying, then 60 bytes that are zero in every record, then a
constant 0x00010000. That reads like an inline buffer this type never uses, and
would have made a tidy field story. Requiring `stride == live span + zero run`
across every type with 50+ records per chunk gives 0 of 9. The zero tail belongs
to that one type and means nothing general.

Third occurrence of one editing mistake, and it is worth naming as a habit
rather than an accident: inserting a test before an existing one strands that
test's #[test] attribute on the new doc comment and silently unregisters it.
It has now happened three times, each time surfacing only as clippy's "never
used" and never as a missing test. Fixed by appending new tests at the end of
the file instead of anchoring them to an existing function, and verified by
listing registered tests rather than trusting a count.

241 tests, clippy clean, all five recent tests confirmed registered.
This commit is contained in:
2026-10-13 07:39:35 +02:00
parent 040386c0fb
commit 59dba3e58d
3 changed files with 140 additions and 1 deletions
+85
View File
@@ -4242,3 +4242,88 @@ fn tfrag_table_pointers_relocate_and_every_patch_is_rewritten() {
morph.0 morph.0
); );
} }
/// Chunk record stride: uniform per signature, always 4-byte aligned, and
/// exactly one signature is not uniform.
///
/// All 55 distinct strides across the 74 signatures are multiples of 4, from 52
/// to 640. Seventy-three of the 74 signatures use a single stride. The
/// exception is asserted rather than skipped, because "every type is uniform"
/// would be the kind of claim that quietly stops being true.
#[test]
fn chunk_record_stride_is_four_byte_aligned_and_uniform_per_signature() {
require_disc!();
let data = std::env::var_os("GC_EXTRACTED")
.map(PathBuf::from)
.unwrap_or_else(|| {
let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
PathBuf::from(home).join("Documents/gc-extracted")
});
if !data.exists() {
eprintln!("skipped: unpack the disc first");
return;
}
use gc_formats::chunks::{chunk_region_start, parse_all_chunks};
use gc_formats::level_core::LevelCoreHeader;
use std::collections::{BTreeMap, BTreeSet};
let mut sig: BTreeMap<Vec<u8>, BTreeSet<usize>> = BTreeMap::new();
for lv in 0..27usize {
let (Ok(idx), Ok(core)) = (
std::fs::read(data.join(format!("level{lv}/core_index.bin"))),
std::fs::read(data.join(format!("level{lv}/core_data.bin"))),
) else {
continue;
};
let _ = LevelCoreHeader::parse(&idx).expect("core header");
let scan = chunk_region_start(&core, 0)
.or_else(|| chunk_region_start(&core, 0x1000))
.unwrap_or(0);
let Ok(found) = parse_all_chunks(&core[scan..]) else {
continue;
};
for rc in found {
sig.entry(rc.chunk.header[..6].to_vec())
.or_default()
.insert(rc.chunk.stride);
}
}
assert_eq!(sig.len(), 74, "chunk signature count moved");
let strides: BTreeSet<usize> = sig.values().flatten().copied().collect();
let misaligned: Vec<usize> = strides.iter().copied().filter(|s| s % 4 != 0).collect();
let variable: Vec<(&Vec<u8>, &BTreeSet<usize>)> =
sig.iter().filter(|(_, v)| v.len() > 1).collect();
eprintln!(
"chunk records: {} signatures, {} distinct strides, range {}..{}; \
non-multiples of 4: {misaligned:?}; variable-length signatures: {}",
sig.len(),
strides.len(),
strides.iter().min().unwrap(),
strides.iter().next_back().unwrap(),
variable.len(),
);
for (k, v) in &variable {
eprintln!(" variable: {:02x?} strides {v:?}", &k[..6]);
}
assert!(
misaligned.is_empty(),
"strides not divisible by 4: {misaligned:?}"
);
assert_eq!(*strides.iter().min().unwrap(), 52);
assert_eq!(*strides.iter().next_back().unwrap(), 640);
assert_eq!(
variable.len(),
1,
"more than one signature has mixed strides"
);
assert_eq!(
variable[0].1.len(),
3,
"the variable signature's stride count moved"
);
}
+48
View File
@@ -2288,3 +2288,51 @@ What would settle it, and none of these are cheap:
through its dispatch records. If an id in the candidate resolved to a class through its dispatch records. If an id in the candidate resolved to a class
whose name is a volume trigger, that would be decisive. No class name whose name is a volume trigger, that would be decisive. No class name
containing "volume" exists on the disc, which is why this is not available. containing "volume" exists on the disc, which is why this is not available.
## Chunk records: what the stride table settles, and one signature it does not
Walked every chunk on all 27 levels and grouped by its six-byte header signature:
1,506 chunks, 34,415 records, 74 distinct signatures
55 distinct strides, from 52 to 640
every one of those 55 strides is a multiple of 4
73 of 74 signatures use exactly ONE stride
**The single exception**, asserted rather than skipped so that "every type is
uniform" cannot quietly stop being true:
signature [75, 42, ef, f2, 71, b5] strides {76, 92, 104}
This is the variable-length type the module documentation already referred to
indirectly. It is now named by its signature and pinned to those three strides
exactly. Everything else is a fixed-stride array, which is the structural fact
that makes per-type field decoding tractable at all: a record's field offsets are
the same for every record of its type.
The 4-byte alignment is a real result rather than an artefact of how the stride
was inferred. A stride recovered from one chunk's first-to-second record delta
could inherit alignment from where that chunk happened to sit; collecting all 55
distinct strides across every type and requiring all of them to be divisible by
4 rules that out, because an off-by-one in one signature would show up.
### A record layout that did not generalise
One type looked like a clean find. Signature `[53, c4, 6c, d7, 9a, 43]`, stride
108, 810 records, 63 of 108 byte positions constant:
+0x00 .. +0x30 48 bytes varying per record
+0x30 .. +0x6c 60 bytes that are ZERO in every record
+0x68 0x00010000, constant
A fixed data block followed by a permanently empty region reads like an inline
buffer that this type never uses, and that would have been a tidy field story.
It does not generalise. Requiring a contiguous always-zero run and checking
`stride == live span + zero run` across every type with 50+ records per chunk
gave **0 of 9**. The zero tail belongs to that one type and means nothing
general. Recorded because "48 bytes then 60 dead bytes" is specific enough to
look like a layout and would otherwise be rediscovered and over-read.
Neither of these decodes any record's *fields*. What is now settled is the
shape the decoder has to handle: uniform stride per type, 4-byte aligned, with
one named exception carrying three strides.
+7 -1
View File
@@ -465,7 +465,13 @@ Phase 1 hands over the extents; this phase works out what each one contains.
fixed within **all 71 types**, values `{1, 2, 4, 6, 8, 10}`. Disproved fixed within **all 71 types**, values `{1, 2, 4, 6, 8, 10}`. Disproved
readings: an entry count (57 distinct fractional "sizes") and a readings: an entry count (57 distinct fractional "sizes") and a
distinct-leading-byte count (0 of 1,506 agree). distinct-leading-byte count (0 of 1,506 agree).
- [ ] The chunk **record fields are not decoded**, and no type is named. The body - [~] The chunk **record fields are not decoded**, and no type is named -- but the
stride table is now measured: **74 signatures, 55 distinct strides, 52..640,
every stride a multiple of 4**, and 73 of 74 signatures use a single stride.
The exception is pinned: `[75 42 ef f2 71 b5]` at strides `{76, 92, 104}`.
Asserted including the exception, so "every type is uniform" cannot rot.
A per-type layout found for one type (48 bytes of data then 60 permanently
zero) does **not** generalise -- 0 of 9 types fit that shape. The body
is dense (74% of `f32` reads plausible against a 49% random control) with is dense (74% of `f32` reads plausible against a 49% random control) with
an index-like tail -- 12 of 163 `u16` slots in the largest type stay below an index-like tail -- 12 of 163 `u16` slots in the largest type stay below
`0x1000` -- but **no count field** exists in the first 64 body bytes at any `0x1000` -- but **no count field** exists in the first 64 body bytes at any