Measure every chunk's record offsets; the cause is ambiguous, so no fix

The previous commit fixed on one signature. Measuring all of them shows the
pattern is the norm. Taking delta = offset[i+1] - offset[i] against the chunk's
own first delta, levels 0-2, 2,953 deltas:

      0  1,723      -4    549      +4    426
     -8    146      +8     32
    -12     71     +12     21
    -16     34     +16      4
    -20     27     +20      1
    -24     13

Every deviation is a multiple of 4 and most are exactly +/-4; worst observed is 24
bytes in a 7-record chunk. 1,252 of 1,509 chunks have at least one, across 66 of
74 signatures.

WHY THIS DOES NOT BECOME A FIX

Two models produce this offset table and they are observationally identical:

  1. records are genuinely variable-length, padded to 4 bytes, stride = largest
  2. records are fixed at stride and the offset table carries inter-record
     padding, so the gap to the next record is record + padding

Nothing in the offsets distinguishes them. Both say where records start. Both
produce 92, 88, 84, 92, 92, 92, 84, 88 for the case already discussed.

I wrote the change that slices each record against its successor -- model 1 -- and
then reverted it. Two things came out of writing it:

First, it broke three tests, which is the informative part. Those tests assert
"1,509 of 1,509 chunks have uniform record length" and they passed BECAUSE the
parser forced uniformity by slicing every record at a single stride. They were
asserting the parser's behaviour back at itself, and would have kept passing
forever regardless of what the disc contains.

Second, shipping it would have replaced one unverified model with another and
silently changed the record boundaries of 1,252 chunks on the strength of a
guess. The repo's own rule applies: a heuristic count is not an identification,
and here it is not even a count -- it is two equally good explanations of one
table.

WHAT WOULD SETTLE IT: a field inside the record whose end is independently
observable -- a terminator, a count implying a length, or a known value pinning
its position. Under model 2 the disputed bytes are padding and will be uniform or
zero; under model 1 they are data and will not be. For [75 42 ef] those 8 bytes
read a0 00 27 20 at both the tail of one slice and the head of the next, which is
arithmetic rather than evidence, since it is the same eight bytes seen twice.

The uniform model stays, the ambiguity is recorded in resolve()'s documentation
and here, and the 1,509-of-1,509 assertion stands as a statement about what the
parser does rather than about what the disc contains. 243 tests, clippy clean.
This commit is contained in:
2026-10-13 07:50:50 +02:00
parent b9e9835639
commit 5f0396e3d7
3 changed files with 94 additions and 11 deletions
+13 -1
View File
@@ -484,7 +484,19 @@ Phase 1 hands over the extents; this phase works out what each one contains.
over-read by 4 and 8 bytes and include the next record's head. Records
still *start* correctly and bounds checking holds, so it is a data-correctness
bug, not a safety one. The "strides {76, 92, 104}" description is a
misdescription of three chunk shapes, one of them variable. The body
misdescription of three chunk shapes, one of them variable.
Measured across **all** chunks rather than that one signature, the pattern
is the norm: **1,252 of 1,509 chunks have non-uniform record offsets**,
across 66 of 74 signatures, with every deviation a word-aligned multiple of
4 and most exactly +/-4. **Which model this is remains open** -- genuinely
variable-length records and fixed records with inter-record padding produce
identical offset tables, and the offsets cannot tell them apart. A fix that
slices each record against its successor was written and reverted: it
implements one of the two models and the evidence does not establish it.
It also revealed that three tests asserting "1,509 uniform record length"
were passing only because the parser forced uniformity by slicing at a
single stride. Settling it needs a field whose end is independently
observable. The body
is dense (74% of `f32` reads plausible against a 49% random control) with
an index-like tail -- 12 of 163 `u16` slots in the largest type stay below
`0x1000` -- but **no count field** exists in the first 64 body bytes at any