Show the chunk record's leading byte is a per-chunk phase counter; every chunk
has a uniform record length Follows the alignment result and settles what the leading byte is. Within one chunk the leading byte steps by a constant and wraps: offsets: 1420 1748 2072 2392 2716 3044 3372 3700 4028 4352 ... byte0: 0 16 32 48 64 80 96 112 128 144 ... Sweeping every (step, phase) over all 27 levels: fully explained: 1,262 of 1,509 chunks by step: 8 -> 51 chunks, 16 -> 554, 32 -> 520, 64 -> 137 unexplained: 247 chunks across 11 chunk types So on 1,262 chunks the leading byte carries no per-record information: it is a deterministic function of the record's index and the chunk's phase, which is a position marker rather than data. That is why the earlier "16 discrete values, uniform counts" pattern appeared at all. ResolvedChunk::bucket_rule returns the (step, phase) when the whole chunk fits and None otherwise, so the 247 unexplained chunks stay distinguishable and nothing skips a leading byte on the strength of a rule that does not apply to it. Every chunk has a uniform record length: 1,509 of 1,509. Length varies BETWEEN chunks of a type -- one type runs 76 to 104 bytes -- but never within one. This withdraws the previous commit's "variable-length type", which came from pooling minlen/maxlen across all of a type's records at once. That is the pooled-claim error rule 3a exists to prevent, and it is the third time this session a pooled measurement produced a claim that partitioning reversed. The type is not variable-length; the chunk is fixed-length and different chunks differ.
This commit is contained in:
@@ -2920,6 +2920,25 @@ fn chunk_records_start_16_byte_aligned() {
|
||||
|
||||
assert!(chunks >= 1500, "only {chunks} chunks parsed");
|
||||
assert!(records >= 34000, "only {records} chunk records");
|
||||
// Every chunk has a uniform record length -- length varies between chunks
|
||||
// of a type, never within one -- and on most chunks the leading byte is a
|
||||
// per-chunk phase counter rather than data.
|
||||
let mut uniform = 0usize;
|
||||
let mut ruled = 0usize;
|
||||
for n in 0..27 {
|
||||
let core = std::fs::read(data.join(format!("level{n}/core_data.bin"))).expect("core_data");
|
||||
for c in parse_all_chunks(&core).expect("chunks") {
|
||||
if c.has_uniform_record_length() {
|
||||
uniform += 1;
|
||||
}
|
||||
if c.bucket_rule().is_some() {
|
||||
ruled += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(uniform, 1509, "uniform-record-length count moved off 1509");
|
||||
assert_eq!(ruled, 1262, "bucket-rule count moved off 1262");
|
||||
|
||||
// Not universal. Three chunk types break even the 8-byte rule, all of them
|
||||
// the variable-length or 8-stepped ones. A parallel probe puts 1,142
|
||||
// records across seven types outside a *16*-byte rule, which is how the
|
||||
@@ -2942,8 +2961,9 @@ fn chunk_records_start_16_byte_aligned() {
|
||||
);
|
||||
|
||||
eprintln!(
|
||||
"chunks: {chunks} chunks, {records} records; leading byte a multiple of 8 on {aligned}, \
|
||||
{bad} exceptions across {} types (8-byte step and one variable-length type)",
|
||||
"chunks: {chunks} chunks, {records} records; uniform record length on {uniform}; \
|
||||
leading byte a multiple of 8 on {aligned}, {bad} exceptions across {} types; \
|
||||
leading byte fully explained by a per-chunk phase counter on {ruled}",
|
||||
unaligned.len()
|
||||
);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user