Show the first two chunk-record bytes encode the record's own index
Continuing from the phase counter. Within a chunk: i : 0 1 2 3 4 5 6 7 8 9 10 11 ... 15 16 17 ... byte0: 0 16 32 48 64 80 96 112 128 144 160 176 ... 240 0 16 32 byte1: 0 0 0 0 0 0 0 0 0 0 0 0 ... 0 1 1 1 byte0 is step*(j mod g) and byte1 is (j / g), where g = 256/step and j = index + phase. The pair is the record's index in mixed radix: byte0 carries the low digits scaled by step, byte1 the quotient. Measured over all 27 levels: 1,235 of 1,509 chunks fit this on both bytes. Byte 0 alone fits 1,262, so the 27-chunk difference are chunks whose first byte follows the rule and whose second does not. On 1,235 chunks a record knows its own position and the two leading bytes hold no per-record information. ResolvedChunk::index_rule returns the (step, phase) only when the whole chunk fits on BOTH bytes, so the 274 that do not stay distinguishable and nothing skips a leading byte on a rule that does not apply to it. A number that moved when the implementation changed: a first probe reported 1,181; the committed method reports 1,235. The method is right. The probe accumulated ok0/ok1 flags across candidate phases with an early-exit that could skip a phase which would have matched both bytes, and it also disagreed with bucket_rule on the byte-0 count (1,208 against 1,262), which is what gave it away. Recorded because both numbers came from the same run and only the discrepancy exposed the probe as wrong. The lesson is narrow: when a one-off probe and a committed helper disagree, check the probe first.
This commit is contained in:
@@ -2925,6 +2925,7 @@ fn chunk_records_start_16_byte_aligned() {
|
||||
// per-chunk phase counter rather than data.
|
||||
let mut uniform = 0usize;
|
||||
let mut ruled = 0usize;
|
||||
let mut indexed = 0usize;
|
||||
for n in 0..27 {
|
||||
let core = std::fs::read(data.join(format!("level{n}/core_data.bin"))).expect("core_data");
|
||||
for c in parse_all_chunks(&core).expect("chunks") {
|
||||
@@ -2934,10 +2935,14 @@ fn chunk_records_start_16_byte_aligned() {
|
||||
if c.bucket_rule().is_some() {
|
||||
ruled += 1;
|
||||
}
|
||||
if c.index_rule().is_some() {
|
||||
indexed += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
assert_eq!(uniform, 1509, "uniform-record-length count moved off 1509");
|
||||
assert_eq!(ruled, 1262, "bucket-rule count moved off 1262");
|
||||
assert_eq!(indexed, 1235, "index-rule count moved off 1235");
|
||||
|
||||
// Not universal. Three chunk types break even the 8-byte rule, all of them
|
||||
// the variable-length or 8-stepped ones. A parallel probe puts 1,142
|
||||
@@ -2963,7 +2968,8 @@ fn chunk_records_start_16_byte_aligned() {
|
||||
eprintln!(
|
||||
"chunks: {chunks} chunks, {records} records; uniform record length on {uniform}; \
|
||||
leading byte a multiple of 8 on {aligned}, {bad} exceptions across {} types; \
|
||||
leading byte fully explained by a per-chunk phase counter on {ruled}",
|
||||
leading byte a phase counter on {ruled}; \
|
||||
leading TWO bytes encode the record index on {indexed}",
|
||||
unaligned.len()
|
||||
);
|
||||
}
|
||||
|
||||
@@ -179,6 +179,51 @@ impl<'a> ResolvedChunk<'a> {
|
||||
};
|
||||
it.all(|r| r.len() == first)
|
||||
}
|
||||
|
||||
/// The per-chunk `(step, phase) that makes the first **two** bytes of every
|
||||
/// record encode that record's own index, or `None` when no such pair fits.
|
||||
///
|
||||
/// With `g = 256 / step` and `j = index + phase`:
|
||||
///
|
||||
/// ```text
|
||||
/// record[0] == step * (j mod g)
|
||||
/// record[1] == (j / g) mod 256
|
||||
/// ```
|
||||
///
|
||||
/// so the pair is the record index in mixed radix, and the leading bytes carry
|
||||
/// **no per-record information** -- a record knows where it is.
|
||||
///
|
||||
/// **Measured: 1,235 of 1,509 chunks** across all 27 levels fit this exactly.
|
||||
/// [`Self::bucket_rule`] -- byte 0 alone -- fits 1,262; the 27-chunk difference
|
||||
/// are chunks whose first byte follows the rule but whose second does not.
|
||||
///
|
||||
/// The remaining 274 chunks are not explained by it.
|
||||
/// Nothing should skip a leading byte on the strength of this where it returns
|
||||
/// `None`.
|
||||
pub fn index_rule(&self) -> Option<(u8, usize)> {
|
||||
for step in [8u16, 16, 32, 64] {
|
||||
let g = (256 / step) as usize;
|
||||
for phase in 0..g {
|
||||
let mut ok = true;
|
||||
for (i, r) in self.records.iter().enumerate() {
|
||||
if r.len() < 2 {
|
||||
continue;
|
||||
}
|
||||
let j = i + phase;
|
||||
if r[0] != ((j % g) * step as usize % 256) as u8
|
||||
|| r[1] != ((j / g) % 256) as u8
|
||||
{
|
||||
ok = false;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if ok && self.records.iter().any(|r| r.len() >= 2) {
|
||||
return Some((step as u8, phase));
|
||||
}
|
||||
}
|
||||
}
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
/// True if `offset` looks like the start of a chunk.
|
||||
|
||||
@@ -1352,3 +1352,36 @@ This **withdraws the previous commit's "variable-length type"**. That came from
|
||||
taking `minlen`/`maxlen` over all of a type's records at once, pooling across
|
||||
chunks; rule 3a's pooled claim, third time this session. Within any one chunk the
|
||||
length is fixed.
|
||||
|
||||
## The first TWO bytes encode the record's own index
|
||||
|
||||
Continuing from the phase counter. Within a chunk:
|
||||
|
||||
i : 0 1 2 3 4 5 6 7 8 9 10 11 ... 15 16 17 ...
|
||||
byte0: 0 16 32 48 64 80 96 112 128 144 160 176 ... 240 0 16 32
|
||||
byte1: 0 0 0 0 0 0 0 0 0 0 0 0 ... 0 1 1 1
|
||||
|
||||
`byte0` is `step * (j mod g)` and **`byte1` is `(j / g)`**, where `g = 256/step`
|
||||
and `j = index + phase`. So the pair is the record's index in mixed radix:
|
||||
`byte0` carries the low digits scaled by `step`, `byte1` the quotient.
|
||||
|
||||
**Measured over all 27 levels: 1,235 of 1,509 chunks** fit this on both bytes.
|
||||
Byte 0 alone fits 1,262, so the 27-chunk difference are chunks whose first byte
|
||||
follows the rule and whose second does not.
|
||||
|
||||
**So on 1,235 chunks a record knows its own position**, and the two leading bytes
|
||||
hold no per-record information. `ResolvedChunk::index_rule` returns the
|
||||
`(step, phase)` only when the *whole* chunk fits on both bytes, so the 274 that
|
||||
do not stay distinguishable.
|
||||
|
||||
### A number that moved when the implementation changed
|
||||
|
||||
A first probe reported **1,181**; the committed method reports **1,235**. The
|
||||
method is right. The probe accumulated `ok0`/`ok1` flags across candidate phases
|
||||
with an early-exit that could skip a phase which would have matched both bytes,
|
||||
and it also disagreed with `bucket_rule` on the byte-0 count (1,208 against
|
||||
1,262), which is what gave it away.
|
||||
|
||||
Recorded because the two numbers came from the same run and only the discrepancy
|
||||
exposed the probe as wrong. The lesson is narrow and worth stating: when a
|
||||
one-off probe and a committed helper disagree, check the probe first.
|
||||
|
||||
Reference in New Issue
Block a user