The control fails my own claim: 0x1c/0x34 select nothing distinctive
2ff2abbended by implying that the class-header offsets point at identifiable content. I said the next step would be a control. Here it is, and it fails the claim. Adjacent-equal-byte rate over 64 bytes, 1,864 classes: 0x1c 0.12688 0x34 0.12106 control 0.14109 (the same offset + 0x2000) random 0.00391 All three are ~30x above the random baseline, so the region is structured data and not noise. That part stands. But 0x1c and 0x34 are no more structured than an arbitrary offset in the same blob, and 0x34 is slightly *less*. So the hex fragments in2ff2abbwere three hand-picked examples from a structured region. "They look like compiled shader programs with small-integer parameters" said something about this part of core_data, not about what those fields address. That is exactly the rule 4 failure -- a pattern that reads as meaningful with no control to test it against -- and I wrote it anyway while explicitly declining to name a format two paragraphs earlier. RETRACTED: the suggestion that the offset targets are identifiable content. Unaffected, because it was measured differently: the offsets are base-relative, non-zero, in range on 1,864 of 1,864 classes, and always land past the class's own tie packet blocks. What they address is still unknown. The earlier finding that the packet's control region is not a tag stream also stands -- that was measured by uniformity within the packet, not by comparison to elsewhere in core_data. A TEST THAT CAUGHT ITS OWN AUTHOR The first version asserted |0x1c - control| < 0.02 symmetrically, and failed on 0x34 at a gap of 0.0200. The correct response was not to loosen the number until it went green. The claim being guarded is "these offsets select distinctive content", and a gap in the *other* direction does not support it -- both offsets came out below control. So the assertion is one-sided, and the comment says why, including that the symmetric form was the wrong shape. That is the AGENTS 3b discipline working in the direction it is meant to: a failing assertion is information about the assertion. 282 passing with the disc present (23 + 19 + 62 + 156 + 22; disc binary 35.16s), clippy clean.
This commit is contained in:
@@ -6621,3 +6621,94 @@ fn tie_class_tail_offsets_point_past_the_tie_packets() {
|
||||
"{classes} offsets fall inside the tie packets"
|
||||
);
|
||||
}
|
||||
|
||||
/// Control for `tie_class_tail_offsets_point_past_the_tie_packets`.
|
||||
///
|
||||
/// The bytes at 0x1c and 0x34 are structured -- but **not more structured than
|
||||
/// core_data anywhere else**, so the offsets select nothing distinctive.
|
||||
#[test]
|
||||
fn tie_tail_offsets_are_not_more_structured_than_control() {
|
||||
require_disc!();
|
||||
let data = std::env::var_os("GC_EXTRACTED")
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| {
|
||||
let home = std::env::var("HOME").unwrap_or_else(|_| ".".into());
|
||||
PathBuf::from(home).join("Documents/gc-extracted")
|
||||
});
|
||||
if !data.exists() {
|
||||
eprintln!("skipped: unpack the disc first");
|
||||
return;
|
||||
}
|
||||
use gc_formats::level_core::{ClassEntry, LevelCoreHeader};
|
||||
use gc_formats::tie::TieClassHeader;
|
||||
|
||||
fn adjacent_equal(b: &[u8]) -> f64 {
|
||||
let eq = b.windows(2).filter(|w| w[0] == w[1]).count();
|
||||
eq as f64 / (b.len() - 1) as f64
|
||||
}
|
||||
let mut n = 0usize;
|
||||
let mut at_1c = 0.0f64;
|
||||
let mut at_34 = 0.0f64;
|
||||
let mut ctl = 0.0f64;
|
||||
|
||||
for lv in 0..27usize {
|
||||
let (Ok(idx), Ok(core)) = (
|
||||
std::fs::read(data.join(format!("level{lv}/core_index.bin"))),
|
||||
std::fs::read(data.join(format!("level{lv}/core_data.bin"))),
|
||||
) else {
|
||||
continue;
|
||||
};
|
||||
let t = LevelCoreHeader::parse(&idx).expect("hdr").tie_classes;
|
||||
for c in 0..t.count as usize {
|
||||
let o = t.offset as usize + c * 0x20;
|
||||
let Ok(e) = ClassEntry::parse(&idx[o..o + 0x20]) else {
|
||||
continue;
|
||||
};
|
||||
let base = e.offset_in_asset_wad as usize;
|
||||
if base == 0 || base + 0x48 > core.len() {
|
||||
continue;
|
||||
}
|
||||
if TieClassHeader::parse(&core[base..base + 0x48]).is_err() {
|
||||
continue;
|
||||
}
|
||||
let word = |k: usize| -> usize {
|
||||
let at = base + 0x1c + k * 4;
|
||||
u32::from_le_bytes(core[at..at + 4].try_into().unwrap()) as usize
|
||||
};
|
||||
let (o1, o3) = (word(0), word(6));
|
||||
if o1 + 64 > core.len() || o3 + 64 > core.len() || core.len() < 64 {
|
||||
continue;
|
||||
}
|
||||
let ctl_off = o1.wrapping_add(0x2000) % (core.len() - 64);
|
||||
n += 1;
|
||||
at_1c += adjacent_equal(&core[o1..o1 + 64]);
|
||||
at_34 += adjacent_equal(&core[o3..o3 + 64]);
|
||||
ctl += adjacent_equal(&core[ctl_off..ctl_off + 64]);
|
||||
}
|
||||
}
|
||||
|
||||
assert!(n > 1_800, "only {n} classes read");
|
||||
let (m1, m3, mc) = (at_1c / n as f64, at_34 / n as f64, ctl / n as f64);
|
||||
eprintln!(
|
||||
"adjacent-equal rate: 0x1c {m1:.5} | 0x34 {m3:.5} | control {mc:.5} | random {:.5}",
|
||||
1.0 / 256.0
|
||||
);
|
||||
|
||||
// Far from random: this region is structured data, not noise.
|
||||
assert!(mc > 20.0 / 256.0, "control fell to the random baseline");
|
||||
// But the two offsets are NOT *more* structured than an arbitrary one.
|
||||
// One-sided on purpose: the claim being guarded is "these offsets select
|
||||
// distinctive content", and a gap in the other direction does not support
|
||||
// it. Both come out slightly BELOW control (0x1269 and 0.1211 against
|
||||
// 0.1411), which is why a symmetric tolerance was the wrong shape -- it
|
||||
// failed on a 0.0200 gap while asserting something the data does not say.
|
||||
//
|
||||
// The margin is 0.03, comfortably above the observed 0.014 and 0.020, so
|
||||
// this still fails if either offset ever becomes markedly more structured
|
||||
// than its neighbourhood -- which is the thing worth catching.
|
||||
assert!(
|
||||
m1 < mc + 0.03 && m3 < mc + 0.03,
|
||||
"0x1c {m1:.5} / 0x34 {m3:.5} are now MORE structured than control \
|
||||
{mc:.5}: they may select real content after all"
|
||||
);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user