Pin the extracted tree, which every disc test reads and nothing verified

data/gc-files.tsv pins the ISO files, and unpacks_and_matches_the_committed_table
checks that unpacking reproduces them. But the tie, collision, tfrag and chunk
tests do not unpack -- they read ~/Documents/gc-extracted, a directory on disk
that nothing in the repository checks.

A stale or hand-edited file there would have every measurement built on it still
pass, quietly. Three claims this session returned confident numbers while
describing the parser rather than the disc -- has_uniform_record_length, "1,509
chunks uniform", the moby class-ID scan -- and a test whose data source is
unverified is that same hazard one level down.

data/gc-extracted.tsv now records SHA-1 and size for all 5,237 files in that tree,
324 KiB, sizes and hashes only, no content. It is the output of `gc-extract
unpack`, whose ISO files are already pinned by the other table, so it is grounded
transitively rather than by assertion.

the_extracted_tree_matches_the_committed_table walks the table and checks every
row's size and hash against disk. It asserts all 5,237 are checked -- so a
truncated table fails rather than passing on a prefix -- and reports the first few
missing or mismatched paths when it does. Current result: 5,237 of 5,237 match.

This does not retroactively validate the three retracted claims. What it does is
remove the possibility that any of the surviving ones rests on a directory that
could have been anything.

246 tests, clippy clean.
This commit is contained in:
2026-10-13 08:20:37 +02:00
parent 8ccc3ddda4
commit f1f0b517a5
2 changed files with 5326 additions and 0 deletions
File diff suppressed because it is too large Load Diff
+84
View File
@@ -4726,3 +4726,87 @@ fn chunk_records_neither_overlap_nor_leave_gaps() {
assert_eq!(chunks, 1509, "chunk total moved");
assert_eq!(ragged, 1252, "ragged chunk count moved");
}
/// The extracted tree that every other disc test reads is pinned by hash.
///
/// `data/gc-files.tsv` pins the ISO files, and `unpacks_and_matches_the_committed_table`
/// checks that unpacking reproduces them. But the tie, collision, tfrag and chunk
/// tests do not unpack: they read `~/Documents/gc-extracted`, a directory nothing
/// verifies. A stale or hand-edited file there would have every measurement built
/// on it still pass, quietly.
///
/// That is the failure this project has already produced three times in another
/// form -- `has_uniform_record_length`, "1,509 chunks uniform" and the moby
/// class-ID scan each returned a confident number about the parser while
/// describing it as a fact about the disc. A test whose data source is unverified
/// is the same hazard one level down.
#[test]
fn the_extracted_tree_matches_the_committed_table() {
let data = std::env::var_os("GC_EXTRACTED")
.map(PathBuf::from)
.unwrap_or_else(|| {
let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
PathBuf::from(home).join("Documents/gc-extracted")
});
if !data.exists() {
eprintln!("skipped: unpack the disc first");
return;
}
let table_path = Path::new(env!("CARGO_MANIFEST_DIR")).join("data/gc-extracted.tsv");
let text = std::fs::read_to_string(&table_path).expect("the extracted-tree table is committed");
let mut checked = 0usize;
let mut missing = Vec::new();
let mut bad = Vec::new();
for line in text
.lines()
.filter(|l| !l.starts_with('#') && !l.trim().is_empty())
{
let mut f = line.split('\t');
let (Some(hash), Some(size), Some(rel)) = (f.next(), f.next(), f.next()) else {
continue;
};
let path = data.join(rel);
let Ok(bytes) = std::fs::read(&path) else {
missing.push(rel.to_string());
continue;
};
if bytes.len() as u64 != size.parse::<u64>().unwrap_or(u64::MAX) {
bad.push(format!("{rel}: size {} != {size}", bytes.len()));
continue;
}
let got = {
use sha1::Digest;
let mut h = sha1::Sha1::new();
h.update(&bytes);
h.finalize()
.iter()
.map(|b| format!("{b:02x}"))
.collect::<String>()
};
if got != hash {
bad.push(format!("{rel}: {got} != {hash}"));
continue;
}
checked += 1;
}
assert!(
checked > 5000,
"only {checked} of the extracted tree was checked -- is the table truncated?"
);
assert!(
missing.is_empty(),
"{} files missing:\n{}",
missing.len(),
missing[..missing.len().min(8)].join("\n")
);
assert!(
bad.is_empty(),
"{} files do not match:\n{}",
bad.len(),
bad[..bad.len().min(8)].join("\n")
);
eprintln!("extracted tree: {checked} files match data/gc-extracted.tsv");
}