Resolve both tie indirections on all 1,864 classes, and disprove a sample artifact

resolve_tie_class walks the chain, which is two steps and both are easy to get
wrong:

  1. The tie class table lives in core_index.bin, not core_data.bin. On level 0
     the tables chain as moby at 192, 238 x 32 bytes later tie at 7808, 74 x 32
     later shrub at 10176, which closes. Reading the same table out of
     core_data.bin produces plausible garbage rather than an error.
  2. ClassEntry::offset_in_asset_wad addresses core_data.bin, despite the field
     name and despite core_data.wad sitting alongside it.

Verified: all 1864 tie classes resolve, o_class agreeing on every one, and all
5592 LOD block offsets land inside core_data.

Disproved, and this is the part worth reading: the word at +0 of a 16-byte unit
inside an LOD block. On the first twelve units of level 0's class 0 it rises
strictly -- 288, 1520, 2848, 3968, 5088, 6096, 7312, 8528, 9792, 10976, 12016,
13072 -- which reads exactly like a linked list, and the accompanying i16s looked
quantised, two distinct values in i16[0] and three in i16[1].

Across the population it is strictly increasing on 0 of 1864 blocks, and i16[0]
takes 8969 distinct values on level 0 alone rather than two. The sample was
twelve units out of 1,724,334. Both the monotonic reading and the quantisation
were artifacts of that sample.

That is the pooled-claim error AGENTS.md rule 3a exists to prevent, committed
almost verbatim: the first twelve units looked clean, and I was one step from
writing "u32[0] is a linked list of runs" into a format spec. The test now
asserts the disproof, so it cannot creep back.

Not established: the block's record size. 3580 non-empty LOD blocks compute as
divisible by 16, but some packets offsets run backwards, so the "next offset
delimits this block" end is wrong for them and that figure should not be trusted.
The tie layout stays partial: indirections and header done, records not.
This commit is contained in:
2026-10-12 03:47:49 +02:00
parent e5d267a810
commit 60e3f129d8
4 changed files with 236 additions and 6 deletions
+113
View File
@@ -2584,3 +2584,116 @@ fn overlay_header_and_constant_block_are_stable() {
+0x10..0x{OVERLAY_CONSTANT_BLOCK_END:x} identical on all"
);
}
/// The two tie indirections resolve on every class on every level, and the
/// LOD block offsets land inside `core_data`.
///
/// See `gc_formats::tie`. This pins the resolution chain and the header; the
/// block's internal layout is *not* decoded, and one tempting reading of it is
/// disproved below.
#[test]
fn tie_classes_resolve_through_both_indirections() {
require_disc!();
let data = std::env::var_os("GC_EXTRACTED")
.map(PathBuf::from)
.unwrap_or_else(|| {
let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
PathBuf::from(home).join("Documents/gc-extracted")
});
if !data.exists() {
eprintln!("skipped: unpack the disc first");
return;
}
use gc_formats::level_core::LevelCoreHeader;
use gc_formats::tie::resolve_tie_class;
let mut classes = 0usize;
let mut o_class_agree = 0usize;
let mut lod_blocks = 0usize;
let mut lod_blocks_in_range = 0usize;
let mut monotonic_u32_0 = 0usize;
for n in 0..27 {
let idx = std::fs::read(data.join(format!("level{n}/core_index.bin"))).expect("core_index");
let core = std::fs::read(data.join(format!("level{n}/core_data.bin"))).expect("core_data");
let h = LevelCoreHeader::parse(&idx).expect("core header");
let t = h.tie_classes;
if t.count == 0 {
continue;
}
for c in 0..t.count as usize {
let r = resolve_tie_class(&idx, &core, t, c).expect("tie class should resolve");
classes += 1;
if r.header.o_class == r.entry.o_class {
o_class_agree += 1;
}
let base = r.entry.offset_in_asset_wad as usize;
for (i, p) in r.header.packets.iter().enumerate() {
let abs = base.wrapping_add(*p as usize);
if abs <= core.len() {
lod_blocks_in_range += 1;
}
lod_blocks += 1;
if *p >= 0 && abs + 16 <= core.len() {
// Is the word at the block start strictly increasing across
// 16-byte units? It looked like it on the first twelve units
// of one class. It is not a property of the format.
let next = if i + 1 < 3 {
r.header.packets[i + 1]
} else {
*p
};
if next > *p {
let end = base + next as usize;
if end <= core.len() {
let n16 = (end - abs) / 16;
let mut prev = 0i64;
let mut ok = true;
for k in 0..n16 {
let q = abs + k * 16;
let a =
u32::from_le_bytes(core[q..q + 4].try_into().unwrap()) as i64;
if a <= prev {
ok = false;
break;
}
prev = a;
}
if ok && n16 > 0 {
monotonic_u32_0 += 1
}
}
}
}
}
}
}
assert!(classes >= 1800, "only {classes} tie classes resolved");
assert_eq!(
o_class_agree,
classes,
"header o_class disagreed with the class entry on {} of {classes}",
classes - o_class_agree
);
assert_eq!(
lod_blocks_in_range,
lod_blocks,
"{} of {lod_blocks} LOD block offsets fell outside core_data",
lod_blocks - lod_blocks_in_range
);
assert_eq!(
monotonic_u32_0, 0,
"the word at +0 of a 16-byte tie unit is strictly increasing on {monotonic_u32_0} blocks; \
if that ever becomes true the disproved reading below needs revisiting"
);
eprintln!(
"tie: {classes} classes resolved through both indirections, o_class agreeing on all; \
{lod_blocks} LOD block offsets all inside core_data; \
strictly-increasing +0 on 0 of {classes} blocks (disproved)"
);
}
+77 -1
View File
@@ -25,7 +25,8 @@
//! * the LOD distances are ordered `0 < near < mid < far`: 315 of 320. The
//! five exceptions are not yet explained and are worth chasing.
use crate::buf::{Buf, Result};
use crate::buf::{Buf, Error, ErrorKind, Result};
use crate::level_core::ClassEntry;
/// Bytes of a tie class header that RAC2 and RAC1 agree on.
pub const TIE_CLASS_HEADER_MATCHED: usize = 0x48;
@@ -91,6 +92,81 @@ impl TieClassHeader {
}
}
/// A tie class resolved through both of its indirections.
///
/// The chain is two steps and both are easy to get wrong, so it is worth
/// stating plainly:
///
/// 1. The class **table** lives in the level's `core_index.bin`, not in
/// `core_data.bin`. On level 0 the tables chain as moby at `192`,
/// 238 x 32 bytes later tie at `7808`, 74 x 32 later shrub at `10176`,
/// which closes. Reading the same table out of `core_data.bin` yields
/// plausible-looking garbage rather than an error.
/// 2. [`ClassEntry::offset_in_asset_wad`] addresses `core_data.bin`, despite the
/// field's name and despite `core_data.wad` existing alongside it.
///
/// With both handled, all **1,864** tie classes on the disc resolve to a header
/// whose `o_class` matches the class entry's own.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct ResolvedTieClass {
/// The index entry this came from.
pub entry: ClassEntry,
/// Its index within the level's tie table.
pub index: usize,
/// The class header, at `offset_in_asset_wad` in `core_data`.
pub header: TieClassHeader,
}
/// Resolve one tie class. `class_index` counts from zero.
///
/// `core_index` is the level's `core_index.bin`; `core_data` is the
/// decompressed `core_data.bin`. Both are bounds-checked.
pub fn resolve_tie_class(
core_index: &[u8],
core_data: &[u8],
classes: crate::level_core::ArrayRange,
class_index: usize,
) -> Result<ResolvedTieClass> {
const ENTRY: usize = 32;
let start = classes.offset as usize + class_index * ENTRY;
if classes.offset < 0 || start.checked_add(ENTRY).is_none() || start + ENTRY > core_index.len()
{
return Err(Error {
offset: start,
context: "tie class table entry",
kind: ErrorKind::OutOfBounds {
needed: ENTRY,
available: core_index.len().saturating_sub(start),
},
});
}
if class_index >= classes.count as usize {
return Err(Error {
offset: start,
context: "tie class index",
kind: ErrorKind::Invalid("class index past the table count"),
});
}
let entry = ClassEntry::parse(&core_index[start..start + ENTRY])?;
let base = entry.offset_in_asset_wad as usize;
if base < TIE_CLASS_HEADER_MATCHED || base + TIE_CLASS_HEADER_MATCHED > core_data.len() {
return Err(Error {
offset: base,
context: "tie class header",
kind: ErrorKind::OutOfBounds {
needed: TIE_CLASS_HEADER_MATCHED,
available: core_data.len().saturating_sub(base),
},
});
}
let header = TieClassHeader::parse(&core_data[base..base + TIE_CLASS_HEADER_MATCHED])?;
Ok(ResolvedTieClass {
entry,
index: class_index,
header,
})
}
#[cfg(test)]
mod tests {
use super::*;
+31
View File
@@ -1097,3 +1097,34 @@ six bytes remain uninterpreted.
hand-computed `o_class` offset instead of `ShrubClassEntry`, and every
`ShrubHeader::region` call failed. The parser was not wrong; the caller was.
The working call shape was already in the disc test.
## tie: both indirections, and one reading disproved
**The chain, which is two steps and easy to get wrong.** The tie class **table**
lives in the level's `core_index.bin`, not `core_data.bin`. On level 0 the three
tables chain as moby at `192`, 238 x 32 bytes later tie at `7808`, 74 x 32 later
shrub at `10176`, which closes. Reading the same table out of `core_data.bin`
produces plausible garbage rather than an error -- `offset_in_asset_wad` comes
back as 1,217,208,842, which is outside every buffer, so the mistake announces
itself if you look. Then `ClassEntry::offset_in_asset_wad` addresses
**`core_data.bin`**, despite its name and despite `core_data.wad` sitting
alongside it.
**Verified:** all **1,864** tie classes resolve, `o_class` agreeing on every one,
and all **5,592** LOD block offsets land inside `core_data`.
**Disproved:** the word at `+0` of a 16-byte unit inside an LOD block. On the
first twelve units of level 0's class 0 it rises strictly -- 288, 1520, 2848,
3968, 5088, 6096, 7312, 8528, 9792, 10976, 12016, 13072 -- which reads exactly
like a linked list, and the accompanying `i16`s looked quantised (two distinct
values in `i16[0]`, three in `i16[1]`).
Across the whole population it is strictly increasing on **0 of 1,864** blocks,
and `i16[0]` takes **8,969** distinct values on level 0 alone rather than two.
The sample was twelve units out of 1,724,334. Both the monotonic reading and the
quantisation were artifacts of that sample, and the disproof is asserted in
`crates/gc-extract/tests/disc.rs` so it cannot creep back.
**Not established:** the block's record size. 3,580 non-empty LOD blocks compute
as divisible by 16, but some `packets` offsets run backwards, so the "next offset
delimits this block" end is wrong for them and the figure should not be trusted.
+15 -5
View File
@@ -241,11 +241,21 @@ Phase 1 hands over the extents; this phase works out what each one contains.
- [~] `tfrag` — vertex format, topology, texture bindings, per-vertex colour
and normals, LOD selection and parent indices are all recovered and
verified. **Still open:** morph targets and the strip/chain bookkeeping.
- [ ] `tie` — class table and the 0x48-byte header agree with R&C1; the
**vertex layout is not decoded**. The class table entries are pointers
into `core_data.wad`, and the header's `packets[0..3]` LOD offsets are
resolved against the lump base -- neither chain has been walked end to
end yet. Level 0 has 74 tie classes, 238 moby and 7 shrub.
- [~] `tie` — **both indirections now resolve.** `resolve_tie_class` walks the
chain: the class table is in `core_index.bin`, and the entry's
`offset_in_asset_wad` addresses `core_data.bin` (not `core_data.wad`,
despite the field name). All **1,864** tie classes resolve, with
`o_class` agreeing on every one, and all **5,592** LOD block offsets land
inside `core_data`.
- [ ] The tie **block layout is not decoded.** LOD0 blocks are not a uniform
stride of records: 3,580 non-empty blocks compute as divisible by 16,
but that figure is not trustworthy because some `packets` offsets run
backwards, so the "next offset" end is wrong for them. One reading is
**disproved**: the word at `+0` of a 16-byte unit is strictly increasing
on **0 of 1,864** blocks, despite looking exactly like a linked list on
the first twelve units of one class. A 12-unit sample of a 1.7M-unit
population is the pooled-claim error rule 3a exists to prevent; the test
now asserts the disproof so it cannot creep back.
- [~] `shrub` — class table done (48-byte entries, not 32; reading shrub at
32 still gives 231 sane-looking entries of 358, which is the hazard).
Geometry header parsed and checked field by field on all 358 spans: