Make the tie record relocation a command, and verify it on two levels

The relocation rule from the previous commit was established by a script that
lived in a shell history, which means the next person would have had to
rediscover it. It is now `gc-trace verify-tie-records <unpacked-dir> <level>
<class> <lod> <src>`: resolve the class, parse the block, search the RAM image
for the loaded record table, report what it found, and exit 0 only when every
record's pointer has become table + word and every field byte is verbatim.

Swept over every (class, lod) slot of two levels, from PCSX2 savestates:

  level 0: 87 slots with a block, 87 verified, 0 mismatched, 61 absent
  level 1: 104 slots with a block, 104 verified, 0 mismatched, 78 absent

So the rule holds on every LOD block of two levels the game had loaded, which
is what makes the record layout a finding rather than a reading of one block.

Writing the command also caught an off-by-one in the parser it exercises:
TieBlockHeader::records started its slice after the 16-byte header, but the
header IS the first record -- the table runs from the block start to
data_offset, and a block with data_offset 0x50 holds five records, the first
being the header itself and its pointer naming where the payload begins. Found
because the tool's first run relocated nothing while the same check in a script
had passed; the unit test fixture was wrong in the same way and is fixed.

Two operational notes now in the format doc: the live emulator is not a stable
input (verifying level 1 over PINE matched 104 of 104 while the user was on
that planet, and the same command minutes later found no matching copy because
they had returned to level 0 -- a miss there is not a counterexample, so the
report says "no copy matched" rather than "failed"), and a class index past the
table count is an error rather than a mismatch, so a sweep should stop at the
level's class count rather than a round number.

crates/gc-extract/tests/disc.rs

@@ -2355,6 +2355,98 @@ fn chunk_container_closes_the_last_class_region() {
     );
 }
 
+/// The chunk record's leading word is fixed per type.
+///
+/// A record is `[leading u16][two index bytes][payload]`. Measured on the
+/// disc: the word is the same at every record of a chunk on 1,439 of 1,509
+/// chunks (`None` on the 70 that vary, which the parser reports rather than
+/// averaging), and fixed within every chunk type, taking only the values
+/// 1, 2, 4, 6, 8 and 10.
+///
+/// Not named. What the value counts is not established, and two readings were
+/// measured and disproved: it is not the payload's entry count -- the payload
+/// length divided by it leaves 57 distinct fractional "sizes" -- and it is not
+/// the number of distinct leading bytes, which agrees on 0 of 1,506 chunks.
+/// See `docs/formats/core-data-regions.md`.
+#[test]
+fn chunk_record_leading_word_is_fixed_per_type() {
+    require_disc!();
+    let data = std::env::var_os("GC_EXTRACTED")
+        .map(PathBuf::from)
+        .unwrap_or_else(|| {
+            let home = std::env::var("HOME").unwrap_or_else(|_| ".".into());
+            PathBuf::from(home).join("Documents/gc-extracted")
+        });
+    if !data.exists() {
+        eprintln!("skipped: unpack the disc first");
+        return;
+    }
+
+    use gc_formats::chunks::parse_all_chunks;
+    use std::collections::{BTreeMap, BTreeSet};
+
+    // header -> the words its chunks carry, and the value histogram. Chunks
+    // whose own records disagree are counted apart rather than folded in.
+    let mut by_type: BTreeMap<[u8; 16], BTreeSet<u16>> = BTreeMap::new();
+    let mut histogram: BTreeMap<u16, usize> = BTreeMap::new();
+    let mut levels = 0usize;
+    let mut chunks = 0usize;
+    let mut varying = 0usize;
+
+    for n in 0..27 {
+        let core = match std::fs::read(data.join(format!("level{n}/core_data.bin"))) {
+            Ok(c) => c,
+            Err(_) => continue,
+        };
+        let found = parse_all_chunks(&core).expect("chunks should parse");
+        if found.is_empty() {
+            continue;
+        }
+        levels += 1;
+        for c in &found {
+            chunks += 1;
+            match c.chunk.prefix {
+                Some(p) => {
+                    by_type.entry(c.chunk.header).or_default().insert(p);
+                    *histogram.entry(p).or_default() += 1;
+                }
+                None => varying += 1,
+            }
+        }
+    }
+
+    assert!(levels >= 25, "only {levels} levels carry chunks");
+    assert!(chunks > 1400, "only {chunks} chunks parsed");
+    // The word is uniform within a chunk on all but a small minority.
+    let uniform = chunks - varying;
+    assert!(
+        uniform > 1400,
+        "only {uniform} of {chunks} chunks carry a uniform leading word"
+    );
+    // Fixed within every type: no type's chunks carry two different words.
+    let varying_types: Vec<_> = by_type.iter().filter(|(_, v)| v.len() > 1).collect();
+    assert!(
+        varying_types.is_empty(),
+        "{} types carry more than one leading word",
+        varying_types.len()
+    );
+    // The word is never zero and never large, and it is not the record count:
+    // 621 chunks carry 2, 520 carry 4, and the rest are spread thin.
+    assert!(
+        histogram.keys().all(|&p| p > 0 && p <= 16),
+        "prefix values {histogram:?}"
+    );
+    let two = histogram.get(&2).copied().unwrap_or(0);
+    let four = histogram.get(&4).copied().unwrap_or(0);
+    assert!(two > 500, "only {two} chunks carry the word 2");
+    assert!(four > 400, "only {four} chunks carry the word 4");
+    eprintln!(
+        "chunk leading word: {uniform} of {chunks} chunks uniform ({varying} vary), \
+         {} types, values {histogram:?}",
+        by_type.len()
+    );
+}
+
 /// Byte positions that hold the same value in every supplied record.
 fn constant_positions(records: &[&[u8]]) -> usize {
     let Some(first) = records.first() else {
@@ -2713,6 +2805,7 @@ fn tie_classes_resolve_through_both_indirections() {
     let mut hdr_ok = 0usize;
     let mut count_mult16 = 0usize;
     let mut reserved_02_zero = 0usize;
+    let mut tables_bounded = 0usize;
     let mut packed_ok = 0usize;
     let mut padding_all_zero = 0usize;
     let mut packed_seen: Vec<u16> = Vec::new();
@@ -2741,9 +2834,16 @@ fn tie_classes_resolve_through_both_indirections() {
                 };
 
                 blocks += 1;
-                if blk.header.count % 16 == 0 {
+                // The word at +0 is the data offset -- the record table's
+                // length -- so it is a non-zero multiple of 16, and it bounds
+                // the table.
+                if blk.header.data_offset % 16 == 0 && blk.header.data_offset != 0 {
                     count_mult16 += 1
                 }
+                let table = blk.header.records(&core[s..]).count();
+                if table == blk.header.record_count() && table > 0 {
+                    tables_bounded += 1
+                }
                 if blk.header.reserved_02 == 0 {
                     reserved_02_zero += 1
                 }
@@ -2779,9 +2879,15 @@ fn tie_classes_resolve_through_both_indirections() {
     assert_eq!(
         count_mult16,
         blocks,
-        "count not a multiple of 16 on {} of {blocks}",
+        "the data offset is not a non-zero multiple of 16 on {} of {blocks}",
         blocks - count_mult16
     );
+    assert_eq!(
+        tables_bounded,
+        blocks,
+        "the record table is not bounded by the data offset on {} of {blocks}",
+        blocks - tables_bounded
+    );
     assert_eq!(
         reserved_02_zero,
         blocks,
@@ -2973,3 +3079,133 @@ fn chunk_records_start_16_byte_aligned() {
         unaligned.len()
     );
 }
+
+/// Moby class IDs live in one body slot per chunk type, and the slot varies.
+///
+/// Measured over all 27 levels: the best slot per type carries 5 to 42 distinct
+/// valid moby class IDs at 153x to 289x enrichment over the chance rate. The
+/// `distinct >= 4` filter is load-bearing -- without it the top slots are
+/// constants that read as 305x because any value below the moby count counts as
+/// a hit. See `docs/formats/core-data-regions.md`.
+#[test]
+fn moby_class_ids_sit_in_one_slot_per_chunk_type() {
+    require_disc!();
+
+    let data = std::env::var_os("GC_EXTRACTED")
+        .map(PathBuf::from)
+        .unwrap_or_else(|| {
+            let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
+            PathBuf::from(home).join("Documents/gc-extracted")
+        });
+    if !data.exists() {
+        eprintln!("skipped: unpack the disc first");
+        return;
+    }
+
+    use gc_formats::chunks::parse_all_chunks;
+    use gc_formats::level_core::LevelCoreHeader;
+    use std::collections::{BTreeMap, BTreeSet};
+
+    // (type, slot) -> (nonzero, hits, expected, distinct)
+    let mut stat: BTreeMap<(u32, usize), (u64, u64, f64, usize)> = BTreeMap::new();
+    let mut seen: BTreeMap<(u32, usize), BTreeSet<u16>> = BTreeMap::new();
+
+    for n in 0..27 {
+        let core = std::fs::read(data.join(format!("level{n}/core_data.bin"))).expect("core_data");
+        let idx = std::fs::read(data.join(format!("level{n}/core_index.bin"))).expect("core_index");
+        let nm = LevelCoreHeader::parse(&idx)
+            .expect("hdr")
+            .moby_classes
+            .count as u64;
+        if nm == 0 {
+            continue;
+        }
+        for c in parse_all_chunks(&core).expect("chunks") {
+            let t = u32::from_le_bytes([
+                c.chunk.header[0],
+                c.chunk.header[1],
+                c.chunk.header[2],
+                c.chunk.header[3],
+            ]);
+            let len = c.records[0].len();
+            if c.records.iter().any(|r| r.len() != len) {
+                continue;
+            }
+            let mut loc: BTreeMap<usize, (u64, u64)> = BTreeMap::new();
+            for r in c.records.iter() {
+                for k in 1..len / 2 {
+                    let v = u16::from_le_bytes([r[k * 2], r[k * 2 + 1]]);
+                    if v == 0 {
+                        continue;
+                    }
+                    seen.entry((t, k)).or_default().insert(v);
+                    let e = loc.entry(k).or_default();
+                    e.0 += 1;
+                    if (v as u64) < nm {
+                        e.1 += 1
+                    }
+                }
+            }
+            for (k, (nn, hh)) in loc {
+                let e = stat.entry((t, k)).or_default();
+                e.0 += nn;
+                e.1 += hh;
+                e.2 += nn as f64 * (nm as f64 / 65536.0);
+            }
+        }
+    }
+    for (key, s) in seen.iter() {
+        stat.entry(*key).or_default().3 = s.len();
+    }
+
+    // best varying slot per type
+    let mut best: BTreeMap<u32, (usize, f64, usize)> = BTreeMap::new();
+    for ((t, k), (nn, hh, e, dist)) in stat.iter() {
+        if *nn < 8 || *e < 1.0 || *dist < 4 {
+            continue;
+        }
+        let r = (*hh as f64) / (*e);
+        if best.get(t).is_none_or(|(_, ce, _)| r > *ce) {
+            best.insert(*t, (*k, *hh as f64 / *e, *dist));
+        }
+    }
+
+    assert!(
+        best.len() >= 20,
+        "only {} chunk types carry a varying class-ID slot",
+        best.len()
+    );
+    let strong: Vec<_> = best.values().filter(|(_, r, _)| *r >= 100.0).collect();
+    assert!(
+        strong.len() >= 20,
+        "only {} types reach 100x enrichment",
+        strong.len()
+    );
+
+    let mut all: Vec<_> = best
+        .iter()
+        .map(|(t, (k, r, dist))| (*t, *k, *r, *dist))
+        .collect();
+    all.sort_by(|a, b| b.2.partial_cmp(&a.2).unwrap());
+    assert!(!all.is_empty(), "no varying class-ID slots found");
+    let mn = all
+        .iter()
+        .map(|a| a.2)
+        .fold(f64::MAX, |g: f64, v: f64| f64::min(g, v));
+    let mx = all
+        .iter()
+        .map(|a| a.2)
+        .fold(0.0f64, |g: f64, v: f64| f64::max(g, v));
+    let dmin = all.iter().map(|a| a.3).min().unwrap();
+    let dmax = all.iter().map(|a| a.3).max().unwrap();
+
+    eprintln!(
+        "chunk class-ID slots: {} types; best-slot enrichment {mn:.0}x..{mx:.0}x; \
+         distinct IDs per slot {dmin}..{dmax}",
+        best.len()
+    );
+    eprintln!(
+        "  strongest (type, byte, ratio, distinct): {:?}",
+        &all[..all.len().min(6)]
+    );
+}

crates/gc-formats/src/chunks.rs

@@ -43,6 +43,16 @@ pub const CHUNK_GAP_BYTES: usize = 10;
 /// The value every tagged offset's top byte carries, on all 27 levels.
 pub const CHUNK_TAG: u8 = 0xf0;
 
+/// Bytes between a chunk record's leading word and the payload the table
+/// points at.
+///
+/// A record is `[leading u16][two index bytes][payload]`, and the disc's
+/// offset table points at the *payload*. The loaded table in RAM points two
+/// bytes earlier, at the leading word -- the payload's index bytes are then at
+/// `+2` from the pointer -- so the game's record starts here and the disc's
+/// table is two bytes late. See [`Chunk::prefix`].
+pub const RECORD_PREFIX_BYTES: usize = 2;
+
 /// The two bytes immediately preceding every chunk region on the disc.
 ///
 /// The last class body in a level -- shrub where the level has shrub, tie
@@ -112,6 +122,26 @@ pub struct Chunk {
     pub offsets: Vec<u32>,
     /// Record stride, from the first two offsets. Zero if fewer than two.
     pub stride: usize,
+    /// The `u16` immediately before the first record's payload -- the record's
+    /// own leading word, which the loaded table points at.
+    ///
+    /// **Measured on the disc:** the same at every record of a chunk on
+    /// **1,439 of 1,509 chunks** -- `None` on the 70 that vary -- and fixed
+    /// within every chunk type, over the 71 types. Its values are 1, 2, 4, 6,
+    /// 8 and 10; 554 chunks carry 2 and 520 carry 4.
+    ///
+    /// **Measured against the running game:** in PCSX2's RAM the rewritten
+    /// offset table points at this word rather than at the payload, two bytes
+    /// earlier than the disc's table, so the game's record starts here.
+    ///
+    /// Not named. What the value counts is not established: it is *not* the
+    /// payload's entry count (the payload length divided by it is a whole
+    /// number on 1,344 chunks and an odd fraction on the other 162, leaving 57
+    /// distinct "sizes"), and it is not the number of distinct leading bytes
+    /// (agrees on 0 of 1,506). An earlier note in this file read the
+    /// `02 00` seen in RAM as a prefix the loader had inserted; those bytes
+    /// are on the disc already, at every record, which is what this field is.
+    pub prefix: Option<u16>,
 }
 
 /// One chunk, with its records already bounds-checked.
@@ -324,12 +354,35 @@ pub fn parse_chunk(region: &[u8], offset: usize) -> Result<Option<Chunk>> {
         0
     };
 
+    // The leading word sits two bytes before the payload the table points at.
+    // Reading it and requiring it to be the same at every record is a
+    // signature check like the header's last byte: it holds on every chunk on
+    // The leading word sits two bytes before the payload the table points at.
+    // It is the same at every record of a chunk on 1,439 of 1,509 chunks; the
+    // other 70 vary, so it stays an Option rather than a value a caller could
+    // mistake for a field that is always there.
+    let read_prefix = |low: u32| -> Option<u16> {
+        let at = offset
+            .checked_add(low as usize)?
+            .checked_sub(RECORD_PREFIX_BYTES)?;
+        region
+            .get(at..at + RECORD_PREFIX_BYTES)
+            .map(|b| u16::from_le_bytes([b[0], b[1]]))
+    };
+    let prefix = offsets.first().copied().and_then(read_prefix).filter(|&p| {
+        offsets
+            .iter()
+            .skip(1)
+            .all(|&low| read_prefix(low) == Some(p))
+    });
+
     Ok(Some(Chunk {
         count: header[14],
         header,
         offset,
         offsets,
         stride,
+        prefix,
     }))
 }
 
@@ -360,6 +413,7 @@ pub fn resolve<'r>(chunk: &Chunk, region: &'r [u8]) -> Result<ResolvedChunk<'r>>
         chunk: Chunk {
             count: chunk.count,
             header: chunk.header,
+            prefix: chunk.prefix,
             offset: chunk.offset,
             offsets: chunk.offsets.clone(),
             stride: chunk.stride,

crates/gc-formats/src/tie.rs

@@ -124,13 +124,23 @@ pub const TIE_PACKED_TEXTURE_VALUES: [u16; 4] = [0x0501, 0x0a02, 0x0f03, 0x1404]
 /// 1,403 classes are unmeasured, not measured-and-different.
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub struct TieBlockHeader {
-    /// `+0x00`: element count. A multiple of 16 on all 2,177 blocks, 42 distinct
+    /// `+0x00`: the byte offset from the start of the block to the vertex
-    /// values, never zero.
+    /// data -- that is, the length of the record table that follows this
+    /// header. A multiple of 16 on all 2,177 blocks (42 distinct values, never
+    /// zero), because the table holds 16-byte records: one block with this
+    /// word at 0x10 has one record, one at 0x50 has five.
     ///
-    /// It is **not** a record count over the block: `block_size - header` is
+    /// This was read as an element *count* for a while, and the reading failed
-    /// divisible by `count` on 0 of 922 blocks, for any record size and any
+    /// in a way worth recording: the block's payload length divided by it is a
-    /// header up to 512 bytes.
+    /// whole number on 0 of 922 blocks, at any record size and any header up
-    pub count: u16,
+    /// to 512 bytes. A count of what is in the payload was never going to fit,
+    /// because the word is not about the payload -- it is where the payload
+    /// starts.
+    ///
+    /// Confirmed against the running game: each record's leading word is
+    /// relocated at load to `table_address + word`, and the table's length is
+    /// this word, on every block checked in RAM (16 of 16 on level 1).
+    pub data_offset: u16,
     /// `+0x02`: zero on all 2,177 blocks.
     pub reserved_02: u16,
     /// `+0x04`: one of exactly [`TIE_PACKED_TEXTURE_VALUES`] on all 2,177 blocks.
@@ -155,7 +165,7 @@ impl TieBlockHeader {
     pub fn parse(block: &[u8]) -> Result<Self> {
         let mut b = Buf::new(block);
         Ok(Self {
-            count: b.u16("tie block count")?,
+            data_offset: b.u16("tie block data offset")?,
             reserved_02: b.u16("tie block reserved")?,
             packed_texture: b.u16("tie block packed texture")?,
             field_06: b.u16("tie block field_06")?,
@@ -168,16 +178,69 @@ impl TieBlockHeader {
 
     /// True when every invariant measured across all 27 levels holds.
     ///
-    /// Only the two that are universal: `count` is a multiple of 16 and
+    /// Only the two that are universal: the data offset is a non-zero multiple
-    /// `reserved_02` is zero, both on all 2,177 blocks.
+    /// of 16 and `reserved_02` is zero, both on all 2,177 blocks.
     pub fn invariants_hold(&self) -> bool {
-        self.count % 16 == 0 && self.reserved_02 == 0
+        self.data_offset % 16 == 0 && self.data_offset != 0 && self.reserved_02 == 0
+    }
+
+    /// The number of 16-byte records in this block's table.
+    pub fn record_count(&self) -> usize {
+        self.data_offset as usize / TIE_RECORD_SIZE
+    }
+
+    /// The block's records, as byte slices of [`TIE_RECORD_SIZE`].
+    ///
+    /// Bounds-checked against `block`, which is the block starting at its
+    /// header. **The header is the first record**: the table runs from the
+    /// block start to `data_offset`, so a block with `data_offset == 0x50`
+    /// holds five records, the first of which is the header itself and whose
+    /// pointer names where the payload begins. A header whose table runs past
+    /// the block yields nothing rather than a short slice.
+    pub fn records<'b>(&self, block: &'b [u8]) -> impl Iterator<Item = &'b [u8]> + 'b {
+        block
+            .get(..self.record_count() * TIE_RECORD_SIZE)
+            .unwrap_or(&[])
+            .chunks_exact(TIE_RECORD_SIZE)
+    }
+}
+
+/// One 16-byte tie block record.
+///
+/// A block is a table of these, followed by the vertex payload at
+/// [`TieBlockHeader::data_offset`].
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct TieRecord {
+    /// The record's leading word: a pointer **relative to the start of the
+    /// record table**, relocating at load to `table_address + pointer`.
+    ///
+    /// Confirmed on every block checked in PCSX2's RAM: the word becomes
+    /// `table + word` with the other twelve bytes untouched, on 16 of 16
+    /// level-1 blocks.
+    pub pointer: u32,
+    /// The remaining twelve bytes, copied verbatim disc to RAM.
+    pub fields: [u8; 12],
+}
+
+impl TieRecord {
+    /// Parse one record.
+    pub fn parse(bytes: &[u8]) -> Result<TieRecord> {
+        let mut b = Buf::new(bytes);
+        let pointer = b.u32("tie record pointer")?;
+        let mut fields = [0u8; 12];
+        for slot in fields.iter_mut() {
+            *slot = b.u8("tie record field")?;
+        }
+        Ok(TieRecord { pointer, fields })
     }
 }
 
 /// Byte that terminates a tie LOD block's payload.
 pub const TIE_BLOCK_TERMINATOR: u8 = 0xf6;
 
+/// Bytes in one tie block record.
+pub const TIE_RECORD_SIZE: usize = 0x10;
+
 /// A tie LOD block: its header, its payload, and the zero padding that follows
 /// the terminator.
 ///
@@ -346,4 +409,57 @@ mod tests {
         assert_eq!(h.lod_for_depth(30.0), Some(2));
         assert_eq!(h.lod_for_depth(50.0), None);
     }
+
+    /// A block whose data offset names a four-record table, the first of
+    /// which is the header itself.
+    fn block() -> Vec<u8> {
+        let mut v = Vec::new();
+        // data_offset 0x40 -- four records -- reserved 0, packed, then the
+        // remaining fields.
+        v.extend_from_slice(&0x40u16.to_le_bytes());
+        v.extend_from_slice(&0u16.to_le_bytes());
+        v.extend_from_slice(&0x0501u16.to_le_bytes());
+        v.extend_from_slice(&0x1234u16.to_le_bytes());
+        v.extend_from_slice(&0x5678u16.to_le_bytes());
+        v.extend_from_slice(&0x9abcu16.to_le_bytes());
+        v.extend_from_slice(&0x0442u16.to_le_bytes());
+        v.extend_from_slice(&0u16.to_le_bytes());
+        assert_eq!(v.len(), TIE_BLOCK_HEADER_SIZE);
+        // Three records, 0x30 bytes, then the payload.
+        for k in 0..3u32 {
+            v.extend_from_slice(&(0x40 + k * 0x20).to_le_bytes());
+            for b in 0..12u8 {
+                v.push(b + k as u8);
+            }
+        }
+        v.extend_from_slice(&[0xa5u8; 8]);
+        v
+    }
+
+    #[test]
+    fn the_data_offset_bounds_the_record_table() {
+        let b = block();
+        let h = TieBlockHeader::parse(&b).unwrap();
+        assert!(h.invariants_hold());
+        assert_eq!(h.record_count(), 4);
+        let records: Vec<_> = h.records(&b).collect();
+        assert_eq!(records.len(), 4);
+        // The first record is the header itself, pointing at the payload.
+        let first = TieRecord::parse(records[0]).unwrap();
+        assert_eq!(first.pointer, 0x40);
+        assert_eq!(first.pointer, 0x40);
+    }
+
+    #[test]
+    fn a_table_running_past_the_block_yields_nothing() {
+        let mut b = block();
+        // Claim a table far longer than the block.
+        b[0..2].copy_from_slice(&0x400u16.to_le_bytes());
+        let h = TieBlockHeader::parse(&b).unwrap();
+        assert_eq!(h.record_count(), 0x40);
+        assert_eq!(h.records(&b).count(), 0, "must not slice past the block");
+        // A 0x400 offset is still a multiple of 16, so the invariant holds even
+        // though the table cannot exist.
+        assert!(h.invariants_hold());
+    }
 }

docs/formats/core-data-regions.md

@@ -1194,6 +1194,69 @@ guess; that it is two fields is not.
 Combined with block sizes always being multiples of 16, that rules out a uniform
 stride without a header.
 
+## tie: the block is a table of 16-byte records (from RAM)
+
+**Range measured:** every tie LOD block of level 1 whose leading word is a
+non-zero multiple of 16 -- 16 blocks -- plus the two read by hand in full.
+
+**Method:** locate each block's record table in PCSX2's RAM by its twelve field
+bytes (the records are the only part of a block with a distinctive 12-byte
+run), then compare every record against its disc copy. Each block has several
+copies in RAM; the one that matches is the loaded copy.
+
+**Result: on 16 of 16 blocks, every record's leading word has become
+`table_address + word`, and the other twelve bytes are byte-identical to the
+disc.** So:
+
+- A block is `[16-byte header][record table][vertex payload]`, and the records
+  are **16 bytes**: a leading `u32` pointer and twelve bytes of fields.
+- The header's word at `+0x00` is the **byte offset from the block start to
+  the vertex payload** -- that is, the length of the record table. This is what
+  the "count" reading was groping at: it is a multiple of 16 on all 2,177 blocks
+  because the table holds 16-byte records, it is never zero because there is
+  always at least the header-sized first record, and its 42 distinct values are
+  table lengths, not counts of anything in the payload.
+- Every record's leading word is a pointer **relative to the start of the record
+  table**, relocated at load to `table + word`. The twelve field bytes are
+  copied verbatim.
+
+This also explains the dead end that ran before it. The payload length divided
+by the word at `+0x00` is a whole number on **0 of 922** blocks, at any record
+size and any header up to 512 bytes -- which is the correct measurement of the
+wrong hypothesis: the word never described the payload, it described where the
+payload starts. A count of what follows a pointer is not the pointer.
+
+`TieBlockHeader::data_offset` and `TieBlockHeader::records` carry it, bound by
+the offset, and the disc test asserts the table is bounded on every block.
+
+### It is now a command, and it verifies two levels
+
+The check above was a script; it is now `gc-trace verify-tie-records
+<unpacked-dir> <level> <class> <lod> <src>`, which resolves the class, parses
+the block, searches the RAM image for the loaded record table and reports what
+it found. Exit status is 0 when every record relocated with verbatim fields.
+
+Run over every (class, lod) slot of two levels, from PCSX2 savestates:
+
+| level | slots with a block | verified | mismatched | absent |
+|---|---|---|---|---|
+| 0 | 87 | **87** | 0 | 61 |
+| 1 | 104 | **104** | 0 | 78 |
+
+So the relocation rule holds on **every** LOD block of two levels that the game
+had loaded. Two operational notes worth keeping:
+
+- **The live emulator is not a stable input.** Verifying level 1 over PINE
+  matched 104 of 104 while the user was on that planet; the same command
+  minutes later, after they had returned to level 0, found the tables in RAM
+  but no matching copy -- because the level was no longer loaded. A miss there
+  is not a counterexample, and the report says "no copy matched" rather than
+  "failed".
+- **A class index past the table count is an error**, not a mismatch, so a
+  sweep should stop at the level's class count rather than at a round number.
+
+What the records' fields name and what the payload holds are still open.
+
 ### What the payload is, and what retired the coordinate reading
 
 The payload is **not random**, and that took a control to establish. Reading each
@@ -1451,3 +1514,40 @@ The third result is the useful one: the byte at `+2` takes exactly
 `0, 16, 32, ... 240` -- the 16 values `index_rule` predicts for `step = 16,
 phase = 0`. The mixed-radix record index, derived from the disc, is corroborated
 on the running game's own copy.
+
+### Correction: the `02 00` is on the disc, and a record starts two bytes earlier
+
+What the loader does to the *pointer* is as described above. What it does to
+the *data* was overstated: the `02 00` read as an inserted prefix is already
+the record's own leading word, present at every record of every chunk on the
+disc. Three measurements, in order:
+
+- **The 2 bytes before the payload are on disc.** For the chunk at level 0
+  `0x100b302` (90 records), the word before each payload is `02 00` in
+  `core_data` and `02 00` in RAM, byte for byte, record for record. The first
+  chunk of the same level carries `08 00` instead. Nothing was inserted.
+- **So the RAM table's pointer is simply two bytes earlier.** The disc table's
+  low 24 bits point at the payload; the RAM table's point at the payload minus
+  2. The record is `[leading u16][two index bytes][payload]`, and the loaded
+  table addresses the leading word -- which is why the index bytes read as
+  `+2` from the pointer, the same `+2` the result above quotes.
+- **The leading word is a field, not a prefix.** It is the same at every
+  record of a chunk on **1,439 of 1,509 chunks** (`None` on the 70 whose
+  records disagree, which the parser reports rather than averaging), it is
+  fixed within **every one of the 71 chunk types**, and its values are only
+  `1, 2, 4, 6, 8, 10` -- 554 chunks carry 2, 520 carry 4. `Chunk::prefix`
+  carries it.
+
+What the value counts is not established, and two readings were measured and
+disproved rather than left as guesses:
+
+- **Not an entry count.** The payload length divided by the word is a whole
+  number on 1,344 chunks, but the other 162 leave an odd fraction and the
+  whole set spans 57 distinct "sizes" (71, 66, 78, 24, 23, ...). A real entry
+  size is a small clean number, not a set like that.
+- **Not the number of distinct leading bytes.** That agrees on 0 of 1,506
+  chunks.
+
+So the record has one more field than the loader note claimed, the disc's
+offset table is two bytes late relative to it, and what it counts is the next
+question -- with the entry-count reading now out of the way.

docs/plan/pcsx2-verification.md

@@ -153,6 +153,11 @@ GC_DISC="$HOME/Desktop/pcsx2/games/Ratchet & Clank - Going Commando.iso" \
 ```sh
 cargo run -q -p gc-trace -- pine-info
 cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 0 --pine
+cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 1 \
+    --state ~/.config/PCSX2/sstates/SCUS-97268\ \(38996035\).02.p2s
+# the tie record relocation, one class and LOD at a time
+cargo run -q -p gc-trace -- verify-tie-records ~/Documents/gc-extracted 0 0 1 \
+    --state ~/.config/PCSX2/sstates/SCUS-97268\ \(38996035\).01.p2s
 cargo run -q -p gc-trace -- compare-tfrag-light ~/Documents/gc-extracted --level 0 \
     --state ~/.config/PCSX2/sstates/<name>.p2s
 ```

docs/roadmap.md

@@ -275,6 +275,16 @@ Phase 1 hands over the extents; this phase works out what each one contains.
       three floats in the billboard record, `size`, `mode_bits`, `count`, and
       **where the mesh geometry lives** — R&C1 puts the mesh packets in this
       blob and R&C2's blob holds only the palette.
+- [x] **The tie LOD block is a table of 16-byte records.** The header's word at
+      `+0x00` is the byte offset from the block start to the vertex payload --
+      the record table's length -- not a count, which is why the payload never
+      divides by it. Each record is a leading `u32` pointer **relative to the
+      table start** plus twelve verbatim field bytes, relocated at load to
+      `table + word`.
+- [x] **The tie relocation verifies on two levels.** `gc-trace
+      verify-tie-records` checks it against PCSX2's RAM: **87 of 87** blocks on
+      level 0 and **104 of 104** on level 1, zero mismatches, which is what
+      makes the record layout a finding rather than a reading of one block.
 - [x] **The shrub mesh geometry is in the class blob.** A packet table at
       `+0x40` names `count` packets between the table and the palette, and
       they tile on **358 of 358 spans** (2,239 packets), which makes `count`
@@ -352,11 +362,17 @@ Phase 1 hands over the extents; this phase works out what each one contains.
 - [x] **The loader's transformation, from PCSX2 RAM.** Disc offsets
       `0xf0000000 | low24` are rewritten in RAM to absolute EE pointers
       `0xf1850000 | low24` -- tag replaced by the chunk's load base, low 24 bits
-      untouched -- and every record gains a constant `02 00` prefix that the
+      untouched. The payload itself is copied **verbatim**. The index finding
-      rewritten table points at *instead of* the record data (**90 of 90**
+      is corroborated live: the byte at `+2` takes exactly `0, 16, ... 240`, the
-      records). The payload itself is copied **verbatim**. The index finding is
+      16 values `index_rule` predicts.
-      corroborated live: the byte at `+2` takes exactly `0, 16, ... 240`, the 16
+- [x] **Corrected: a record starts two bytes before the table's offset.** The
-      values `index_rule` predicts.
+      `02 00` read in RAM as a loader-inserted prefix is the record's own
+      leading word, already on the disc at every record. The loaded table
+      simply points two bytes earlier than the disc's, at that word. It is a
+      field: uniform across a chunk's records on **1,439 of 1,509** chunks,
+      fixed within **all 71 types**, values `{1, 2, 4, 6, 8, 10}`. Disproved
+      readings: an entry count (57 distinct fractional "sizes") and a
+      distinct-leading-byte count (0 of 1,506 agree).
 - [ ] The chunk **record fields are not decoded**, and no type is named. The body
       is dense (74% of `f32` reads plausible against a 49% random control) with
       an index-like tail -- 12 of 163 `u16` slots in the largest type stay below

tools/trace/src/lib.rs

@@ -28,6 +28,7 @@ pub mod ee;
 pub mod p2s;
 pub mod pine;
 pub mod tfrag_cmp;
+pub mod tie_verify;
 pub mod zip;
 
 pub use ee::EeRam;

tools/trace/src/main.rs

@@ -64,6 +64,34 @@ fn run(args: &[String]) -> Result<i32> {
             let len = require(&rest, "--len")?;
             read(&src, &addr, &len)
         }
+        "verify-tie-records" => {
+            // gc-trace verify-tie-records <unpacked-dir> <level> <class> <lod> <src>
+            let (src, rest) = Source::parse_with_rest(&args[1..])?;
+            let (dir, level, class, lod) = match rest.as_slice() {
+                [dir, level, class, lod, ..] => (
+                    PathBuf::from(dir),
+                    level.parse::<usize>().context("level")?,
+                    class.parse::<usize>().context("class")?,
+                    lod.parse::<usize>().context("lod")?,
+                ),
+                _ => {
+                    return Err(anyhow!(
+                    "usage: gc-trace verify-tie-records <unpacked-dir> <level> <class> <lod> <src>"
+                ))
+                }
+            };
+            let idx = std::fs::read(dir.join(format!("level{level}/core_index.bin")))
+                .with_context(|| format!("level{level}/core_index.bin"))?;
+            let core = std::fs::read(dir.join(format!("level{level}/core_data.bin")))
+                .with_context(|| format!("level{level}/core_data.bin"))?;
+            let bytes = src.load()?;
+            let ram = EeRam::new(&bytes)?;
+            let report = gc_trace::tie_verify::verify(&idx, &core, class, lod, &ram)?;
+            let mut text = String::new();
+            gc_trace::tie_verify::print_report(&report, &mut text)?;
+            println!("{text}");
+            Ok(report.exit_code())
+        }
         "compare-tfrag-light" => {
             let (src, rest) = Source::parse_with_rest(&args[1..])?;
             let dir = rest
@@ -95,6 +123,7 @@ fn run(args: &[String]) -> Result<i32> {
 const USAGE: &str = "\
 usage: gc-trace <command> ...
   compare-tfrag-light <unpacked-dir> [--level N] <src>
+  verify-tie-records <unpacked-dir> <level> <class> <lod> <src>
   info <src>
   dump-ee <src> [--out FILE]
   find <src> --bytes HEX

tools/trace/src/tie_verify.rs

@@ -0,0 +1,172 @@
+//! The tie record relocation, verified against the game's own RAM.
+//!
+//! A tie LOD block is a table of 16-byte records -- a leading `u32` pointer plus
+//! twelve bytes of fields -- and the loader relocates every pointer to
+//! `table_address + word`, leaving the twelve bytes verbatim. That is what
+//! [`verify`] checks, and it is the only way to see it: nothing in the disc
+//! bytes distinguishes a relocated pointer from any other small integer.
+//!
+//! The search is by content, because each block has **several copies in RAM**
+//! and only one of them is the loaded record table. Checking the first hit is
+//! how a first attempt at this reported 0 of 16 blocks: the copy it found had
+//! *modified* field bytes, which is itself interesting and is why every
+//! candidate is tried rather than the first.
+
+use anyhow::{anyhow, Result};
+
+use gc_formats::level_core::LevelCoreHeader;
+use gc_formats::tie::{
+    parse_tie_block, resolve_tie_class, TieBlockHeader, TieRecord, TIE_BLOCK_HEADER_SIZE,
+};
+
+use crate::ee::EeRam;
+
+/// One verification, everything a report needs.
+#[derive(Debug, Clone)]
+pub struct TieReport {
+    pub class_index: usize,
+    pub lod: usize,
+    /// Block offset from the class header, and its length in bytes.
+    pub block_offset: usize,
+    pub block_len: usize,
+    /// Records in the block's table.
+    pub records: usize,
+    /// Copies of the record table found in RAM.
+    pub copies: usize,
+    /// RAM address of the loaded table, when one matched.
+    pub table_ram: Option<usize>,
+    /// Pointers relocated as `table + word`, and records whose twelve field
+    /// bytes are verbatim.
+    pub pointers_relocated: usize,
+    pub fields_verbatim: usize,
+}
+
+impl TieReport {
+    /// 0 = every record relocated with verbatim fields, 2 = mismatches.
+    pub fn exit_code(&self) -> i32 {
+        if self.table_ram.is_some()
+            && self.pointers_relocated == self.records
+            && self.fields_verbatim == self.records
+        {
+            0
+        } else {
+            2
+        }
+    }
+}
+
+/// Verify one class's LOD block against a RAM image.
+///
+/// `lod` is 1 or 2 -- slot 0 is the constant [`gc_formats::tie::TIE_PACKETS0`]
+/// and never a block.
+pub fn verify(
+    core_index: &[u8],
+    core_data: &[u8],
+    class_index: usize,
+    lod: usize,
+    ram: &EeRam,
+) -> Result<TieReport> {
+    let header = LevelCoreHeader::parse(core_index)?;
+    let class = resolve_tie_class(core_index, core_data, header.tie_classes, class_index)?;
+    let offset = class
+        .header
+        .packets
+        .get(lod)
+        .copied()
+        .ok_or_else(|| anyhow!("lod must be 1 or 2, got {lod}"))?;
+    if offset <= 0 {
+        anyhow::bail!("class {class_index} has no block at LOD {lod}: the offset is {offset}");
+    }
+    let start = class.entry.offset_in_asset_wad as usize + offset as usize;
+    let parsed = parse_tie_block(core_data, start)
+        .ok_or_else(|| anyhow!("no tie block at {start:#x} in core_data"))?;
+    // Header, then everything up to and including the terminator and padding.
+    let block_len = TIE_BLOCK_HEADER_SIZE + parsed.payload.len() + parsed.padding.len();
+    let slice = core_data
+        .get(start..start + block_len)
+        .ok_or_else(|| anyhow!("the tie block at {start:#x} runs past core_data"))?;
+
+    let table = TieBlockHeader::parse(slice)?;
+    let count = table.record_count();
+    if count == 0 {
+        anyhow::bail!("the block's table is empty");
+    }
+    let records: Vec<TieRecord> = table
+        .records(slice)
+        .map(|r| TieRecord::parse(r).map_err(Into::into))
+        .collect::<Result<_>>()?;
+
+    // The probe is the first record whose twelve field bytes are distinctive
+    // enough to find: a run of one repeated byte occurs all over RAM.
+    let probe = records
+        .iter()
+        .map(|r| r.fields)
+        .find(|f| f.iter().collect::<std::collections::HashSet<_>>().len() >= 4)
+        .ok_or_else(|| anyhow!("no record has a distinctive field run"))?;
+
+    // Every copy of the table in RAM. The one where every pointer has become
+    // table + word and every field run is verbatim is the loaded copy.
+    let mut copies = 0usize;
+    let mut table_ram = None;
+    let (mut relocated, mut verbatim) = (0usize, 0usize);
+    for hit in ram.find_all(&probe) {
+        let base = match hit.checked_sub(4) {
+            Some(b) => b,
+            None => continue,
+        };
+        copies += 1;
+        let ok_ptr = records.iter().enumerate().all(|(k, r)| {
+            ram.slice(base + k * 16, 4)
+                .map(|b| u32::from_le_bytes(b.try_into().unwrap()) as usize)
+                == Some(r.pointer as usize + base)
+        });
+        let ok_fields = records
+            .iter()
+            .enumerate()
+            .all(|(k, r)| ram.slice(base + k * 16 + 4, 12) == Some(&r.fields[..]));
+        if ok_ptr && ok_fields {
+            table_ram = Some(base);
+            relocated = records.len();
+            verbatim = records.len();
+            break;
+        }
+    }
+
+    Ok(TieReport {
+        class_index,
+        lod,
+        block_offset: offset as usize,
+        block_len: slice.len(),
+        records: count,
+        copies,
+        table_ram,
+        pointers_relocated: relocated,
+        fields_verbatim: verbatim,
+    })
+}
+
+/// Render a report for a terminal.
+pub fn print_report(r: &TieReport, out: &mut dyn std::fmt::Write) -> std::fmt::Result {
+    let _ = writeln!(
+        out,
+        "class {} LOD {}: block at +{:#x}, {} bytes, {} records of 16",
+        r.class_index, r.lod, r.block_offset, r.block_len, r.records
+    );
+    let _ = writeln!(out, "  copies of the record table in RAM: {}", r.copies);
+    match r.table_ram {
+        Some(at) => {
+            writeln!(out, "  loaded table at {at:#x}")?;
+            writeln!(
+                out,
+                "  pointers relocated to table + word: {}/{}",
+                r.pointers_relocated, r.records
+            )?;
+            writeln!(
+                out,
+                "  field bytes verbatim:               {}/{}",
+                r.fields_verbatim, r.records
+            )
+        }
+        None => writeln!(out, "  no copy of the table matched the relocation rule"),
+    }
+}