Decode the tie LOD block: it is a table of 16-byte records

The tie block's word at +0x00 was read as an element count, and the reading
failed in a way worth recording: the payload length divided by it is a whole
number on 0 of 922 blocks, at any record size and any header up to 512 bytes.
That is the correct measurement of the wrong hypothesis -- the word never
described the payload.

Range measured: every LOD block of level 1 whose word is a non-zero multiple
of 16, plus the two read by hand in full.

Method: locate each block's record table in PCSX2's RAM by its twelve field
bytes, which are the only distinctive 12-byte run in a block, then compare
every record against its disc copy. Each block has several copies in RAM, and
the one that matches is the loaded one -- checking the first hit is what made
the first attempt report 0 of 16.

Result: on 16 of 16 blocks, every record's leading word has become
table_address + word, and the other twelve bytes are byte-identical to the
disc. So a block is [16-byte header][record table][vertex payload], the records
are 16 bytes -- a leading u32 pointer plus twelve bytes of fields -- and the
header's word at +0x00 is the byte offset from the block start to the payload,
that is, the length of the table.

That is what the multiple-of-16 was: the table holds 16-byte records. It is
never zero because there is always at least the first record, and its 42
distinct values are table lengths. Each record's pointer is relative to the
table start, relocated at load to table + word.

TieBlockHeader::data_offset and TieBlockHeader::records carry it, bound by the
offset, with TieRecord for the twelve field bytes, and the disc test asserts
the table is bounded on every block. What the records' fields name and what the
payload holds are still open -- the payload is known to be variable-length,
float-shaped, not LZ77 and not VIF1.

crates/gc-extract/tests/disc.rs

@@ -2805,6 +2805,7 @@ fn tie_classes_resolve_through_both_indirections() {
     let mut hdr_ok = 0usize;
     let mut count_mult16 = 0usize;
     let mut reserved_02_zero = 0usize;
+    let mut tables_bounded = 0usize;
     let mut packed_ok = 0usize;
     let mut padding_all_zero = 0usize;
     let mut packed_seen: Vec<u16> = Vec::new();
@@ -2833,9 +2834,16 @@ fn tie_classes_resolve_through_both_indirections() {
                 };
 
                 blocks += 1;
-                if blk.header.count % 16 == 0 {
+                // The word at +0 is the data offset -- the record table's
+                // length -- so it is a non-zero multiple of 16, and it bounds
+                // the table.
+                if blk.header.data_offset % 16 == 0 && blk.header.data_offset != 0 {
                     count_mult16 += 1
                 }
+                let table = blk.header.records(&core[s..]).count();
+                if table == blk.header.record_count() && table > 0 {
+                    tables_bounded += 1
+                }
                 if blk.header.reserved_02 == 0 {
                     reserved_02_zero += 1
                 }
@@ -2871,9 +2879,15 @@ fn tie_classes_resolve_through_both_indirections() {
     assert_eq!(
         count_mult16,
         blocks,
-        "count not a multiple of 16 on {} of {blocks}",
+        "the data offset is not a non-zero multiple of 16 on {} of {blocks}",
         blocks - count_mult16
     );
+    assert_eq!(
+        tables_bounded,
+        blocks,
+        "the record table is not bounded by the data offset on {} of {blocks}",
+        blocks - tables_bounded
+    );
     assert_eq!(
         reserved_02_zero,
         blocks,

crates/gc-formats/src/tie.rs

@@ -124,13 +124,23 @@ pub const TIE_PACKED_TEXTURE_VALUES: [u16; 4] = [0x0501, 0x0a02, 0x0f03, 0x1404]
 /// 1,403 classes are unmeasured, not measured-and-different.
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub struct TieBlockHeader {
-    /// `+0x00`: element count. A multiple of 16 on all 2,177 blocks, 42 distinct
-    /// values, never zero.
+    /// `+0x00`: the byte offset from the start of the block to the vertex
+    /// data -- that is, the length of the record table that follows this
+    /// header. A multiple of 16 on all 2,177 blocks (42 distinct values, never
+    /// zero), because the table holds 16-byte records: one block with this
+    /// word at 0x10 has one record, one at 0x50 has five.
     ///
-    /// It is **not** a record count over the block: `block_size - header` is
-    /// divisible by `count` on 0 of 922 blocks, for any record size and any
-    /// header up to 512 bytes.
-    pub count: u16,
+    /// This was read as an element *count* for a while, and the reading failed
+    /// in a way worth recording: the block's payload length divided by it is a
+    /// whole number on 0 of 922 blocks, at any record size and any header up
+    /// to 512 bytes. A count of what is in the payload was never going to fit,
+    /// because the word is not about the payload -- it is where the payload
+    /// starts.
+    ///
+    /// Confirmed against the running game: each record's leading word is
+    /// relocated at load to `table_address + word`, and the table's length is
+    /// this word, on every block checked in RAM (16 of 16 on level 1).
+    pub data_offset: u16,
     /// `+0x02`: zero on all 2,177 blocks.
     pub reserved_02: u16,
     /// `+0x04`: one of exactly [`TIE_PACKED_TEXTURE_VALUES`] on all 2,177 blocks.
@@ -155,7 +165,7 @@ impl TieBlockHeader {
     pub fn parse(block: &[u8]) -> Result<Self> {
         let mut b = Buf::new(block);
         Ok(Self {
-            count: b.u16("tie block count")?,
+            data_offset: b.u16("tie block data offset")?,
             reserved_02: b.u16("tie block reserved")?,
             packed_texture: b.u16("tie block packed texture")?,
             field_06: b.u16("tie block field_06")?,
@@ -168,16 +178,70 @@ impl TieBlockHeader {
 
     /// True when every invariant measured across all 27 levels holds.
     ///
-    /// Only the two that are universal: `count` is a multiple of 16 and
-    /// `reserved_02` is zero, both on all 2,177 blocks.
+    /// Only the two that are universal: the data offset is a non-zero multiple
+    /// of 16 and `reserved_02` is zero, both on all 2,177 blocks.
     pub fn invariants_hold(&self) -> bool {
-        self.count % 16 == 0 && self.reserved_02 == 0
+        self.data_offset % 16 == 0 && self.data_offset != 0 && self.reserved_02 == 0
+    }
+
+    /// The number of 16-byte records in this block's table.
+    pub fn record_count(&self) -> usize {
+        self.data_offset as usize / TIE_RECORD_SIZE
+    }
+
+    /// The block's records, as byte slices of [`TIE_RECORD_SIZE`].
+    ///
+    /// Bounds-checked against `block`, which is the block starting at its
+    /// header: the table runs from the end of the header to `data_offset`. A
+    /// header whose table runs past the block yields nothing rather than a
+    /// short slice.
+    pub fn records<'b>(&self, block: &'b [u8]) -> impl Iterator<Item = &'b [u8]> + 'b {
+        block
+            .get(
+                TIE_BLOCK_HEADER_SIZE
+                    ..TIE_BLOCK_HEADER_SIZE + self.record_count() * TIE_RECORD_SIZE,
+            )
+            .unwrap_or(&[])
+            .chunks_exact(TIE_RECORD_SIZE)
+    }
+}
+
+/// One 16-byte tie block record.
+///
+/// A block is a table of these, followed by the vertex payload at
+/// [`TieBlockHeader::data_offset`].
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct TieRecord {
+    /// The record's leading word: a pointer **relative to the start of the
+    /// record table**, relocating at load to `table_address + pointer`.
+    ///
+    /// Confirmed on every block checked in PCSX2's RAM: the word becomes
+    /// `table + word` with the other twelve bytes untouched, on 16 of 16
+    /// level-1 blocks.
+    pub pointer: u32,
+    /// The remaining twelve bytes, copied verbatim disc to RAM.
+    pub fields: [u8; 12],
+}
+
+impl TieRecord {
+    /// Parse one record.
+    pub fn parse(bytes: &[u8]) -> Result<TieRecord> {
+        let mut b = Buf::new(bytes);
+        let pointer = b.u32("tie record pointer")?;
+        let mut fields = [0u8; 12];
+        for slot in fields.iter_mut() {
+            *slot = b.u8("tie record field")?;
+        }
+        Ok(TieRecord { pointer, fields })
     }
 }
 
 /// Byte that terminates a tie LOD block's payload.
 pub const TIE_BLOCK_TERMINATOR: u8 = 0xf6;
 
+/// Bytes in one tie block record.
+pub const TIE_RECORD_SIZE: usize = 0x10;
+
 /// A tie LOD block: its header, its payload, and the zero padding that follows
 /// the terminator.
 ///
@@ -346,4 +410,54 @@ mod tests {
         assert_eq!(h.lod_for_depth(30.0), Some(2));
         assert_eq!(h.lod_for_depth(50.0), None);
     }
+
+    /// A block whose data offset names a three-record table.
+    fn block() -> Vec<u8> {
+        let mut v = Vec::new();
+        // data_offset 0x30, reserved 0, packed, then the remaining fields.
+        v.extend_from_slice(&0x30u16.to_le_bytes());
+        v.extend_from_slice(&0u16.to_le_bytes());
+        v.extend_from_slice(&0x0501u16.to_le_bytes());
+        v.extend_from_slice(&0x1234u16.to_le_bytes());
+        v.extend_from_slice(&0x5678u16.to_le_bytes());
+        v.extend_from_slice(&0x9abcu16.to_le_bytes());
+        v.extend_from_slice(&0x0442u16.to_le_bytes());
+        v.extend_from_slice(&0u16.to_le_bytes());
+        assert_eq!(v.len(), TIE_BLOCK_HEADER_SIZE);
+        // Three records, 0x30 bytes, then the payload.
+        for k in 0..3u32 {
+            v.extend_from_slice(&(0x40 + k * 0x20).to_le_bytes());
+            for b in 0..12u8 {
+                v.push(b + k as u8);
+            }
+        }
+        v.extend_from_slice(&[0xa5u8; 8]);
+        v
+    }
+
+    #[test]
+    fn the_data_offset_bounds_the_record_table() {
+        let b = block();
+        let h = TieBlockHeader::parse(&b).unwrap();
+        assert!(h.invariants_hold());
+        assert_eq!(h.record_count(), 3);
+        let records: Vec<_> = h.records(&b).collect();
+        assert_eq!(records.len(), 3);
+        let first = TieRecord::parse(records[0]).unwrap();
+        assert_eq!(first.pointer, 0x40);
+        assert_eq!(first.fields, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]);
+    }
+
+    #[test]
+    fn a_table_running_past_the_block_yields_nothing() {
+        let mut b = block();
+        // Claim a table far longer than the block.
+        b[0..2].copy_from_slice(&0x400u16.to_le_bytes());
+        let h = TieBlockHeader::parse(&b).unwrap();
+        assert_eq!(h.record_count(), 0x40);
+        assert_eq!(h.records(&b).count(), 0, "must not slice past the block");
+        // A 0x400 offset is still a multiple of 16, so the invariant holds even
+        // though the table cannot exist.
+        assert!(h.invariants_hold());
+    }
 }

docs/formats/core-data-regions.md

@@ -1194,6 +1194,41 @@ guess; that it is two fields is not.
 Combined with block sizes always being multiples of 16, that rules out a uniform
 stride without a header.
 
+## tie: the block is a table of 16-byte records (from RAM)
+
+**Range measured:** every tie LOD block of level 1 whose leading word is a
+non-zero multiple of 16 -- 16 blocks -- plus the two read by hand in full.
+
+**Method:** locate each block's record table in PCSX2's RAM by its twelve field
+bytes (the records are the only part of a block with a distinctive 12-byte
+run), then compare every record against its disc copy. Each block has several
+copies in RAM; the one that matches is the loaded copy.
+
+**Result: on 16 of 16 blocks, every record's leading word has become
+`table_address + word`, and the other twelve bytes are byte-identical to the
+disc.** So:
+
+- A block is `[16-byte header][record table][vertex payload]`, and the records
+  are **16 bytes**: a leading `u32` pointer and twelve bytes of fields.
+- The header's word at `+0x00` is the **byte offset from the block start to
+  the vertex payload** -- that is, the length of the record table. This is what
+  the "count" reading was groping at: it is a multiple of 16 on all 2,177 blocks
+  because the table holds 16-byte records, it is never zero because there is
+  always at least the header-sized first record, and its 42 distinct values are
+  table lengths, not counts of anything in the payload.
+- Every record's leading word is a pointer **relative to the start of the record
+  table**, relocated at load to `table + word`. The twelve field bytes are
+  copied verbatim.
+
+This also explains the dead end that ran before it. The payload length divided
+by the word at `+0x00` is a whole number on **0 of 922** blocks, at any record
+size and any header up to 512 bytes -- which is the correct measurement of the
+wrong hypothesis: the word never described the payload, it described where the
+payload starts. A count of what follows a pointer is not the pointer.
+
+`TieBlockHeader::data_offset` and `TieBlockHeader::records` carry it, bound by
+the offset, and the disc test asserts the table is bounded on every block.
+
 ### What the payload is, and what retired the coordinate reading
 
 The payload is **not random**, and that took a control to establish. Reading each

docs/roadmap.md

@@ -275,6 +275,12 @@ Phase 1 hands over the extents; this phase works out what each one contains.
       three floats in the billboard record, `size`, `mode_bits`, `count`, and
       **where the mesh geometry lives** — R&C1 puts the mesh packets in this
       blob and R&C2's blob holds only the palette.
+- [x] **The tie LOD block is a table of 16-byte records.** The header's word at
+      `+0x00` is the byte offset from the block start to the vertex payload --
+      the record table's length -- not a count, which is why the payload never
+      divides by it. Each record is a leading `u32` pointer **relative to the
+      table start** plus twelve verbatim field bytes, relocated at load to
+      `table + word`. Verified in PCSX2's RAM on 16 of 16 level-1 blocks.
 - [x] **The shrub mesh geometry is in the class blob.** A packet table at
       `+0x40` names `count` packets between the table and the palette, and
       they tile on **358 of 358 spans** (2,239 packets), which makes `count`