Decode the tie LOD block header; establish the payload is variable-length

TieBlockHeader parses the block's own 16 bytes, measured over the 922 blocks
where "the next packets offset ends this block" holds, across all 27 levels. A
real-disc test asserts every figure.

  +0x00  count        multiple of 16 on 922/922; 16..2096; 72 distinct disc-wide
  +0x02               zero on 922/922
  +0x04               ONE OF FOUR VALUES DISC-WIDE: 0x501, 0xa02, 0xf03, 0x1404
  +0x06               15 distinct, 0x0206..0x0512
  +0x08               135 distinct, 0x0b05..0x4ee4
  +0x0a               45 distinct
  +0x0c               94 distinct, 0x010e..0x0553
  +0x0e               zero on 914/922

+0x04 is packed rather than a magnitude: four values across the whole disc with
the two bytes moving independently, low 1-4 and high 5/10/15/20 in lockstep. That
it is a texture index paired with a scale is a guess; that it is two fields is
not.

The count is not a record count -- block_size minus the header is divisible by it
on 0 of 922, for any record size and any header up to 512 bytes.

On the payload, a control was needed and changed the answer. Reading blocks as
3 x f32 triples, 53-64% of reads land within +/-5000 of origin, against 16% for
a uniform-random control of identical length. So the content is float-shaped and
genuinely not random.

But the rate is flat across every start offset and every stride tried, 53-65%
throughout. A statistic that cannot separate the right stride from the wrong one
cannot identify a stride -- rule 4's exact case. Flatness plus a count that does
not tile means the records are variable length.

That killed two readings. A VIF1 command walk consumes 0 of 88 blocks exactly,
so it is not a VIF1 CPU-format stream. wad::decompress rejects the block at every
start offset, so it is not WAD LZ77.

Scanning every (start, stride) pair from 0x10 to 0x7f for byte positions with a
small distinct-value vocabulary finds none: every position takes all 256 values
at every alignment. For float payload that is expected, so the scan found nothing
rather than confirming anything -- worth saying, because "no fields found" here is
uninformative, not evidence against fields.

Still open: what the payload holds. Established: variable-length, float-shaped,
non-random, not LZ77, not VIF1. Not established: any record boundary.
This commit is contained in:
2026-10-12 04:09:08 +02:00
parent 881fd48e9a
commit 02342de31b
3 changed files with 204 additions and 0 deletions
+85
View File
@@ -2702,6 +2702,91 @@ fn tie_classes_resolve_through_both_indirections() {
} }
} }
// Tie LOD block headers: measured invariants across all 27 levels.
use gc_formats::tie::{TieBlockHeader, TIE_BLOCK_HEADER_SIZE, TIE_PACKED_TEXTURE_VALUES};
let mut blocks = 0usize;
let mut hdr_ok = 0usize;
let mut count_mult16 = 0usize;
let mut reserved_02_zero = 0usize;
let mut packed_ok = 0usize;
let mut reserved_0e_zero = 0usize;
let mut packed_seen: Vec<u16> = Vec::new();
for n in 0..27 {
let idx = std::fs::read(data.join(format!("level{n}/core_index.bin"))).expect("core_index");
let core = std::fs::read(data.join(format!("level{n}/core_data.bin"))).expect("core_data");
let h = LevelCoreHeader::parse(&idx).expect("core header");
let t = h.tie_classes;
for c in 0..t.count as usize {
let r = resolve_tie_class(&idx, &core, t, c).expect("tie class");
let p = r.header.packets;
if !(p[0] < p[1] && p[1] < p[2]) {
continue;
}
let base = r.entry.offset_in_asset_wad as usize;
for i in 0..2 {
let s = base + p[i] as usize;
let e = base + p[i + 1] as usize;
if e > core.len() || e <= s + TIE_BLOCK_HEADER_SIZE {
continue;
}
let bh = TieBlockHeader::parse(&core[s..e]).expect("block header");
blocks += 1;
if bh.count % 16 == 0 {
count_mult16 += 1
}
if bh.reserved_02 == 0 {
reserved_02_zero += 1
}
if TIE_PACKED_TEXTURE_VALUES.contains(&bh.packed_texture) {
packed_ok += 1
}
if bh.reserved_0e == 0 {
reserved_0e_zero += 1
}
if bh.invariants_hold() {
hdr_ok += 1
}
if !packed_seen.contains(&bh.packed_texture) {
packed_seen.push(bh.packed_texture);
}
}
}
}
packed_seen.sort_unstable();
assert!(blocks >= 900, "only {blocks} measurable tie blocks");
assert_eq!(
count_mult16,
blocks,
"count not a multiple of 16 on {} of {blocks}",
blocks - count_mult16
);
assert_eq!(
reserved_02_zero,
blocks,
"reserved_02 non-zero on {} of {blocks}",
blocks - reserved_02_zero
);
assert_eq!(
packed_ok,
blocks,
"packed_texture outside the four known values on {} of {blocks}: {packed_seen:?}",
blocks - packed_ok
);
assert_eq!(
hdr_ok,
blocks,
"invariants failed on {} of {blocks}",
blocks - hdr_ok
);
eprintln!(
"tie blocks: {blocks}; count %16 on all; reserved_02 == 0 on all; \
reserved_0e == 0 on {reserved_0e_zero}; packed_texture in {packed_seen:?} on all"
);
assert!(classes >= 1800, "only {classes} tie classes resolved"); assert!(classes >= 1800, "only {classes} tie classes resolved");
assert_eq!( assert_eq!(
packets0_constant, packets0_constant,
+64
View File
@@ -107,6 +107,70 @@ impl TieClassHeader {
} }
} }
/// Size of a tie LOD block's own header.
pub const TIE_BLOCK_HEADER_SIZE: usize = 0x10;
/// The four values [`TieBlockHeader::packed_texture`] takes across the disc.
///
/// Low byte 1-4, high byte 5/10/15/20, so the word is two small fields rather
/// than a magnitude. Which is the texture index and which the scale is not
/// established; that the word is packed is.
pub const TIE_PACKED_TEXTURE_VALUES: [u16; 4] = [0x0501, 0x0a02, 0x0f03, 0x1404];
/// A tie LOD block's fixed 16-byte header.
///
/// Every figure here is measured over the **922** blocks where "the next
/// `packets` offset ends this block" holds, across all 27 levels. The other
/// 1,403 classes are unmeasured, not measured-and-different.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct TieBlockHeader {
/// `+0x00`: element count. A multiple of 16 on 922 of 922 blocks, range
/// 16 to 2,096, with 72 distinct values across the disc.
///
/// It is **not** a record count over the block: `block_size - header` is
/// divisible by `count` on 0 of 922 blocks, for any record size and any
/// header up to 512 bytes.
pub count: u16,
/// `+0x02`: zero on 922 of 922.
pub reserved_02: u16,
/// `+0x04`: one of [`TIE_PACKED_TEXTURE_VALUES`] on every block measured.
pub packed_texture: u16,
/// `+0x06`: 15 distinct values, 0x0206 to 0x0512.
pub field_06: u16,
/// `+0x08`: 135 distinct values, 0x0b05 to 0x4ee4.
pub field_08: u16,
/// `+0x0a`: 45 distinct values.
pub field_0a: u16,
/// `+0x0c`: 94 distinct values, 0x010e to 0x0553.
pub field_0c: u16,
/// `+0x0e`: zero on 914 of 922.
pub reserved_0e: u16,
}
impl TieBlockHeader {
/// Parse the block header, bounds-checked against `block`.
pub fn parse(block: &[u8]) -> Result<Self> {
let mut b = Buf::new(block);
Ok(Self {
count: b.u16("tie block count")?,
reserved_02: b.u16("tie block reserved")?,
packed_texture: b.u16("tie block packed texture")?,
field_06: b.u16("tie block field_06")?,
field_08: b.u16("tie block field_08")?,
field_0a: b.u16("tie block field_0a")?,
field_0c: b.u16("tie block field_0c")?,
reserved_0e: b.u16("tie block reserved_0e")?,
})
}
/// True when every invariant measured across all 27 levels holds.
pub fn invariants_hold(&self) -> bool {
self.count % 16 == 0
&& self.reserved_02 == 0
&& TIE_PACKED_TEXTURE_VALUES.contains(&self.packed_texture)
}
}
/// A tie class resolved through both of its indirections. /// A tie class resolved through both of its indirections.
/// ///
/// The chain is two steps and both are easy to get wrong, so it is worth /// The chain is two steps and both are easy to get wrong, so it is worth
+55
View File
@@ -1166,3 +1166,58 @@ the kind of field that gets mistaken for a pointer. It is now
**Still not decoded:** what the 16 bytes hold. `i16[0]` at `+4` takes 8,969 **Still not decoded:** what the 16 bytes hold. `i16[0]` at `+4` takes 8,969
distinct values on level 0 and is not the two-valued field a 12-unit sample distinct values on level 0 and is not the two-valued field a 12-unit sample
suggested; the word at `+0` is not a run offset. suggested; the word at `+0` is not a run offset.
## tie: the LOD block header, decoded
Over the **922** blocks where "the next `packets` offset ends this block" holds,
the block's own 16-byte header is measured. `gc_formats::tie::TieBlockHeader`
parses it and a real-disc test asserts every line below.
+0x00 u16 count multiple of 16 on 922/922; range 16..2096;
72 distinct values disc-wide
+0x02 u16 zero on 922/922
+0x04 u16 ONE OF FOUR VALUES DISC-WIDE: 0x501, 0xa02, 0xf03,
0x1404 -- low byte 1..4, high byte 5/10/15/20
+0x06 u16 15 distinct, 0x0206..0x0512
+0x08 u16 135 distinct, 0x0b05..0x4ee4
+0x0a u16 45 distinct
+0x0c u16 94 distinct, 0x010e..0x0553
+0x0e u16 zero on 914/922
**`+0x04` is packed, not a magnitude.** Four values across the whole disc, with
the two bytes moving independently (low 1-4, high 5/10/15/20 in lockstep), is two
small fields sharing a word. That it is a texture index paired with a scale is a
guess; that it is two fields is not.
**The count is not a record count.** `block_size - header` is divisible by it on
**0 of 922** blocks, for *any* record size and *any* header up to 512 bytes.
Combined with block sizes always being multiples of 16, that rules out a uniform
stride without a header.
### What the payload is, and what retired the coordinate reading
The payload is **not random**, and that took a control to establish. Reading each
block as 3 x `f32` triples, 53-64% of reads land within +/-5000 of origin; a
uniform-random control of identical length gives **16%**. So the content really
is float-shaped.
But the rate is **flat across every start offset and every stride tried**
(0x10..0x40 start, 8/12/16/20/24 stride: 53-65% throughout). A statistic that
cannot distinguish the right stride from the wrong one cannot identify a stride,
which is rule 4's exact case. The flatness is the informative part: combined
with the count not tiling, it says the records are **variable length**.
A VIF1 command walk was implemented and tested against the blocks: it consumes
**0 of 88** exactly. So the payload is variable-length and not a VIF1 CPU-format
stream. Not WAD LZ77 either -- `wad::decompress` rejects the block at every
start offset tried.
Scanning every (start, stride) pair from 0x10 to 0x7f for byte positions with a
small distinct-value vocabulary -- the signature of integer index fields -- finds
**none**: every byte position takes all 256 values at every alignment. For float
payload that is expected, and it is why the field scan found nothing rather than
confirming anything.
**Still open: what the payload holds.** Established is that it is variable-length,
float-shaped, non-random, not LZ77, and not VIF1. Not established is any record
boundary.