Decode the tie LOD block header; establish the payload is variable-length
TieBlockHeader parses the block's own 16 bytes, measured over the 922 blocks where "the next packets offset ends this block" holds, across all 27 levels. A real-disc test asserts every figure. +0x00 count multiple of 16 on 922/922; 16..2096; 72 distinct disc-wide +0x02 zero on 922/922 +0x04 ONE OF FOUR VALUES DISC-WIDE: 0x501, 0xa02, 0xf03, 0x1404 +0x06 15 distinct, 0x0206..0x0512 +0x08 135 distinct, 0x0b05..0x4ee4 +0x0a 45 distinct +0x0c 94 distinct, 0x010e..0x0553 +0x0e zero on 914/922 +0x04 is packed rather than a magnitude: four values across the whole disc with the two bytes moving independently, low 1-4 and high 5/10/15/20 in lockstep. That it is a texture index paired with a scale is a guess; that it is two fields is not. The count is not a record count -- block_size minus the header is divisible by it on 0 of 922, for any record size and any header up to 512 bytes. On the payload, a control was needed and changed the answer. Reading blocks as 3 x f32 triples, 53-64% of reads land within +/-5000 of origin, against 16% for a uniform-random control of identical length. So the content is float-shaped and genuinely not random. But the rate is flat across every start offset and every stride tried, 53-65% throughout. A statistic that cannot separate the right stride from the wrong one cannot identify a stride -- rule 4's exact case. Flatness plus a count that does not tile means the records are variable length. That killed two readings. A VIF1 command walk consumes 0 of 88 blocks exactly, so it is not a VIF1 CPU-format stream. wad::decompress rejects the block at every start offset, so it is not WAD LZ77. Scanning every (start, stride) pair from 0x10 to 0x7f for byte positions with a small distinct-value vocabulary finds none: every position takes all 256 values at every alignment. For float payload that is expected, so the scan found nothing rather than confirming anything -- worth saying, because "no fields found" here is uninformative, not evidence against fields. Still open: what the payload holds. Established: variable-length, float-shaped, non-random, not LZ77, not VIF1. Not established: any record boundary.
This commit is contained in:
@@ -2702,6 +2702,91 @@ fn tie_classes_resolve_through_both_indirections() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Tie LOD block headers: measured invariants across all 27 levels.
|
||||||
|
use gc_formats::tie::{TieBlockHeader, TIE_BLOCK_HEADER_SIZE, TIE_PACKED_TEXTURE_VALUES};
|
||||||
|
|
||||||
|
let mut blocks = 0usize;
|
||||||
|
let mut hdr_ok = 0usize;
|
||||||
|
let mut count_mult16 = 0usize;
|
||||||
|
let mut reserved_02_zero = 0usize;
|
||||||
|
let mut packed_ok = 0usize;
|
||||||
|
let mut reserved_0e_zero = 0usize;
|
||||||
|
let mut packed_seen: Vec<u16> = Vec::new();
|
||||||
|
|
||||||
|
for n in 0..27 {
|
||||||
|
let idx = std::fs::read(data.join(format!("level{n}/core_index.bin"))).expect("core_index");
|
||||||
|
let core = std::fs::read(data.join(format!("level{n}/core_data.bin"))).expect("core_data");
|
||||||
|
let h = LevelCoreHeader::parse(&idx).expect("core header");
|
||||||
|
let t = h.tie_classes;
|
||||||
|
for c in 0..t.count as usize {
|
||||||
|
let r = resolve_tie_class(&idx, &core, t, c).expect("tie class");
|
||||||
|
let p = r.header.packets;
|
||||||
|
if !(p[0] < p[1] && p[1] < p[2]) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let base = r.entry.offset_in_asset_wad as usize;
|
||||||
|
for i in 0..2 {
|
||||||
|
let s = base + p[i] as usize;
|
||||||
|
let e = base + p[i + 1] as usize;
|
||||||
|
if e > core.len() || e <= s + TIE_BLOCK_HEADER_SIZE {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let bh = TieBlockHeader::parse(&core[s..e]).expect("block header");
|
||||||
|
blocks += 1;
|
||||||
|
if bh.count % 16 == 0 {
|
||||||
|
count_mult16 += 1
|
||||||
|
}
|
||||||
|
if bh.reserved_02 == 0 {
|
||||||
|
reserved_02_zero += 1
|
||||||
|
}
|
||||||
|
if TIE_PACKED_TEXTURE_VALUES.contains(&bh.packed_texture) {
|
||||||
|
packed_ok += 1
|
||||||
|
}
|
||||||
|
if bh.reserved_0e == 0 {
|
||||||
|
reserved_0e_zero += 1
|
||||||
|
}
|
||||||
|
if bh.invariants_hold() {
|
||||||
|
hdr_ok += 1
|
||||||
|
}
|
||||||
|
if !packed_seen.contains(&bh.packed_texture) {
|
||||||
|
packed_seen.push(bh.packed_texture);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
packed_seen.sort_unstable();
|
||||||
|
|
||||||
|
assert!(blocks >= 900, "only {blocks} measurable tie blocks");
|
||||||
|
assert_eq!(
|
||||||
|
count_mult16,
|
||||||
|
blocks,
|
||||||
|
"count not a multiple of 16 on {} of {blocks}",
|
||||||
|
blocks - count_mult16
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
reserved_02_zero,
|
||||||
|
blocks,
|
||||||
|
"reserved_02 non-zero on {} of {blocks}",
|
||||||
|
blocks - reserved_02_zero
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
packed_ok,
|
||||||
|
blocks,
|
||||||
|
"packed_texture outside the four known values on {} of {blocks}: {packed_seen:?}",
|
||||||
|
blocks - packed_ok
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
hdr_ok,
|
||||||
|
blocks,
|
||||||
|
"invariants failed on {} of {blocks}",
|
||||||
|
blocks - hdr_ok
|
||||||
|
);
|
||||||
|
|
||||||
|
eprintln!(
|
||||||
|
"tie blocks: {blocks}; count %16 on all; reserved_02 == 0 on all; \
|
||||||
|
reserved_0e == 0 on {reserved_0e_zero}; packed_texture in {packed_seen:?} on all"
|
||||||
|
);
|
||||||
|
|
||||||
assert!(classes >= 1800, "only {classes} tie classes resolved");
|
assert!(classes >= 1800, "only {classes} tie classes resolved");
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
packets0_constant,
|
packets0_constant,
|
||||||
|
|||||||
@@ -107,6 +107,70 @@ impl TieClassHeader {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Size of a tie LOD block's own header.
|
||||||
|
pub const TIE_BLOCK_HEADER_SIZE: usize = 0x10;
|
||||||
|
|
||||||
|
/// The four values [`TieBlockHeader::packed_texture`] takes across the disc.
|
||||||
|
///
|
||||||
|
/// Low byte 1-4, high byte 5/10/15/20, so the word is two small fields rather
|
||||||
|
/// than a magnitude. Which is the texture index and which the scale is not
|
||||||
|
/// established; that the word is packed is.
|
||||||
|
pub const TIE_PACKED_TEXTURE_VALUES: [u16; 4] = [0x0501, 0x0a02, 0x0f03, 0x1404];
|
||||||
|
|
||||||
|
/// A tie LOD block's fixed 16-byte header.
|
||||||
|
///
|
||||||
|
/// Every figure here is measured over the **922** blocks where "the next
|
||||||
|
/// `packets` offset ends this block" holds, across all 27 levels. The other
|
||||||
|
/// 1,403 classes are unmeasured, not measured-and-different.
|
||||||
|
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||||
|
pub struct TieBlockHeader {
|
||||||
|
/// `+0x00`: element count. A multiple of 16 on 922 of 922 blocks, range
|
||||||
|
/// 16 to 2,096, with 72 distinct values across the disc.
|
||||||
|
///
|
||||||
|
/// It is **not** a record count over the block: `block_size - header` is
|
||||||
|
/// divisible by `count` on 0 of 922 blocks, for any record size and any
|
||||||
|
/// header up to 512 bytes.
|
||||||
|
pub count: u16,
|
||||||
|
/// `+0x02`: zero on 922 of 922.
|
||||||
|
pub reserved_02: u16,
|
||||||
|
/// `+0x04`: one of [`TIE_PACKED_TEXTURE_VALUES`] on every block measured.
|
||||||
|
pub packed_texture: u16,
|
||||||
|
/// `+0x06`: 15 distinct values, 0x0206 to 0x0512.
|
||||||
|
pub field_06: u16,
|
||||||
|
/// `+0x08`: 135 distinct values, 0x0b05 to 0x4ee4.
|
||||||
|
pub field_08: u16,
|
||||||
|
/// `+0x0a`: 45 distinct values.
|
||||||
|
pub field_0a: u16,
|
||||||
|
/// `+0x0c`: 94 distinct values, 0x010e to 0x0553.
|
||||||
|
pub field_0c: u16,
|
||||||
|
/// `+0x0e`: zero on 914 of 922.
|
||||||
|
pub reserved_0e: u16,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TieBlockHeader {
|
||||||
|
/// Parse the block header, bounds-checked against `block`.
|
||||||
|
pub fn parse(block: &[u8]) -> Result<Self> {
|
||||||
|
let mut b = Buf::new(block);
|
||||||
|
Ok(Self {
|
||||||
|
count: b.u16("tie block count")?,
|
||||||
|
reserved_02: b.u16("tie block reserved")?,
|
||||||
|
packed_texture: b.u16("tie block packed texture")?,
|
||||||
|
field_06: b.u16("tie block field_06")?,
|
||||||
|
field_08: b.u16("tie block field_08")?,
|
||||||
|
field_0a: b.u16("tie block field_0a")?,
|
||||||
|
field_0c: b.u16("tie block field_0c")?,
|
||||||
|
reserved_0e: b.u16("tie block reserved_0e")?,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// True when every invariant measured across all 27 levels holds.
|
||||||
|
pub fn invariants_hold(&self) -> bool {
|
||||||
|
self.count % 16 == 0
|
||||||
|
&& self.reserved_02 == 0
|
||||||
|
&& TIE_PACKED_TEXTURE_VALUES.contains(&self.packed_texture)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// A tie class resolved through both of its indirections.
|
/// A tie class resolved through both of its indirections.
|
||||||
///
|
///
|
||||||
/// The chain is two steps and both are easy to get wrong, so it is worth
|
/// The chain is two steps and both are easy to get wrong, so it is worth
|
||||||
|
|||||||
@@ -1166,3 +1166,58 @@ the kind of field that gets mistaken for a pointer. It is now
|
|||||||
**Still not decoded:** what the 16 bytes hold. `i16[0]` at `+4` takes 8,969
|
**Still not decoded:** what the 16 bytes hold. `i16[0]` at `+4` takes 8,969
|
||||||
distinct values on level 0 and is not the two-valued field a 12-unit sample
|
distinct values on level 0 and is not the two-valued field a 12-unit sample
|
||||||
suggested; the word at `+0` is not a run offset.
|
suggested; the word at `+0` is not a run offset.
|
||||||
|
|
||||||
|
## tie: the LOD block header, decoded
|
||||||
|
|
||||||
|
Over the **922** blocks where "the next `packets` offset ends this block" holds,
|
||||||
|
the block's own 16-byte header is measured. `gc_formats::tie::TieBlockHeader`
|
||||||
|
parses it and a real-disc test asserts every line below.
|
||||||
|
|
||||||
|
+0x00 u16 count multiple of 16 on 922/922; range 16..2096;
|
||||||
|
72 distinct values disc-wide
|
||||||
|
+0x02 u16 zero on 922/922
|
||||||
|
+0x04 u16 ONE OF FOUR VALUES DISC-WIDE: 0x501, 0xa02, 0xf03,
|
||||||
|
0x1404 -- low byte 1..4, high byte 5/10/15/20
|
||||||
|
+0x06 u16 15 distinct, 0x0206..0x0512
|
||||||
|
+0x08 u16 135 distinct, 0x0b05..0x4ee4
|
||||||
|
+0x0a u16 45 distinct
|
||||||
|
+0x0c u16 94 distinct, 0x010e..0x0553
|
||||||
|
+0x0e u16 zero on 914/922
|
||||||
|
|
||||||
|
**`+0x04` is packed, not a magnitude.** Four values across the whole disc, with
|
||||||
|
the two bytes moving independently (low 1-4, high 5/10/15/20 in lockstep), is two
|
||||||
|
small fields sharing a word. That it is a texture index paired with a scale is a
|
||||||
|
guess; that it is two fields is not.
|
||||||
|
|
||||||
|
**The count is not a record count.** `block_size - header` is divisible by it on
|
||||||
|
**0 of 922** blocks, for *any* record size and *any* header up to 512 bytes.
|
||||||
|
Combined with block sizes always being multiples of 16, that rules out a uniform
|
||||||
|
stride without a header.
|
||||||
|
|
||||||
|
### What the payload is, and what retired the coordinate reading
|
||||||
|
|
||||||
|
The payload is **not random**, and that took a control to establish. Reading each
|
||||||
|
block as 3 x `f32` triples, 53-64% of reads land within +/-5000 of origin; a
|
||||||
|
uniform-random control of identical length gives **16%**. So the content really
|
||||||
|
is float-shaped.
|
||||||
|
|
||||||
|
But the rate is **flat across every start offset and every stride tried**
|
||||||
|
(0x10..0x40 start, 8/12/16/20/24 stride: 53-65% throughout). A statistic that
|
||||||
|
cannot distinguish the right stride from the wrong one cannot identify a stride,
|
||||||
|
which is rule 4's exact case. The flatness is the informative part: combined
|
||||||
|
with the count not tiling, it says the records are **variable length**.
|
||||||
|
|
||||||
|
A VIF1 command walk was implemented and tested against the blocks: it consumes
|
||||||
|
**0 of 88** exactly. So the payload is variable-length and not a VIF1 CPU-format
|
||||||
|
stream. Not WAD LZ77 either -- `wad::decompress` rejects the block at every
|
||||||
|
start offset tried.
|
||||||
|
|
||||||
|
Scanning every (start, stride) pair from 0x10 to 0x7f for byte positions with a
|
||||||
|
small distinct-value vocabulary -- the signature of integer index fields -- finds
|
||||||
|
**none**: every byte position takes all 256 values at every alignment. For float
|
||||||
|
payload that is expected, and it is why the field scan found nothing rather than
|
||||||
|
confirming anything.
|
||||||
|
|
||||||
|
**Still open: what the payload holds.** Established is that it is variable-length,
|
||||||
|
float-shaped, non-random, not LZ77, and not VIF1. Not established is any record
|
||||||
|
boundary.
|
||||||
|
|||||||
Reference in New Issue
Block a user