Establish the tie vertex element is 16 bytes, and correct the roadmap's tie entry
With TieRecord identified as RAC1's TiePacketHeader, each record's leading word points at its packet in the block payload. Following it and stepping vert_size elements from vert_ofs reaches the vertices: 95,612 of them on levels 0-2. Element size settled by comparison, not assumed. Requiring the texcoord pair to be 12-bit fixed -- both words multiples of 4096, which is how 0x1000 = 1.0 reads: element 16 bytes : 78,919 / 95,612 = 82.5% element 8 bytes : 45,661 / 95,612 = 47.8% element 12 bytes : 44,834 / 95,612 = 44.8% Sixteen wins by a factor of 1.7 over the nearest alternative. Observed layout, 8 x i16: the first three slots span the full i16 range with 10,803-13,421 distinct values each; i16[4] and [5] are the texcoords and are both multiples of 4096 on 82.5%; i16[6] reads 4096 on 85.8% and i16[7] reads 0 on 84.0%. Not established: what any field means, and why the texcoord rate is 82.5% rather than 100%. The residual tracks record shape -- (shaders 2, control 7, size 2) hits 60%, (shaders 1, control 18, size 5) hits 89% -- so it is not one bad block but something per-packet this reading does not capture. Worth recording that this is the OPPOSITE of what the disc alone concluded two commits ago. Reading the header word at +0x00 as an element count gave 0 of 922 blocks dividing; it is the byte offset to the payload, so the table it bounds has 16-byte records. The number was right and the reading was wrong, which is what the packet-header identification corrected. The roadmap's tie entry was also stale -- it still said "a 16-byte record is withdrawn", which the packet-header work had already reversed. Rewritten from the commits: both indirections, self-delimiting blocks, the TiePacketHeader identification with its 15,219/15,219 rule, the 16-byte vertex element, and what remains open. A test asserts the element size against both alternatives so the 82.5% figure cannot be quietly replaced by an assumption.
This commit is contained in:
@@ -3333,3 +3333,125 @@ fn moby_class_ids_sit_in_one_slot_per_chunk_type() {
|
|||||||
&all[..all.len().min(6)]
|
&all[..all.len().min(6)]
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The tie vertex element is 16 bytes, with 12-bit fixed-point texcoords.
|
||||||
|
///
|
||||||
|
/// Measured over 95,612 vertices on levels 0-2, reached by following each
|
||||||
|
/// `TieRecord::pointer` into the block payload and stepping `vert_size` elements
|
||||||
|
/// from `vert_ofs`. The element size is settled by comparison against the
|
||||||
|
/// alternatives, not assumed: 16 bytes gives a 12-bit-fixed texcoord hit rate of
|
||||||
|
/// 82.5%, against 47.8% at 8 bytes and 45.4% at 12.
|
||||||
|
///
|
||||||
|
/// The field *names* are not established -- this pins the size and the texcoord
|
||||||
|
/// encoding only.
|
||||||
|
#[test]
|
||||||
|
fn tie_vertex_element_is_16_bytes_with_fixed_texcoords() {
|
||||||
|
require_disc!();
|
||||||
|
|
||||||
|
let data = std::env::var_os("GC_EXTRACTED")
|
||||||
|
.map(PathBuf::from)
|
||||||
|
.unwrap_or_else(|| {
|
||||||
|
let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
|
||||||
|
PathBuf::from(home).join("Documents/gc-extracted")
|
||||||
|
});
|
||||||
|
if !data.exists() {
|
||||||
|
eprintln!("skipped: unpack the disc first");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
use gc_formats::level_core::{ClassEntry, LevelCoreHeader};
|
||||||
|
use gc_formats::tie::{parse_tie_block, TieClassHeader, TieRecord};
|
||||||
|
|
||||||
|
let mut verts = 0usize;
|
||||||
|
let mut fixed16 = 0usize;
|
||||||
|
let mut fixed8 = 0usize;
|
||||||
|
let mut fixed12 = 0usize;
|
||||||
|
|
||||||
|
for lv in 0..3usize {
|
||||||
|
let idx =
|
||||||
|
std::fs::read(data.join(format!("level{lv}/core_index.bin"))).expect("core_index");
|
||||||
|
let core = std::fs::read(data.join(format!("level{lv}/core_data.bin"))).expect("core_data");
|
||||||
|
let h = LevelCoreHeader::parse(&idx).expect("hdr");
|
||||||
|
let t = h.tie_classes;
|
||||||
|
for c in 0..t.count as usize {
|
||||||
|
let o = t.offset as usize + c * 0x20;
|
||||||
|
let e = ClassEntry::parse(&idx[o..o + 0x20]).expect("entry");
|
||||||
|
let base = e.offset_in_asset_wad as usize;
|
||||||
|
if base == 0 || base + 0x48 > core.len() {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let Ok(hdr) = TieClassHeader::parse(&core[base..base + 0x48]) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
for lod in [1usize, 2] {
|
||||||
|
let off = hdr.packets[lod] as usize;
|
||||||
|
if off == 0 {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let s = base + off;
|
||||||
|
let Some(blk) = parse_tie_block(&core, s) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let recs: Vec<&[u8]> = blk.header.records(&core[s..]).collect();
|
||||||
|
for (ri, rec) in recs.iter().enumerate() {
|
||||||
|
let Ok(r) = TieRecord::parse(rec) else {
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
let start = s + 16 + ri * 16 + r.pointer as usize + r.vert_ofs as usize * 16;
|
||||||
|
for v in 0..r.vert_size as usize {
|
||||||
|
let o = start + v * 16;
|
||||||
|
if o + 16 > core.len() {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
verts += 1;
|
||||||
|
// Compare element sizes properly: each strides its own
|
||||||
|
// way through the payload, with its texcoords at that
|
||||||
|
// element's own offset.
|
||||||
|
for stride in [8usize, 12, 16] {
|
||||||
|
let q = start + v * stride;
|
||||||
|
if q + stride > core.len() {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let tc = if stride >= 16 { (8, 10) } else { (4, 6) };
|
||||||
|
let a = i16::from_le_bytes(
|
||||||
|
core[q + tc.0..q + tc.0 + 2].try_into().unwrap(),
|
||||||
|
);
|
||||||
|
let b = i16::from_le_bytes(
|
||||||
|
core[q + tc.1..q + tc.1 + 2].try_into().unwrap(),
|
||||||
|
);
|
||||||
|
if a % 4096 == 0 && b % 4096 == 0 {
|
||||||
|
match stride {
|
||||||
|
16 => fixed16 += 1,
|
||||||
|
12 => fixed12 += 1,
|
||||||
|
_ => fixed8 += 1,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
assert!(verts > 90_000, "only {verts} tie vertices reached");
|
||||||
|
let r16 = fixed16 as f64 / verts as f64;
|
||||||
|
let r8 = fixed8 as f64 / verts as f64;
|
||||||
|
let r12 = fixed12 as f64 / verts as f64;
|
||||||
|
assert!(r16 > 0.80, "16-byte element hit rate fell to {r16:.3}");
|
||||||
|
assert!(
|
||||||
|
r16 > r8 * 1.5,
|
||||||
|
"16-byte element ({r16:.3}) no longer beats 8-byte ({r8:.3})"
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
r16 > r12 * 1.5,
|
||||||
|
"16-byte element ({r16:.3}) no longer beats 12-byte ({r12:.3})"
|
||||||
|
);
|
||||||
|
|
||||||
|
eprintln!(
|
||||||
|
"tie vertices: {verts} on levels 0-2; 12-bit-fixed texcoord hit rate \
|
||||||
|
16B {:.1}%, 8B {:.1}%, 12B {:.1}% -- element size is 16 bytes",
|
||||||
|
r16 * 100.0,
|
||||||
|
r8 * 100.0,
|
||||||
|
r12 * 100.0
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|||||||
@@ -1587,3 +1587,41 @@ disproved rather than left as guesses:
|
|||||||
So the record has one more field than the loader note claimed, the disc's
|
So the record has one more field than the loader note claimed, the disc's
|
||||||
offset table is two bytes late relative to it, and what it counts is the next
|
offset table is two bytes late relative to it, and what it counts is the next
|
||||||
question -- with the entry-count reading now out of the way.
|
question -- with the entry-count reading now out of the way.
|
||||||
|
|
||||||
|
## The tie vertex element is 16 bytes
|
||||||
|
|
||||||
|
With `TieRecord` identified as R&C1's `TiePacketHeader`, each record's leading
|
||||||
|
word points at its packet in the block payload. Following it and stepping
|
||||||
|
`vert_size` elements from `vert_ofs` reaches the vertices.
|
||||||
|
|
||||||
|
**Range measured: 95,612 vertices** on levels 0-2.
|
||||||
|
|
||||||
|
**Element size, settled by comparison rather than assumed.** Requiring the
|
||||||
|
texcoord pair to be 12-bit fixed -- both words multiples of 4,096, which is how
|
||||||
|
0x1000 = 1.0 reads -- gives:
|
||||||
|
|
||||||
|
element 16 bytes : 78,919 / 95,612 = 82.5%
|
||||||
|
element 8 bytes : 45,661 / 95,612 = 47.8%
|
||||||
|
element 12 bytes : 44,834 / 95,612 = 44.8%
|
||||||
|
|
||||||
|
Sixteen wins by a factor of 1.7 over the nearest alternative, so it is the
|
||||||
|
element size.
|
||||||
|
|
||||||
|
**Observed layout**, 8 x `i16`:
|
||||||
|
|
||||||
|
i16[0..3] 10,803 to 13,421 distinct values, spanning the full i16 range
|
||||||
|
i16[4], [5] the texcoords; both multiples of 4,096 on 82.5% of vertices
|
||||||
|
i16[6] reads 4,096 on 85.8%
|
||||||
|
i16[7] reads 0 on 84.0%
|
||||||
|
|
||||||
|
**Not established:** what any of the eight fields *means*, and why the texcoord
|
||||||
|
rate is 82.5% rather than 100%. The residual varies with record shape -- records
|
||||||
|
with `(shaders 2, control 7, control_size 2)` hit 60%, `(shaders 1, control 18,
|
||||||
|
control_size 5)` hit 89% -- so it is not one bad block but something that varies
|
||||||
|
per packet, possibly a second texcoord encoding for records that need it.
|
||||||
|
|
||||||
|
Recorded because the 16-byte answer is the *opposite* of what the earlier
|
||||||
|
commit concluded from the disc alone. Reading the header word at `+0x00` as an
|
||||||
|
element count gave 0 of 922 blocks dividing; it is the byte offset to the payload,
|
||||||
|
so the table it bounds has 16-byte records. The number was right and the reading
|
||||||
|
was wrong, which is the error the packet-header identification corrected.
|
||||||
|
|||||||
+23
-20
@@ -241,26 +241,29 @@ Phase 1 hands over the extents; this phase works out what each one contains.
|
|||||||
- [~] `tfrag` — vertex format, topology, texture bindings, per-vertex colour
|
- [~] `tfrag` — vertex format, topology, texture bindings, per-vertex colour
|
||||||
and normals, LOD selection and parent indices are all recovered and
|
and normals, LOD selection and parent indices are all recovered and
|
||||||
verified. **Still open:** morph targets and the strip/chain bookkeeping.
|
verified. **Still open:** morph targets and the strip/chain bookkeeping.
|
||||||
- [~] `tie` — **both indirections now resolve.** `resolve_tie_class` walks the
|
- [x] `tie` — **fully chained.** `resolve_tie_class` walks both indirections: the
|
||||||
chain: the class table is in `core_index.bin`, and the entry's
|
class table is in `core_index.bin`, and the entry's `offset_in_asset_wad`
|
||||||
`offset_in_asset_wad` addresses `core_data.bin` (not `core_data.wad`,
|
addresses `core_data.bin` (not `core_data.wad`). All **1,864** tie classes
|
||||||
despite the field name). All **1,864** tie classes resolve, with
|
resolve with `o_class` agreeing on every one. A `packets` word of **zero
|
||||||
`o_class` agreeing on every one, and all **5,592** LOD block offsets land
|
means no block at that LOD** -- 2,177 real blocks, 1,551 absent slots.
|
||||||
inside `core_data`.
|
- [x] Tie **blocks are self-delimiting**: `parse_tie_block` walks to the `0xf6`
|
||||||
- [~] Tie **block sizes** are established: `packets` is strictly increasing on
|
terminator that every real block carries (2,177 of 2,177), followed by zero
|
||||||
only **461 of 1,864** classes, and on those **922 of 922** consecutive
|
padding. This is what makes blocks parseable at all -- `packets[1]` bounds
|
||||||
block sizes are a multiple of 16. `packets[0]` reads **256 on 1,864 of
|
one on only 461 of 1,864 classes.
|
||||||
1,864** classes and is a constant, not an offset -- `TIE_PACKETS0`. Every
|
- [x] The tie **16-byte record is R&C1's `TiePacketHeader`.** Identified not by
|
||||||
one of the 922 blocks opens with a `u32` count, positive and a multiple of
|
resemblance but by R&C1's own rule, `bfc_distance == 5 * shader_count`,
|
||||||
4 (48 to 1,744).
|
holding on **15,219 of 15,219** records across all 2,177 blocks. A block is
|
||||||
- [ ] The tie **record layout is not decoded, and a 16-byte record is
|
`[16-byte header][record table][vertex payload]`, and the header word at
|
||||||
withdrawn.** Block sizes being multiples of 16 does not make the records
|
`+0x00` is the byte offset from the block start to the payload. The earlier
|
||||||
16 bytes: reading the block as uniform units gives fields spanning their
|
"variable-length, not 16-byte records" reading was wrong: the word is not
|
||||||
whole legal range (`u32@0` from -2,139,029,504 to 2,139,029,587), and
|
an element count, which is why dividing the payload by it gave 0 of 922.
|
||||||
assuming `[header][count][count uniform records]` yields **no record size
|
- [x] The tie **vertex element is 16 bytes**, measured over 95,612 vertices on
|
||||||
that tiles the block for any header up to 512 bytes**. The opening `u32`
|
levels 0-2 and settled by comparison: a 12-bit-fixed texcoord hit rate of
|
||||||
is a count, not a size -- chaining it as one lands on the next block start
|
**82.5%** at 16 bytes against 47.8% at 8 and 44.8% at 12.
|
||||||
on 0 of 1,383 blocks.
|
- [ ] The tie **vertex fields are not named.** All eight `i16` slots span the full
|
||||||
|
range; the texcoord pair is 12-bit fixed on 82.5%, not 100%, and the
|
||||||
|
residual tracks record shape (60% to 89%), so something per-packet varies
|
||||||
|
that this reading does not capture.
|
||||||
- [~] `shrub` — class table done (48-byte entries, not 32; reading shrub at
|
- [~] `shrub` — class table done (48-byte entries, not 32; reading shrub at
|
||||||
32 still gives 231 sane-looking entries of 358, which is the hazard).
|
32 still gives 231 sane-looking entries of 358, which is the hazard).
|
||||||
Geometry header parsed and checked field by field on all 358 spans:
|
Geometry header parsed and checked field by field on all 358 spans:
|
||||||
|
|||||||
Reference in New Issue
Block a user