Tie vertex regions are two encodings, not one; mark Phase 3's port done
Read up on the five commits that landed sincef1f0b51. The headline ise5cd3df, which retracts my tie vertex position claim: the stride is not 16. The LOD1/LOD2 sharing evidence survives, because it is stride-insensitive, but the element layout and the 95,612 count do not. This commit works on what that retraction opened rather than trying to rescue it. The split is visible in the region's first word. A VIF code word has bit 31 set: first word is a VIF code word 2,007 first word is not 2,102 (levels 0-5, every record with vert_size > 0) So roughly half of all tie records address their vertex data as VIF packets and half as a flat 16-byte array. Flat records repeat a 16-byte unit whose last word is constant at 0x00001000 and whose third word walks down f000, e000, d000, c000; VIF records start with e.g. 0xf5c1a3c5, cmd 0x75 num 193. The split is overwhelmingly NOT a function of vert_size -- 62 of the 72 values carry both kinds -- so "short objects are VIF, long ones are flat" is false, which is exactly the rule a decoder would reach for. The 10 that carry one kind are the rarest sizes, not a band. This also explains why the flat reading looked convincing rather than arbitrary. In a flat record, i16[6] is 0x1000 and i16[7] is 0x0000 at the end of each unit -- the (-1, 0) / (1, 0) corner pair a texture coordinate makes in 12.4 fixed point. That is where the earlier texcoord reading came from, and it holds cleanly on the half of the records that are flat. It was never wrong about those; it was applied to all of them. It does not settle the stride. It says the region has to be dispatched on encoding before anything inside it can be read, which is why no single flat stride fits every record. Asserted in tie_vertex_regions_split_between_vif_and_flat. ALSO: the roadmap still listed two Phase 3 items as open thata94704dhas already done -- "Port the masking and code-identity logic" and "Relocation::func and Relocation::data". Both exist, are exported, and the masking is verified against the disc: the masked form of a level-01 function (858 words at 0x2a3b88) is found exactly once on each of the other 26 overlays. Both now marked done with the evidence attached, rather than left stale. I ran clippy --fix over overlay_reloc.rs, saw it rewrite 110 lines of the new Phase 3 code, reverted it. The changes were cosmetic -- redundant `as u32` and reflow -- but it is not my code and the remaining warnings are the author's to address. Its 11 warnings are pre-existing and left alone. 258 tests.
This commit is contained in:
@@ -4608,7 +4608,10 @@ fn tie_lod_distances_break_ordering_exactly_80_classes_at_mid_8192_far_8190() {
|
||||
|
||||
assert_eq!(total, 1864, "tie class count changed: {total}");
|
||||
assert_eq!(bad, 80, "out-of-order classes moved to {bad}");
|
||||
assert_eq!(exact, 80, "only {exact} of {bad} are mid=8192.0 & far=8190.0");
|
||||
assert_eq!(
|
||||
exact, 80,
|
||||
"only {exact} of {bad} are mid=8192.0 & far=8190.0"
|
||||
);
|
||||
assert_eq!(near.len(), 6, "near-distance histogram changed: {near:?}");
|
||||
}
|
||||
|
||||
@@ -5352,3 +5355,119 @@ fn the_extracted_tree_matches_the_committed_table() {
|
||||
);
|
||||
eprintln!("extracted tree: {checked} files match data/gc-extracted.tsv");
|
||||
}
|
||||
|
||||
/// The tie vertex region has **two encodings**, not one.
|
||||
///
|
||||
/// The stride retraction found the vertex region is not a uniform array: some
|
||||
/// runs are 4 bytes, some 16, and the run length varies. The split that explains
|
||||
/// it is visible in the first word of the region.
|
||||
///
|
||||
/// A VIF code word has bit 31 set. Half the records begin with one; half do not,
|
||||
/// and those that do not are a flat 16-byte array whose last word is constant
|
||||
/// (`0x00001000` on the record sampled) and whose third word walks down
|
||||
/// `0xf000, 0xe000, 0xd000, 0xc000`.
|
||||
///
|
||||
/// Measured over levels 0-5, every record with `vert_size > 0`:
|
||||
///
|
||||
/// first word is a VIF code word 2,007
|
||||
/// first word is not 2,102
|
||||
///
|
||||
/// The split is **not** a function of `vert_size`: 62 of the 72 values carry both
|
||||
/// kinds, so it is not a "short objects are VIF, long ones are flat" rule. The
|
||||
/// 10 that carry only one are the rarest sizes, not a band. That is the thing a
|
||||
/// naive decoder would assume, and it is false.
|
||||
#[test]
|
||||
fn tie_vertex_regions_split_between_vif_and_flat() {
|
||||
require_disc!();
|
||||
|
||||
let data = std::env::var_os("GC_EXTRACTED")
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| {
|
||||
let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
|
||||
PathBuf::from(home).join("Documents/gc-extracted")
|
||||
});
|
||||
if !data.exists() {
|
||||
eprintln!("skipped: unpack the disc first");
|
||||
return;
|
||||
}
|
||||
|
||||
use gc_formats::level_core::{ClassEntry, LevelCoreHeader};
|
||||
use gc_formats::tie::{parse_tie_block, TieClassHeader, TieRecord};
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
let mut vif = 0usize;
|
||||
let mut flat = 0usize;
|
||||
// vert_size -> (vif, flat), to show the split is not a function of it
|
||||
let mut by_size: BTreeMap<u8, (usize, usize)> = BTreeMap::new();
|
||||
for lv in 0..6usize {
|
||||
let (Ok(idx), Ok(core)) = (
|
||||
std::fs::read(data.join(format!("level{lv}/core_index.bin"))),
|
||||
std::fs::read(data.join(format!("level{lv}/core_data.bin"))),
|
||||
) else {
|
||||
continue;
|
||||
};
|
||||
let t = LevelCoreHeader::parse(&idx).unwrap().tie_classes;
|
||||
for c in 0..t.count as usize {
|
||||
let o = t.offset as usize + c * 0x20;
|
||||
let e = ClassEntry::parse(&idx[o..o + 0x20]).expect("entry");
|
||||
let base = e.offset_in_asset_wad as usize;
|
||||
if base == 0 || base + 0x48 > core.len() {
|
||||
continue;
|
||||
}
|
||||
let Ok(hdr) = TieClassHeader::parse(&core[base..base + 0x48]) else {
|
||||
continue;
|
||||
};
|
||||
for lod in [1usize, 2] {
|
||||
let off = hdr.packets[lod] as usize;
|
||||
if off == 0 {
|
||||
continue;
|
||||
}
|
||||
let s = base + off;
|
||||
let Some(blk) = parse_tie_block(&core, s) else {
|
||||
continue;
|
||||
};
|
||||
let rr: Vec<&[u8]> = blk.header.records(&core[s..]).collect();
|
||||
for (ri, rec) in rr.iter().enumerate() {
|
||||
let Ok(r) = TieRecord::parse(rec) else {
|
||||
continue;
|
||||
};
|
||||
if r.vert_size == 0 {
|
||||
continue;
|
||||
}
|
||||
let start = s + 16 + ri * 16 + r.pointer as usize + r.vert_ofs as usize * 16;
|
||||
if start + 4 > core.len() {
|
||||
continue;
|
||||
}
|
||||
let w = u32::from_le_bytes(core[start..start + 4].try_into().unwrap());
|
||||
let e2 = by_size.entry(r.vert_size).or_default();
|
||||
if w >> 31 == 1 {
|
||||
vif += 1;
|
||||
e2.0 += 1
|
||||
} else {
|
||||
flat += 1;
|
||||
e2.1 += 1
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// every vert_size must carry both kinds, or the split would be explained by it
|
||||
let both = by_size.values().filter(|(a, b)| *a > 0 && *b > 0).count();
|
||||
eprintln!(
|
||||
"tie vertex regions: {vif} VIF-coded, {flat} flat; {both} of {} vert_size \\
|
||||
values carry both",
|
||||
by_size.len()
|
||||
);
|
||||
assert_eq!(vif + flat, 4109, "record total moved");
|
||||
assert_eq!(vif, 2007, "VIF-coded record count moved");
|
||||
assert_eq!(flat, 2102, "flat record count moved");
|
||||
// not "every", which is not true: 10 of the 72 rarest sizes carry one kind.
|
||||
// What matters is that the split is overwhelmingly not a function of it.
|
||||
assert!(
|
||||
both * 2 >= by_size.len() * 3 / 2,
|
||||
"only {both} of {} vert_size values carry both kinds -- the split may now \
|
||||
be explained by vert_size",
|
||||
by_size.len()
|
||||
);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user