The tfrag morph is not in the data block: positions are verbatim in RAM

The morph was recorded as unverifiable because "reading a weight back out of
RAM does not work -- the positions live inside regenerated VU packets". That
is right about the correspondence and wrong about the cause. The cause is
measurable, and it comes back clean:

    patches                         833
    positions                     40,600   disc == ram
    positions identical           40,600
    block bytes differing         166,465
      inside a position unpack             0
      inside another unpack           49,484
      in a VIF command word          116,981

Range: level 0, all 833 tfrag patches, live PINE RAM against the disc. Two
independent measurements agree. Walking the VIF lists on both copies and
comparing parsed positions gives 40,600 identical and zero changed; a raw
per-byte diff of each patch's own extent classifies every differing byte, and
none of them is in a position unpack. The position ranges total 490,224 bytes.

So the blend is not precomputed into the geometry. The rewrites are packet
framing -- 70% command words -- plus other unpack payloads, which is the loader
rebuilding the VIF packet for the tier it selected while leaving the vertices
alone. The morph happens at draw time in the VU stream.

The standing note becomes a sharper one: there was never a correspondence
problem. The values were identical, so there was nothing to read.

gc-trace gains verify-morph, which does both measurements. TfragGeometry
gains position_ranges and other_ranges so the walk needed to classify a diff
is already paid for, and the ranges come from the same walk the rest of the
parse uses rather than a second one that could drift.

Per-patch extents are the gap to the next patch's data pointer. That is exact
and needs no guess about which header field marks a patch's end; the first
attempt at this compared each patch against the whole rest of the region and
reported 65 million differing bytes, which is what an unbounded comparison
looks like.

tfrag_positions_are_verbatim_and_the_rewrites_are_elsewhere is gated on
GC_SSTATE and asserts the four bounds, plus two guards: the rewrites must be
non-trivial (over 100,000 bytes) and mostly command words, so a RAM image that
was a verbatim copy of the disc -- which would make the zero look good for the
wrong reason -- fails instead.

Suite with the disc and GC_SSTATE: 68 disc tests, 156 gc-formats, 22
gc-trace, 23 gc-engine, 19 gc-game. Clippy clean.
This commit is contained in:
2026-10-14 12:27:38 +02:00
parent 4d66cf6214
commit 3a1156d8e1
8 changed files with 698 additions and 8 deletions
+193
View File
@@ -4858,6 +4858,199 @@ fn ram_slice(ram: &[u8], addr: usize, len: usize) -> Option<&[u8]> {
ram.get(addr..addr.checked_add(len)?)
}
/// The tfrag morph is not a rewrite of the position arrays.
///
/// The game rewrites bytes inside every tfrag data block, and this test says
/// where those bytes are. Positions are byte-identical between disc and RAM on
/// every patch, so the blend cannot be precomputed into the geometry -- it
/// happens at draw time, in the VU stream, which is why reading a weight back
/// out of RAM never worked.
#[test]
fn tfrag_positions_are_verbatim_and_the_rewrites_are_elsewhere() {
let Some(ram) = ram_image() else {
eprintln!("skipped: set GC_SSTATE to a PCSX2 .p2s savestate");
return;
};
let Some(data) = extracted_root() else {
eprintln!("skipped: unpack the disc first");
return;
};
require_disc!();
use gc_formats::level_core::LevelCoreHeader;
use gc_formats::tfrag::{parse_tfrag, TfragHeader, TfragsBlock, TFRAG_HEADER_SIZE};
let idx = std::fs::read(data.join("level0/core_index.bin")).expect("core_index");
let core = std::fs::read(data.join("level0/core_data.bin")).expect("core_data");
let h = LevelCoreHeader::parse(&idx).expect("hdr");
let tfrags_ofs = h.tfrags as usize;
let region_end = [0x0c_usize, 0x10, 0x14]
.iter()
.filter_map(|&o| i32_at(&idx, o))
.filter(|&v| v as usize > tfrags_ofs)
.map(|v| v as usize)
.min()
.unwrap_or(core.len());
let region = &core[tfrags_ofs..region_end];
let block = TfragsBlock::parse(region).expect("tfrags block");
let table_off = block.table_offset as usize;
let count = block.tfrag_count as usize;
assert!(
count > 700,
"level0 should hold the full 833 patches, got {count}"
);
// Find the block in RAM the same way gc-trace does: the first record of the
// table, matched byte for byte, with the invariant mask applied.
let key = &region[table_off..table_off + 16];
let mut base = None;
for cand in occ(ram.as_ref(), key) {
if cand < table_off {
continue;
}
let b = cand - table_off;
let Some(hdr) = ram_slice(&ram, b, 0x10) else {
continue;
};
if hdr != &region[..0x10] {
continue;
}
let mut matched = 0usize;
for i in 0..count.min(64) {
let at = i * TFRAG_HEADER_SIZE;
let d = region.get(table_off + at..table_off + at + TFRAG_HEADER_SIZE);
let r = ram_slice(&ram, cand + at, TFRAG_HEADER_SIZE);
if let (Some(a), Some(b2)) = (d, r) {
if a.iter()
.zip(b2.iter())
.enumerate()
.filter(|&(j, _)| !(0x10..0x14).contains(&j) && !(0x3a..0x3c).contains(&j))
.all(|(_, (x, y))| x == y)
{
matched += 1;
}
}
}
if matched == count.min(64) {
base = Some(b);
break;
}
}
let base = base.expect("the tfrags block should be locatable in RAM");
let table = base + table_off;
// Patch extents from the gap to the next `data`, which is exact.
let mut starts: Vec<(usize, usize)> = (0..count)
.filter_map(|i| {
let v = i32_at(region, table_off + i * TFRAG_HEADER_SIZE + 0x10).filter(|&d| d > 0)?;
Some((table_off + v as usize, i))
})
.collect();
starts.sort_unstable();
starts.dedup_by_key(|&mut (v, _)| v);
let mut positions = 0usize;
let mut identical = 0usize;
let mut changed = 0usize;
let mut differing = 0usize;
let mut differing_in_positions = 0usize;
let mut differing_in_other_unpacks = 0usize;
let mut differing_in_command_words = 0usize;
let mut patches = 0usize;
for i in 0..count {
let at = i * TFRAG_HEADER_SIZE;
let disc_h =
TfragHeader::parse(&region[table_off + at..table_off + at + TFRAG_HEADER_SIZE])
.expect("disc header");
if disc_h.vert_count == 0 || disc_h.data == 0 {
continue;
}
let ram_h = TfragHeader::parse(ram_slice(&ram, table + at, TFRAG_HEADER_SIZE).unwrap())
.expect("ram header");
let Some(k) = starts.iter().position(|&(_, j)| j == i) else {
continue;
};
let start = starts[k].0;
let end = starts.get(k + 1).map(|&(v, _)| v).unwrap_or(region.len());
if end <= start {
continue;
}
let disc_block = &region[start..end];
let ram_data = ram_h.data as usize;
if ram_data < base || ram_data + (end - start) > ram.len() {
continue;
}
let ram_block = &ram[ram_data..ram_data + (end - start)];
let Ok(g) = parse_tfrag(disc_block, &disc_h) else {
continue;
};
let Ok(rg) = parse_tfrag(ram_block, &ram_h) else {
continue;
};
patches += 1;
positions += g.positions.len();
for (v, p) in g.positions.iter().zip(rg.positions.iter()) {
if p == v {
identical += 1;
} else {
changed += 1;
}
}
for off in 0..disc_block.len().min(ram_block.len()) {
if disc_block[off] == ram_block[off] {
continue;
}
differing += 1;
if g.position_ranges.iter().any(|&(a, b)| off >= a && off < b) {
differing_in_positions += 1;
} else if g.other_ranges.iter().any(|&(a, b)| off >= a && off < b) {
differing_in_other_unpacks += 1;
} else {
differing_in_command_words += 1;
}
}
}
// Bounds: 833 patches, 40,600 positions, 166,465 rewritten bytes.
assert!(patches > 800, "expected every patch, got {patches}");
assert_eq!(positions, 40_600, "position count");
assert_eq!(identical, positions, "every position should be verbatim");
assert_eq!(changed, 0, "no position may differ between disc and RAM");
assert_eq!(
differing_in_positions, 0,
"a rewritten byte inside a position unpack falsifies the claim"
);
assert!(
differing > 100_000,
"the game does rewrite the blocks, so this must be non-trivial: {differing}"
);
assert!(
differing_in_command_words * 2 > differing,
"the rewrites should be mostly packet framing, not payload: {differing_in_command_words} of {differing}"
);
assert!(
differing_in_command_words + differing_in_other_unpacks == differing,
"every differing byte must be classified"
);
}
/// A little-endian `i32` at `at`, or `None` if it does not fit.
fn i32_at(bytes: &[u8], at: usize) -> Option<i32> {
bytes
.get(at..at + 4)
.map(|b| i32::from_le_bytes(b.try_into().unwrap()))
}
/// Occurrences of `needle` in `hay`, for the RAM search above.
fn occ(hay: &[u8], needle: &[u8]) -> Vec<usize> {
hay.windows(needle.len().max(1))
.enumerate()
.filter(|(_, w)| *w == needle)
.map(|(i, _)| i)
.collect()
}
/// Claim 1: the collision payload is byte-for-byte identical between disc and RAM.
#[test]
fn collision_payload_is_verbatim_in_ram() {
+17
View File
@@ -688,6 +688,16 @@ pub struct TfragGeometry {
pub rgba: Vec<TfragRgba>,
/// Per-vertex light and normal records, indexed by position index.
pub lights: Vec<TfragLight>,
/// Byte ranges of the V3_16 position unpacks, relative to the start of the
/// block the lists were walked in. Two walks of the same header over two
/// block copies agree on these, so a per-byte diff can be classified as
/// "inside a position unpack" or not.
pub position_ranges: Vec<(usize, usize)>,
/// Byte ranges of every other unpack's payload in the same walks. Kept
/// alongside `position_ranges` so a per-byte diff can be classified as
/// "inside a position unpack", "inside another unpack", or "in a VIF
/// command word" without a second walk.
pub other_ranges: Vec<(usize, usize)>,
}
impl TfragGeometry {
@@ -794,6 +804,12 @@ pub fn parse_tfrag(block: &[u8], header: &TfragHeader) -> Result<TfragGeometry>
last_strow = read_i32x4(payload, 0);
}
} else if v.is_unpack() {
let payload_range = (pos + 4, pos + v.packet_size());
if (v.vn(), v.vl()) == (2, 1) {
g.position_ranges.push(payload_range);
} else {
g.other_ranges.push(payload_range);
}
let addr = v.vu_address();
match (v.vn(), v.vl()) {
(3, 1) if v.usn() && addr == 0 => {
@@ -837,6 +853,7 @@ pub fn parse_tfrag(block: &[u8], header: &TfragHeader) -> Result<TfragGeometry>
2
};
g.positions_tier[tier] = v.count();
g.position_ranges.push((pos + 4, pos + v.packet_size()));
for p3 in read_i16x3(payload, v.count())? {
g.positions.push([p3[0] as i32, p3[1] as i32, p3[2] as i32]);
}
+45 -1
View File
@@ -2570,6 +2570,50 @@ correct stride, which is not yet known. The LOD1/LOD2-sharing test
(96.1% same class against 0.4% different) remains the strongest evidence that
the first slots are positional, and it is insensitive to stride.
## The tfrag morph is not a rewrite of the position arrays (2026-10-14)
**Range measured:** level 0, all 833 tfrag patches, live PINE RAM against the
disc.
The morph was recorded as unverifiable because "reading a weight back out of
RAM does not work -- the positions live inside regenerated VU packets, so there
is no stable per-vertex correspondence". That reading is right about the
correspondence and wrong about the cause, and the cause is now measurable.
**The game rewrites bytes inside the tfrag blocks, and none of them are in a
position.**
patches 833
positions 40,600 (disc == ram)
positions identical 40,600
block bytes differing 166,465
inside a position unpack 0
inside another unpack 49,484
in a VIF command word 116,981
The position ranges total 490,224 bytes and not one of them differs. The two
measurements agree independently: walking the VIF lists and comparing parsed
positions gives 40,600 identical and zero changed, and a raw per-byte diff of
each patch's own extent classifies every differing byte.
So the blend is not precomputed into the geometry. It happens at draw time, in
the VU stream -- which is the standard PS2 arrangement and is why no weight was
ever readable from RAM: there was nothing there to read. The rewrites are
mostly packet framing (70% command words) plus some other unpack payloads
(30%), which is consistent with the loader rebuilding the VIF packet for the
tier it selected while leaving the vertex data alone.
**Extents.** Each patch's extent is the gap to the next patch's `data` pointer,
which is exact and needs no guess about which header field marks a patch's end.
The block is located in RAM by the first table record matched under the
invariant mask, the same way `gc-trace` locates it.
Asserted by `tfrag_positions_are_verbatim_and_the_rewrites_are_elsewhere`,
gated on `GC_SSTATE`. The test also requires the rewrites to be non-trivial
(more than 100,000 bytes) and to be mostly command words, so a RAM image that
happened to be a verbatim copy of the disc -- which would make the zero look
good for the wrong reason -- fails instead.
## Most of the chunk container is level-independent
Comparing every chunk record against the same record index in the other 26
@@ -3483,7 +3527,7 @@ the time.
the shuffle margin, so a claim that the period is not 64 fails rather than
sailing through.
## Attempting the moby_anim unblock: the class header, and where it stalls## Attempting the moby_anim unblock: the class header, and where it stalls
## Attempting the moby_anim unblock: the class header, and where it stalls
`MobyInstance` names `animSeq`, `animLayers`, `jointCnt` and `jointCache` as
runtime pointers. Those pointers are useless on the disc -- they are RAM
+21
View File
@@ -17,6 +17,7 @@ gc-trace find <src> --bytes HEX # content search (the locators use it
gc-trace read <src> --addr HEX --len N
gc-trace info <src> # savestate metadata
gc-trace compare-tfrag-light <unpacked-dir> [--level N] <src>
gc-trace verify-morph <unpacked-dir> [--level N] <src>
```
`<src>` is `--pine [slot]`, `--state FILE` / `--state latest`, or `--ee FILE`.
@@ -37,6 +38,26 @@ The comparison:
few mismatching vertices' *terms* (`gc_engine::tfrag_light::
light_vertex_terms`) printed so a mismatch can be attributed.
## verify-morph: the morph is not in the tfrag data block (2026-10-14)
`verify-morph` answers a different question than `compare-tfrag-light`. It
walks every patch's packet lists **twice** -- once over the disc copy, once over
the RAM copy -- and compares the parsed positions, then diffs the raw bytes of
each patch and classifies every differing byte against the walk's own unpack
ranges.
The result on level 0, live over PINE: 40,600 positions, all identical. The
game does rewrite the blocks -- 166,465 bytes -- and not one lands inside a
position unpack. So the LOD blend is not written back into the geometry.
That sharpens the standing note in the roadmap. "Reading a weight back out of
RAM does not work" was read as the positions being regenerated. They are not
regenerated in *value*: the loader rebuilds the packet framing (116,981
command-word bytes, 70% of the differences) and some other unpack payloads
(49,484), and leaves the vertices alone. The blend happens at draw time in the
VU stream. Nothing was there to read, which is a different finding from there
being a correspondence problem.
## Result: levels 0 and 1, verified 2026-10-11
Level 0 (Aranos), three inputs -- the live emulator over PINE and two PCSX2
+17 -7
View File
@@ -267,14 +267,24 @@ Phase 1 hands over the extents; this phase works out what each one contains.
short structured runs with no common `u32` addend. The loader regenerates
each patch's **VU packet stream for the LOD tier it selected**, which
happens to every patch regardless of morphing.
- [x] **The tfrag morph is not a rewrite of the position arrays.** Across all
**833** patches of level 0, live PINE RAM against the disc: **40,600**
positions, **40,600 identical, zero changed**. The game does rewrite the
blocks -- **166,465** bytes differ -- and **none** of them fall inside a
position unpack: 116,981 land in VIF command words and 49,484 in other
unpack payloads. The blend is therefore not precomputed into the geometry;
it happens at draw time in the VU stream, which is why reading a weight back
out of RAM never worked -- there was nothing there to read. This sharpens
the earlier "positions live inside regenerated VU packets, so there is no
stable per-vertex correspondence": the correspondence was never missing, the
values are just identical. `gc-trace verify-morph` and the
`GC_SSTATE`-gated test both carry the numbers.
- [ ] What tfrag is **not** yet verified is that the morph matches
`resolve_vertex`'s formula. Reading a weight back out of RAM does **not**
work -- the positions live inside regenerated VU packets, so there is no
stable per-vertex correspondence. The LOD switch distances, morph weight and
collapse edge remain derived from the data (tiers classified from the four
V4_8 parent arrays, weight running zero to one across each band); verifying
them needs the **camera distance at the savestate** so the weight can be
computed and compared against the rendered result.
`resolve_vertex`'s formula. The blend target is still untested against the
game itself, and now the search space is bounded: it is in the VU stream,
not in the tfrag data block. Verifying it needs the **camera distance at the
savestate** so the weight can be computed, or a read of the VU packet stream
the game actually submits.
- [x] The tie class header's **`+0x0c` word is the three LOD record
counts** — LOD 0 at `+0x0c`, LOD 1 at `+0x0d`, LOD 2 at `+0x0e` — verified
on **1,864 of 1,864** classes by reading the counts independently from
+1
View File
@@ -25,6 +25,7 @@
//! facts it needs into a small fixture and test against that.
pub mod ee;
pub mod morph;
pub mod overlay_base;
pub mod p2s;
pub mod pine;
+30
View File
@@ -26,6 +26,22 @@ use anyhow::{anyhow, Context, Result};
use gc_trace::ee::EeRam;
use gc_trace::{compare, LevelInputs};
/// Compare every patch's positions in RAM against the disc's, and solve the
/// blend weight for the ones that moved.
fn verify_morph(dir: &std::path::Path, level: usize, src: &Source) -> Result<i32> {
let idx = std::fs::read(dir.join(format!("level{level}/core_index.bin")))
.with_context(|| format!("level{level}/core_index.bin"))?;
let core = std::fs::read(dir.join(format!("level{level}/core_data.bin")))
.with_context(|| format!("level{level}/core_data.bin"))?;
let bytes = src.load()?;
let ram = EeRam::new(&bytes)?;
let report = gc_trace::morph::verify(&idx, &core, level, &ram)?;
let mut text = String::new();
gc_trace::morph::print_report(&report, &mut text)?;
print!("{text}");
Ok(report.exit_code())
}
fn main() {
let args: Vec<String> = std::env::args().skip(1).collect();
match run(&args) {
@@ -85,6 +101,19 @@ fn run(args: &[String]) -> Result<i32> {
println!("{text}");
Ok(i32::from(!report.names.iter().all(|&b| b)))
}
"verify-morph" => {
// gc-trace verify-morph <unpacked-dir> [--level N] <src>
let (src, rest) = Source::parse_with_rest(&args[1..])?;
let dir = rest
.first()
.map(PathBuf::from)
.ok_or_else(|| anyhow!("which unpacked level directory?"))?;
let level: usize = flag(&rest, "--level")
.unwrap_or_else(|| "0".into())
.parse()
.context("--level")?;
verify_morph(&dir, level, &src)
}
"verify-tie-records" => {
// gc-trace verify-tie-records <unpacked-dir> <level> <class> <lod> <src>
let (src, rest) = Source::parse_with_rest(&args[1..])?;
@@ -144,6 +173,7 @@ fn run(args: &[String]) -> Result<i32> {
const USAGE: &str = "\
usage: gc-trace <command> ...
compare-tfrag-light <unpacked-dir> [--level N] <src>
verify-morph <unpacked-dir> [--level N] <src>
verify-tie-records <unpacked-dir> <level> <class> <lod> <src>
verify-overlay-base <unpacked-dir> <level> <src>
info <src>
+374
View File
@@ -0,0 +1,374 @@
//! Verifying the tfrag morph against RAM.
//!
//! `resolve_vertex`'s formula is a blend toward the **parent vertices'
//! midpoint**, and reading a weight back out of RAM was reported as not working
//! because the positions live inside regenerated VU packets. There is a route
//! in that does not need the camera distance:
//!
//! * the loader regenerates each patch's packet stream for the tier it
//! selected, and it does that to **every** patch -- but only the entries that
//! actually morph change value;
//! * so on positions, disc versus RAM is identity where nothing morphs, and a
//! difference where it does. The diff names the morphed vertices, which is
//! the correspondence the problem lacked;
//! * with the vertex named, its morph link gives the two parents from the disc
//! side, and the observed RAM value gives the blend weight as
//! `u = (Q - P) / (mid - P)` per component.
//!
//! Three components give three estimates of one weight. If the formula is right
//! they agree and land in 0..1; if the game blends toward anything else they
//! disagree, and the spread is the finding.
use anyhow::{anyhow, Context, Result};
use gc_formats::tfrag::{
parse_tfrag, tfrag_lod_link, tfrag_morph_parents, TfragHeader, TfragsBlock, TFRAG_HEADER_SIZE,
};
use crate::ee::EeRam;
/// One patch's morph verdict.
#[derive(Debug, Clone)]
pub struct PatchMorph {
pub patch: usize,
pub positions_disc: usize,
pub positions_ram: usize,
pub identical: usize,
pub changed: usize,
/// Bytes that differ between the two block copies, classified by whether
/// they fall inside a V3_16 position unpack.
pub differing_bytes: usize,
pub differing_in_position_ranges: usize,
pub position_range_bytes: usize,
}
/// A single solved weight, with the spread across the three components.
#[derive(Debug, Clone)]
pub struct SolvedWeight {
pub patch: usize,
pub vertex: usize,
pub base: [i32; 3],
pub ram: [i32; 3],
pub mid: [i32; 3],
/// `u` solved per component. `None` where the component does not move.
pub per_component: [Option<f32>; 3],
pub spread: f32,
pub weight: f32,
}
#[derive(Debug, Default)]
pub struct Report {
pub level: usize,
pub block_base: usize,
pub patch_count: usize,
pub patches: Vec<PatchMorph>,
pub solved: Vec<SolvedWeight>,
/// Solved weights that are consistent and inside 0..1.
pub agreeing: usize,
/// Disc-side totals across every patch, for the diff classification.
pub differing_bytes: usize,
pub differing_in_positions: usize,
pub position_range_bytes: usize,
/// Of the differing bytes, how many land in another unpack's payload and
/// how many in a VIF command word.
pub differing_in_other_unpacks: usize,
pub differing_in_command_words: usize,
}
impl Report {
pub fn exit_code(&self) -> i32 {
i32::from(self.agreeing < 4)
}
}
pub fn print_report(r: &Report, out: &mut dyn std::fmt::Write) -> std::fmt::Result {
writeln!(
out,
"level {}: {} patches at block base {:#x}",
r.level, r.patch_count, r.block_base
)?;
let discs: usize = r.patches.iter().map(|p| p.positions_disc).sum();
let rams: usize = r.patches.iter().map(|p| p.positions_ram).sum();
let same: usize = r.patches.iter().map(|p| p.identical).sum();
let changed: usize = r.patches.iter().map(|p| p.changed).sum();
writeln!(
out,
" positions: disc {discs}, ram {rams}, identical {same}, changed {changed}"
)?;
writeln!(
out,
" solved weights: {} ({} agree)",
r.solved.len(),
r.agreeing
)?;
writeln!(
out,
" block bytes differing: {} -- {} in a position unpack ({} position bytes), \
{} in another unpack, {} in command words",
r.differing_bytes,
r.differing_in_positions,
r.position_range_bytes,
r.differing_in_other_unpacks,
r.differing_in_command_words
)?;
for s in r.solved.iter().take(8) {
writeln!(
out,
" patch {} v{}: base {:?} -> mid {:?}, ram {:?}, u {} pc {:?}",
s.patch, s.vertex, s.base, s.mid, s.ram, s.weight, s.per_component
)?;
}
Ok(())
}
/// Where the tfrags block sits in RAM, and the header table inside it.
fn locate(region: &[u8], ram: &EeRam, table_off: usize, count: usize) -> Result<usize> {
let key = &region[table_off..table_off + 16];
let mut best: Option<(usize, usize)> = None;
for cand in ram.find_all(key) {
if cand < table_off {
continue;
}
let base = cand - table_off;
let Some(hdr) = ram.slice(base, 0x10) else {
continue;
};
if hdr != &region[..0x10.min(region.len())] {
continue;
}
let mut matched = 0usize;
for i in 0..count.min(64) {
let at = i * TFRAG_HEADER_SIZE;
let d = region.get(table_off + at..table_off + at + TFRAG_HEADER_SIZE);
let r = ram.slice(cand + at, TFRAG_HEADER_SIZE);
if let (Some(a), Some(b)) = (d, r) {
if invariant_equal(a, b) {
matched += 1;
}
}
}
if best.is_none_or(|(_, m)| matched > m) {
best = Some((base, matched));
}
}
match best {
Some((base, m)) if m == count.min(64) => Ok(base),
Some((base, m)) => Err(anyhow!(
"tfrags block at {base:#x} matches {m} of {} headers; refusing a partial match",
count.min(64)
)),
None => Err(anyhow!("no tfrags block in RAM matches this level")),
}
}
fn invariant_equal(disc: &[u8], ram: &[u8]) -> bool {
let keep = |j: usize| !(0x10..0x14).contains(&j) && !(0x3a..0x3c).contains(&j);
disc.iter()
.zip(ram.iter())
.enumerate()
.filter(|&(j, _)| keep(j))
.all(|(_, (a, b))| a == b)
}
pub fn verify(disc_index: &[u8], disc_core: &[u8], level: usize, ram: &EeRam) -> Result<Report> {
let tfrags_ofs = i32_at(disc_index, 0x08).context("tfrags offset")?;
let region_end = [0x0c_usize, 0x10, 0x14]
.iter()
.filter_map(|&o| i32_at(disc_index, o))
.filter(|&v| v > tfrags_ofs)
.min()
.unwrap_or(disc_core.len() as i32);
let region = &disc_core[tfrags_ofs as usize..region_end as usize];
let block = TfragsBlock::parse(region).context("tfrags block")?;
let table_off = block.table_offset as usize;
let count = block.tfrag_count as usize;
if count == 0 {
anyhow::bail!("level{level} has no tfrags");
}
let base = locate(region, ram, table_off, count)?;
let table = base + table_off;
let _ = level;
let mut report = Report {
block_base: base,
patch_count: count,
..Default::default()
};
// Each patch's extent is the gap to the next one's `data`, which is exact
// and needs no guess about which header field marks the patch's end.
// Spans are `(start, length)` relative to the start of the disc region.
let mut starts: Vec<(usize, usize)> = (0..count)
.filter_map(|i| {
let v = i32_at(region, table_off + i * TFRAG_HEADER_SIZE + 0x10).filter(|&d| d > 0)?;
// `data` is relative to the table, and these spans index the
// region, so carry the table base along.
Some((table_off + v as usize, i))
})
.collect();
starts.sort_unstable();
starts.dedup_by_key(|&mut (v, _)| v);
let region_len = region.len();
let span_of = |i: usize| -> Option<(usize, usize)> {
let k = starts.iter().position(|&(_, j)| j == i)?;
let (start, _) = starts[k];
let end = starts.get(k + 1).map(|&(v, _)| v).unwrap_or(region_len);
Some((start, end.saturating_sub(start)))
};
for i in 0..count {
let at = i * TFRAG_HEADER_SIZE;
let disc_h = TfragHeader::parse(
region
.get(table_off + at..table_off + at + TFRAG_HEADER_SIZE)
.ok_or_else(|| anyhow!("tfrag header {i} is out of bounds"))?,
)
.with_context(|| format!("tfrag {i} header"))?;
let ram_h = TfragHeader::parse(
ram.slice(table + at, TFRAG_HEADER_SIZE)
.ok_or_else(|| anyhow!("tfrag {i} header is out of RAM"))?,
)
.with_context(|| format!("tfrag {i} RAM header"))?;
if disc_h.vert_count == 0 || disc_h.data == 0 {
continue;
}
// The patch's own extent, from the gap to the next one.
let Some((span_start, span_len)) = span_of(i) else {
continue;
};
let disc_block = region
.get(span_start..span_start + span_len)
.ok_or_else(|| anyhow!("tfrag {i} disc data runs past the region"))?;
let disc_geo = match parse_tfrag(disc_block, &disc_h) {
Ok(g) => g,
Err(_) => continue,
};
// The RAM copy: behind the relocated pointer, bounded by the region's
// RAM end. `parse_tfrag` indexes from the start of what it is given.
let ram_data = ram_h.data as usize;
let region_end_ram = base + region.len();
if ram_data < base {
anyhow::bail!("tfrag {i} data pointer {ram_data:#x} is before the block");
}
if ram_data + span_len > region_end_ram || ram_data + span_len > ram.bytes().len() {
continue;
}
let block_ram = &ram.bytes()[ram_data..ram_data + span_len];
let ram_geo = match parse_tfrag(block_ram, &ram_h) {
Ok(g) => g,
Err(_) => continue,
};
let dp = disc_geo.positions.clone();
let rp = ram_geo.positions;
let n = dp.len().min(rp.len());
let mut identical = 0usize;
let mut changed = 0usize;
for v in 0..n {
if dp[v] == rp[v] {
identical += 1;
} else {
changed += 1;
}
}
// Classify the raw byte diff. The two block copies start at their own
// absolute addresses but the header's field offsets are invariant, so a
// range from the disc walk indexes the RAM copy the same way.
let ranges = &disc_geo.position_ranges;
let other = &disc_geo.other_ranges;
let position_range_bytes: usize = ranges.iter().map(|&(a, b)| b.saturating_sub(a)).sum();
let mut differing_bytes = 0usize;
let mut differing_in_position_ranges = 0usize;
let mut differing_in_other_unpacks = 0usize;
let mut differing_in_command_words = 0usize;
let cmp_len = disc_block.len().min(block_ram.len());
for off in 0..cmp_len {
if disc_block[off] == block_ram[off] {
continue;
}
differing_bytes += 1;
if ranges.iter().any(|&(a, b)| off >= a && off < b) {
differing_in_position_ranges += 1;
} else if other.iter().any(|&(a, b)| off >= a && off < b) {
differing_in_other_unpacks += 1;
} else {
differing_in_command_words += 1;
}
}
report.differing_bytes += differing_bytes;
report.differing_in_positions += differing_in_position_ranges;
report.position_range_bytes += position_range_bytes;
report.differing_in_other_unpacks += differing_in_other_unpacks;
report.differing_in_command_words += differing_in_command_words;
report.patches.push(PatchMorph {
patch: i,
positions_disc: dp.len(),
positions_ram: rp.len(),
identical,
changed,
differing_bytes,
differing_in_position_ranges,
position_range_bytes,
});
// Every changed position whose morph link resolves gives a weight.
for v in 0..n {
if dp[v] == rp[v] {
continue;
}
let Some(link) = tfrag_lod_link(&disc_geo, v) else {
continue;
};
if !link.morphs {
continue;
}
let Some((p1, p2)) = tfrag_morph_parents(&disc_geo, v) else {
continue;
};
let a = dp[p1];
let b = dp[p2];
let mid = [(a[0] + b[0]) / 2, (a[1] + b[1]) / 2, (a[2] + b[2]) / 2];
let base_pos = dp[v];
let ram_pos = rp[v];
let mut per_component = [None, None, None];
for k in 0..3 {
let denom = (mid[k] - base_pos[k]) as f32;
if denom.abs() > f32::EPSILON {
per_component[k] = Some((ram_pos[k] - base_pos[k]) as f32 / denom);
}
}
let present: Vec<f32> = per_component.iter().flatten().copied().collect();
if present.is_empty() {
continue;
}
let weight = present.iter().sum::<f32>() / present.len() as f32;
let spread =
present.iter().fold(0.0f32, |a, &u| a + (u - weight).abs()) / present.len() as f32;
let in_band = present.iter().all(|&u| (-0.05..=1.05).contains(&u));
if in_band && spread < 0.02 {
report.agreeing += 1;
}
report.solved.push(SolvedWeight {
patch: i,
vertex: v,
base: base_pos,
ram: ram_pos,
mid,
per_component,
spread,
weight,
});
}
}
Ok(report)
}
fn i32_at(bytes: &[u8], at: usize) -> Option<i32> {
bytes
.get(at..at + 4)
.map(|b| i32::from_le_bytes(b.try_into().unwrap()))
}