The tfrag morph is not in the data block: positions are verbatim in RAM

The morph was recorded as unverifiable because "reading a weight back out of
RAM does not work -- the positions live inside regenerated VU packets". That
is right about the correspondence and wrong about the cause. The cause is
measurable, and it comes back clean:

    patches                         833
    positions                     40,600   disc == ram
    positions identical           40,600
    block bytes differing         166,465
      inside a position unpack             0
      inside another unpack           49,484
      in a VIF command word          116,981

Range: level 0, all 833 tfrag patches, live PINE RAM against the disc. Two
independent measurements agree. Walking the VIF lists on both copies and
comparing parsed positions gives 40,600 identical and zero changed; a raw
per-byte diff of each patch's own extent classifies every differing byte, and
none of them is in a position unpack. The position ranges total 490,224 bytes.

So the blend is not precomputed into the geometry. The rewrites are packet
framing -- 70% command words -- plus other unpack payloads, which is the loader
rebuilding the VIF packet for the tier it selected while leaving the vertices
alone. The morph happens at draw time in the VU stream.

The standing note becomes a sharper one: there was never a correspondence
problem. The values were identical, so there was nothing to read.

gc-trace gains verify-morph, which does both measurements. TfragGeometry
gains position_ranges and other_ranges so the walk needed to classify a diff
is already paid for, and the ranges come from the same walk the rest of the
parse uses rather than a second one that could drift.

Per-patch extents are the gap to the next patch's data pointer. That is exact
and needs no guess about which header field marks a patch's end; the first
attempt at this compared each patch against the whole rest of the region and
reported 65 million differing bytes, which is what an unbounded comparison
looks like.

tfrag_positions_are_verbatim_and_the_rewrites_are_elsewhere is gated on
GC_SSTATE and asserts the four bounds, plus two guards: the rewrites must be
non-trivial (over 100,000 bytes) and mostly command words, so a RAM image that
was a verbatim copy of the disc -- which would make the zero look good for the
wrong reason -- fails instead.

Suite with the disc and GC_SSTATE: 68 disc tests, 156 gc-formats, 22
gc-trace, 23 gc-engine, 19 gc-game. Clippy clean.
This commit is contained in:
2026-10-14 12:27:38 +02:00
parent 4d66cf6214
commit 3a1156d8e1
8 changed files with 698 additions and 8 deletions
+193
View File
@@ -4858,6 +4858,199 @@ fn ram_slice(ram: &[u8], addr: usize, len: usize) -> Option<&[u8]> {
ram.get(addr..addr.checked_add(len)?)
}
/// The tfrag morph is not a rewrite of the position arrays.
///
/// The game rewrites bytes inside every tfrag data block, and this test says
/// where those bytes are. Positions are byte-identical between disc and RAM on
/// every patch, so the blend cannot be precomputed into the geometry -- it
/// happens at draw time, in the VU stream, which is why reading a weight back
/// out of RAM never worked.
#[test]
fn tfrag_positions_are_verbatim_and_the_rewrites_are_elsewhere() {
let Some(ram) = ram_image() else {
eprintln!("skipped: set GC_SSTATE to a PCSX2 .p2s savestate");
return;
};
let Some(data) = extracted_root() else {
eprintln!("skipped: unpack the disc first");
return;
};
require_disc!();
use gc_formats::level_core::LevelCoreHeader;
use gc_formats::tfrag::{parse_tfrag, TfragHeader, TfragsBlock, TFRAG_HEADER_SIZE};
let idx = std::fs::read(data.join("level0/core_index.bin")).expect("core_index");
let core = std::fs::read(data.join("level0/core_data.bin")).expect("core_data");
let h = LevelCoreHeader::parse(&idx).expect("hdr");
let tfrags_ofs = h.tfrags as usize;
let region_end = [0x0c_usize, 0x10, 0x14]
.iter()
.filter_map(|&o| i32_at(&idx, o))
.filter(|&v| v as usize > tfrags_ofs)
.map(|v| v as usize)
.min()
.unwrap_or(core.len());
let region = &core[tfrags_ofs..region_end];
let block = TfragsBlock::parse(region).expect("tfrags block");
let table_off = block.table_offset as usize;
let count = block.tfrag_count as usize;
assert!(
count > 700,
"level0 should hold the full 833 patches, got {count}"
);
// Find the block in RAM the same way gc-trace does: the first record of the
// table, matched byte for byte, with the invariant mask applied.
let key = &region[table_off..table_off + 16];
let mut base = None;
for cand in occ(ram.as_ref(), key) {
if cand < table_off {
continue;
}
let b = cand - table_off;
let Some(hdr) = ram_slice(&ram, b, 0x10) else {
continue;
};
if hdr != &region[..0x10] {
continue;
}
let mut matched = 0usize;
for i in 0..count.min(64) {
let at = i * TFRAG_HEADER_SIZE;
let d = region.get(table_off + at..table_off + at + TFRAG_HEADER_SIZE);
let r = ram_slice(&ram, cand + at, TFRAG_HEADER_SIZE);
if let (Some(a), Some(b2)) = (d, r) {
if a.iter()
.zip(b2.iter())
.enumerate()
.filter(|&(j, _)| !(0x10..0x14).contains(&j) && !(0x3a..0x3c).contains(&j))
.all(|(_, (x, y))| x == y)
{
matched += 1;
}
}
}
if matched == count.min(64) {
base = Some(b);
break;
}
}
let base = base.expect("the tfrags block should be locatable in RAM");
let table = base + table_off;
// Patch extents from the gap to the next `data`, which is exact.
let mut starts: Vec<(usize, usize)> = (0..count)
.filter_map(|i| {
let v = i32_at(region, table_off + i * TFRAG_HEADER_SIZE + 0x10).filter(|&d| d > 0)?;
Some((table_off + v as usize, i))
})
.collect();
starts.sort_unstable();
starts.dedup_by_key(|&mut (v, _)| v);
let mut positions = 0usize;
let mut identical = 0usize;
let mut changed = 0usize;
let mut differing = 0usize;
let mut differing_in_positions = 0usize;
let mut differing_in_other_unpacks = 0usize;
let mut differing_in_command_words = 0usize;
let mut patches = 0usize;
for i in 0..count {
let at = i * TFRAG_HEADER_SIZE;
let disc_h =
TfragHeader::parse(&region[table_off + at..table_off + at + TFRAG_HEADER_SIZE])
.expect("disc header");
if disc_h.vert_count == 0 || disc_h.data == 0 {
continue;
}
let ram_h = TfragHeader::parse(ram_slice(&ram, table + at, TFRAG_HEADER_SIZE).unwrap())
.expect("ram header");
let Some(k) = starts.iter().position(|&(_, j)| j == i) else {
continue;
};
let start = starts[k].0;
let end = starts.get(k + 1).map(|&(v, _)| v).unwrap_or(region.len());
if end <= start {
continue;
}
let disc_block = &region[start..end];
let ram_data = ram_h.data as usize;
if ram_data < base || ram_data + (end - start) > ram.len() {
continue;
}
let ram_block = &ram[ram_data..ram_data + (end - start)];
let Ok(g) = parse_tfrag(disc_block, &disc_h) else {
continue;
};
let Ok(rg) = parse_tfrag(ram_block, &ram_h) else {
continue;
};
patches += 1;
positions += g.positions.len();
for (v, p) in g.positions.iter().zip(rg.positions.iter()) {
if p == v {
identical += 1;
} else {
changed += 1;
}
}
for off in 0..disc_block.len().min(ram_block.len()) {
if disc_block[off] == ram_block[off] {
continue;
}
differing += 1;
if g.position_ranges.iter().any(|&(a, b)| off >= a && off < b) {
differing_in_positions += 1;
} else if g.other_ranges.iter().any(|&(a, b)| off >= a && off < b) {
differing_in_other_unpacks += 1;
} else {
differing_in_command_words += 1;
}
}
}
// Bounds: 833 patches, 40,600 positions, 166,465 rewritten bytes.
assert!(patches > 800, "expected every patch, got {patches}");
assert_eq!(positions, 40_600, "position count");
assert_eq!(identical, positions, "every position should be verbatim");
assert_eq!(changed, 0, "no position may differ between disc and RAM");
assert_eq!(
differing_in_positions, 0,
"a rewritten byte inside a position unpack falsifies the claim"
);
assert!(
differing > 100_000,
"the game does rewrite the blocks, so this must be non-trivial: {differing}"
);
assert!(
differing_in_command_words * 2 > differing,
"the rewrites should be mostly packet framing, not payload: {differing_in_command_words} of {differing}"
);
assert!(
differing_in_command_words + differing_in_other_unpacks == differing,
"every differing byte must be classified"
);
}
/// A little-endian `i32` at `at`, or `None` if it does not fit.
fn i32_at(bytes: &[u8], at: usize) -> Option<i32> {
bytes
.get(at..at + 4)
.map(|b| i32::from_le_bytes(b.try_into().unwrap()))
}
/// Occurrences of `needle` in `hay`, for the RAM search above.
fn occ(hay: &[u8], needle: &[u8]) -> Vec<usize> {
hay.windows(needle.len().max(1))
.enumerate()
.filter(|(_, w)| *w == needle)
.map(|(i, _)| i)
.collect()
}
/// Claim 1: the collision payload is byte-for-byte identical between disc and RAM.
#[test]
fn collision_payload_is_verbatim_in_ram() {
+17
View File
@@ -688,6 +688,16 @@ pub struct TfragGeometry {
pub rgba: Vec<TfragRgba>,
/// Per-vertex light and normal records, indexed by position index.
pub lights: Vec<TfragLight>,
/// Byte ranges of the V3_16 position unpacks, relative to the start of the
/// block the lists were walked in. Two walks of the same header over two
/// block copies agree on these, so a per-byte diff can be classified as
/// "inside a position unpack" or not.
pub position_ranges: Vec<(usize, usize)>,
/// Byte ranges of every other unpack's payload in the same walks. Kept
/// alongside `position_ranges` so a per-byte diff can be classified as
/// "inside a position unpack", "inside another unpack", or "in a VIF
/// command word" without a second walk.
pub other_ranges: Vec<(usize, usize)>,
}
impl TfragGeometry {
@@ -794,6 +804,12 @@ pub fn parse_tfrag(block: &[u8], header: &TfragHeader) -> Result<TfragGeometry>
last_strow = read_i32x4(payload, 0);
}
} else if v.is_unpack() {
let payload_range = (pos + 4, pos + v.packet_size());
if (v.vn(), v.vl()) == (2, 1) {
g.position_ranges.push(payload_range);
} else {
g.other_ranges.push(payload_range);
}
let addr = v.vu_address();
match (v.vn(), v.vl()) {
(3, 1) if v.usn() && addr == 0 => {
@@ -837,6 +853,7 @@ pub fn parse_tfrag(block: &[u8], header: &TfragHeader) -> Result<TfragGeometry>
2
};
g.positions_tier[tier] = v.count();
g.position_ranges.push((pos + 4, pos + v.packet_size()));
for p3 in read_i16x3(payload, v.count())? {
g.positions.push([p3[0] as i32, p3[1] as i32, p3[2] as i32]);
}