ReaperOfVeriod/Going-Commando
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Verify the tfrag lighting magnitudes against the running game

Browse Source 
The harness that compares the port against the game's own memory existed but
had never been compiled: tools/trace did not build (27 errors, all in the
argument plumbing), and the tfrag_light.rs light-bank correction it motivated
was unverified. This finishes it and runs it, and the run finds two real bugs.

Result, on level 0 against three inputs -- the live emulator over PINE and two
PCSX2 savestates from the same level, agreeing byte for byte:

  headers matching the disc on invariant bytes: 833/833
  data pointers relocated as table + disc offset: 833/833
  light bank in RAM at 0x50ab20, byte-identical to the disc's
  vertices compared: 40600 (162400 colour bytes)
  equal: 162400 (100.00%)

The locator finds the level's tfrags block in RAM by tfrag 0's bounding
sphere and accepts a candidate only when every header matches the disc on the
bytes the game leaves alone and every data pointer is the relocated form, so
"the block was found" is itself a verification rather than an assumption. The
bank is searched in RAM by its exact section bytes, which is what settles its
framing: offset at +0x04, an i32 set count, twelve bytes of pad, then the sets.
The previous reading (offset at +0x00, count at +0x04) was wrong.

Two bugs the comparison found, both of which produced plausible output:

1. vertex_normal_bits negated x and y and left z positive, while
   NormalTable::normal negates all three. dA came out -0.707 where the game
   had +0.707, so max(dA, dA*wA) took the wrong branch and every vertex lit
   near its base colour: dark, but not wrong-dark. 95.88% agreement with the
   bug, 100.00% without.
2. A record whose select names a set the bank lacks drew transparent black.
   The game writes the record's base colour; 3,130 of level 0's 40,600
   vertices do this (select 0x000f, set 15 -- a runtime slot the level's
   directional bank does not have). The vertex is unlit, not invisible.

Both are now regression tests in the engine. The comparison prints the pass's
per-vertex terms (light_vertex_terms) for the first mismatches, because a
colour mismatch alone cannot be attributed and the comparison is worthless
without attribution -- that is how the second bug was identified from a
three-line read rather than guessed at.

Docs: docs/plan/pcsx2-verification.md records the harness, the result, the
two bugs and what to compare next; roadmap and level-assets.md stop carrying
lighting magnitudes as an open item. Findings reach tests only as numbers --
no test reads a savestate, a dump or a live emulator.
This commit is contained in:
ReaperOfVeriod
2026-10-11 19:51:25 +02:00
parent 2c7555fe16
commit 69a8398f6e
15 changed files with 2144 additions and 25 deletions
Download Patch File Download Diff File Expand all files Collapse all files
crates/gc-engine/src/tfrag_light.rs
+124 -14
View File
@@ -136,6 +136,44 @@ fn active_set(bank: &LightBank, select: u16) -> Option<ActiveSet> {
})
}
/// The pass's terms for one vertex.
///
/// This exists because a colour mismatch on its own cannot be attributed: the
/// final byte is a sum of five things, and the PCSX2 comparison -- the only
/// check on this pass there is -- is worthless without knowing *which* of them
/// the game disagrees with. It is a snapshot of what the pass computed, not a
/// second implementation: `light_vertex_n` and this share one body.
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct VertexTerms {
/// The record's azimuth and elevation indices.
pub azimuth: u8,
pub elevation: u8,
/// The record's base colour, PEXT5-expanded.
pub base: [u8; 4],
/// The `select` word, the set it named, and whether it blended two.
pub select: u16,
pub set_index: Option<usize>,
pub blended: bool,
/// The vertex's normal.
pub normal: [f32; 4],
/// The active set's colours and directions, with the w lanes cleared as
/// the pass clears them.
pub color_a: [f32; 4],
pub color_b: [f32; 4],
pub dir_a: [f32; 4],
pub dir_b: [f32; 4],
/// The back-face factors taken from the colour w lanes.
pub wa: f32,
pub wb: f32,
/// The two dot products, and each after its `max(d, d*w)`.
pub dot_a: f32,
pub dot_b: f32,
pub max_a: f32,
pub max_b: f32,
/// The final colour.
pub out: [u8; 4],
}
/// Light one vertex: `base + A.rgb * max(dA, dA*wA) + B.rgb * max(dB, dB*wB)`.
pub fn light_vertex(
bank: &LightBank,
@@ -151,20 +189,65 @@ fn light_vertex_n(
light: &gc_formats::tfrag::TfragLight,
select: u16,
) -> Option<[u8; 4]> {
Some(light_vertex_terms(bank, n, light, select)?.out)
}
/// Light one vertex, keeping the terms for the comparison.
pub fn light_vertex_terms(
bank: &LightBank,
n: V4,
light: &gc_formats::tfrag::TfragLight,
select: u16,
) -> Option<VertexTerms> {
let s = active_set(bank, select)?;
let da = dot3(n, s.da);
let db = dot3(n, s.db);
let da = ps2::max(da, ps2::mul(da, s.wa));
let db = ps2::max(db, ps2::mul(db, s.wb));
let acc = vmuls(color_float(pext5(light.color as u16)), ps2::ONE);
let acc = vmadds(acc, s.ca, da);
Some(pack_color(vmadds(acc, s.cb, db)))
let da_max = ps2::max(da, ps2::mul(da, s.wa));
let db_max = ps2::max(db, ps2::mul(db, s.wb));
let base = pext5(light.color as u16);
let acc = vmuls(color_float(base), ps2::ONE);
let acc = vmadds(acc, s.ca, da_max);
let out = pack_color(vmadds(acc, s.cb, db_max));
let unbits = |v: V4| v.map(f32::from_bits);
Some(VertexTerms {
azimuth: light.azimuth_index(),
elevation: light.elevation_index(),
base,
select,
set_index: if select & 0xff00 == 0 {
bank.sets
.get((select & 0xf) as usize)
.map(|_| (select & 0xf) as usize)
} else {
let low = bank.sets.get((select & 0xf) as usize);
let high = bank.sets.get(((select >> 4) & 0xf) as usize);
low.and(high).map(|_| (select & 0xf) as usize)
},
blended: select & 0xff00 != 0,
normal: unbits(n),
color_a: unbits(s.ca),
color_b: unbits(s.cb),
dir_a: unbits(s.da),
dir_b: unbits(s.db),
wa: f32::from_bits(s.wa),
wb: f32::from_bits(s.wb),
dot_a: f32::from_bits(da),
dot_b: f32::from_bits(db),
max_a: f32::from_bits(da_max),
max_b: f32::from_bits(db_max),
out,
})
}
/// The normal a vertex lights with, as raw PS2 floats.
///
/// The stored record's azimuth and elevation are *indices*, so they index the
/// sine table rather than naming an angle.
///
/// All three components are negated: the stored normals point into the
/// surface. The z sign is what the PCSX2 comparison settled -- negating only x
/// and y still produces a plausible level, and it cost 48 percentage points of
/// agreement on level 0 (95.88% vs 100.00%) before it was measured.
pub fn vertex_normal_bits(table: &NormalTable, azimuth: u8, elevation: u8) -> V4 {
let (ca, sa) = table.get(azimuth);
let (ce, se) = table.get(elevation);
@@ -173,22 +256,23 @@ pub fn vertex_normal_bits(table: &NormalTable, azimuth: u8, elevation: u8) -> V4
[
ps2::sub(0, x),
ps2::sub(0, y),
ps2::add(0, se.to_bits()),
ps2::sub(0, se.to_bits()),
se.to_bits(),
]
}
/// Run the lighting pass over a patch's light records.
///
/// Returns one colour per light record, in the same order. Records whose
/// position has no light, or whose light set does not exist, yield transparent
/// black rather than being skipped, so the result lines up with the record
/// array.
/// Returns one colour per light record, in the same order. A record whose
/// `select` names a light set the bank does not have is left at its base
/// colour, which is what the game writes: the PCSX2 comparison shows those
/// records' RAM colours equal to their PEXT5 base exactly. Transparent black
/// was the first guess and it was wrong -- the vertex is unlit, not invisible.
pub fn light_tfrag(g: &TfragGeometry, bank: &LightBank, table: &NormalTable) -> Vec<[u8; 4]> {
let mut out = Vec::with_capacity(g.lights.len());
for l in &g.lights {
let n = vertex_normal_bits(table, l.azimuth_index(), l.elevation_index());
out.push(light_vertex_n(bank, n, l, l.select).unwrap_or([0, 0, 0, 0]));
out.push(light_vertex_n(bank, n, l, l.select).unwrap_or(pext5(l.color as u16)));
}
out
}
@@ -273,11 +357,20 @@ mod tests {
}
#[test]
fn an_empty_bank_lights_nothing() {
fn an_empty_bank_leaves_the_base_colour() {
// The game's own answer, over PINE: a record whose select names a set
// the bank does not have keeps its base colour. Transparent black was
// the first guess, and the comparison read it as a 4.4% mismatch on
// level 0 before this was measured.
let bank = LightBank::default();
let t = table();
let out = light_tfrag(&patch(vec![light(0)]), &bank, &t);
assert_eq!(out, vec![[0, 0, 0, 0]]);
let mut l = light(0);
// 5:5:5:1 with the top red step and the alpha bit set: (8, 0, 0,
// 128). Stored as the i16 the record carries, expanded by PEXT5.
l.color = 0x8001u16.cast_signed();
let out = light_tfrag(&patch(vec![l]), &bank, &t);
// PEXT5 of 1 is 1 << 3 = 8, with the alpha bit at 0x80.
assert_eq!(out, vec![[8, 0, 0, 128]]);
}
#[test]
@@ -289,4 +382,21 @@ mod tests {
assert_eq!(n[1], 0);
assert_eq!(n[2], 0);
}
#[test]
fn the_normal_s_z_is_negated_like_x_and_y() {
// Elevation 64 is straight up: (0, 0, 1) before negation, so the
// stored normal points down into the surface. Getting the z sign
// wrong still produces a plausible level -- it flips which vertices
// a light reaches -- and it cost 48 percentage points of agreement
// before the PCSX2 comparison caught it.
let t = table();
let n = vertex_normal_bits(&t, 0, 64);
let f = |b: u32| f32::from_bits(b);
// The table's cos(pi/2) is 1.5e-8 rather than exactly zero, which is
// what the game's own table holds, so x and y are ~0 rather than 0.
assert!(f(n[0]).abs() < 1e-6, "x should be ~0, got {}", f(n[0]));
assert!(f(n[1]).abs() < 1e-6, "y should be ~0, got {}", f(n[1]));
assert_eq!(n[2], f32::to_bits(-1.0));
}
}
crates/gc-formats/src/tfrag_light.rs
+45 -6
View File
@@ -151,19 +151,34 @@ const MAX_LIGHTS: usize = 64;
impl LightBank {
/// Parse the bank from a level's `gameplay` extent.
///
/// The header at +0x04 is an offset to a count followed by the sets; a
/// negative count means the level has no lights.
/// The extent opens with a table of section offsets in bytes; the word at
/// **+0x04** is the directional-light section's offset. That section is an
/// `i32` set count, twelve bytes of pad, then the sets. This was read the
/// other way round — offset at +0x00, count at +0x04 — until the PCSX2
/// comparison showed the pass running on denormal garbage and reporting
/// every vertex lit; see `docs/plan/pcsx2-verification.md`.
pub fn parse(gameplay: &[u8]) -> Result<LightBank> {
let mut b = Buf::new(gameplay);
b.skip(4, "gameplay extent header")?;
let ofs = b.u32("light bank offset")? as usize;
let count = b.i32("directional light count")?;
if ofs + 0x10 > gameplay.len() {
return Err(Error {
offset: ofs,
context: "light bank offset runs past the gameplay extent",
kind: ErrorKind::OutOfBounds {
needed: 0x10,
available: gameplay.len().saturating_sub(ofs),
},
});
}
let count = i32::from_le_bytes(gameplay[ofs..ofs + 4].try_into().unwrap());
if count < 0 {
return Ok(LightBank::default());
}
let n = (count as usize).min(MAX_LIGHTS);
let mut sets = Vec::with_capacity(n);
for _ in 0..n {
sets.push(read_set(gameplay, ofs + 0x10 + sets.len() * 0x40)?);
for i in 0..n {
sets.push(read_set(gameplay, ofs + 0x10 + i * 0x40)?);
}
Ok(LightBank {
sets,
@@ -264,8 +279,32 @@ mod tests {
#[test]
fn a_level_with_no_lights_parses_as_empty() {
let mut g = vec![0u8; 32];
g[0x04..0x08].copy_from_slice(&(-1i32).to_le_bytes());
g[0x04..0x08].copy_from_slice(&8u32.to_le_bytes());
g[0x08..0x0c].copy_from_slice(&(-1i32).to_le_bytes());
let b = LightBank::parse(&g).expect("should parse");
assert!(b.sets.is_empty());
}
#[test]
fn the_bank_offset_is_at_plus_four_not_zero() {
// A synthetic extent laid out the way the game writes it: an offset at
// +0x04, a count at that offset, then the sets at +0x10.
let mut g = vec![0u8; 0x100];
let at = 0x40usize;
g[0x04..0x08].copy_from_slice(&(at as u32).to_le_bytes());
g[at..at + 4].copy_from_slice(&1i32.to_le_bytes());
let set = |g: &mut [u8], o: usize, v: [f32; 4]| {
for (i, f) in v.iter().enumerate() {
g[o + i * 4..o + i * 4 + 4].copy_from_slice(&f.to_bits().to_le_bytes());
}
};
set(&mut g, at + 0x10, [1.0, 2.0, 3.0, 4.0]);
set(&mut g, at + 0x20, [0.0, 0.0, -1.0, 0.0]);
let b = LightBank::parse(&g).expect("should parse");
assert_eq!(b.sets.len(), 1);
assert_eq!(b.sets[0].color_a, [1.0, 2.0, 3.0, 4.0]);
assert_eq!(b.sets[0].dir_a, [0.0, 0.0, -1.0, 0.0]);
// The old reading — offset at +0, count at +4 — must not be what this
// does: at +0 the extent is 0, so it would find an empty bank.
}
}