The harness that compares the port against the game's own memory existed but had never been compiled: tools/trace did not build (27 errors, all in the argument plumbing), and the tfrag_light.rs light-bank correction it motivated was unverified. This finishes it and runs it, and the run finds two real bugs. Result, on level 0 against three inputs -- the live emulator over PINE and two PCSX2 savestates from the same level, agreeing byte for byte: headers matching the disc on invariant bytes: 833/833 data pointers relocated as table + disc offset: 833/833 light bank in RAM at 0x50ab20, byte-identical to the disc's vertices compared: 40600 (162400 colour bytes) equal: 162400 (100.00%) The locator finds the level's tfrags block in RAM by tfrag 0's bounding sphere and accepts a candidate only when every header matches the disc on the bytes the game leaves alone and every data pointer is the relocated form, so "the block was found" is itself a verification rather than an assumption. The bank is searched in RAM by its exact section bytes, which is what settles its framing: offset at +0x04, an i32 set count, twelve bytes of pad, then the sets. The previous reading (offset at +0x00, count at +0x04) was wrong. Two bugs the comparison found, both of which produced plausible output: 1. vertex_normal_bits negated x and y and left z positive, while NormalTable::normal negates all three. dA came out -0.707 where the game had +0.707, so max(dA, dA*wA) took the wrong branch and every vertex lit near its base colour: dark, but not wrong-dark. 95.88% agreement with the bug, 100.00% without. 2. A record whose select names a set the bank lacks drew transparent black. The game writes the record's base colour; 3,130 of level 0's 40,600 vertices do this (select 0x000f, set 15 -- a runtime slot the level's directional bank does not have). The vertex is unlit, not invisible. Both are now regression tests in the engine. The comparison prints the pass's per-vertex terms (light_vertex_terms) for the first mismatches, because a colour mismatch alone cannot be attributed and the comparison is worthless without attribution -- that is how the second bug was identified from a three-line read rather than guessed at. Docs: docs/plan/pcsx2-verification.md records the harness, the result, the two bugs and what to compare next; roadmap and level-assets.md stop carrying lighting magnitudes as an open item. Findings reach tests only as numbers -- no test reads a savestate, a dump or a live emulator.
403 lines
13 KiB
Rust
403 lines
13 KiB
Rust
//! `LightTfrags`: the per-vertex lighting pass the game runs on tfrag geometry.
|
|
//!
|
|
//! Spec: `docs/formats/tfrag-lighting.md`.
|
|
//!
|
|
//! Every arithmetic step goes through [`crate::ps2`]. This is not
|
|
//! fastidiousness: the FPU truncates toward zero and has no denormals, and the
|
|
//! pass accumulates across every vertex of every patch. Computing it in `f32`
|
|
//! produces a visibly different level, not a subtly different one.
|
|
|
|
use gc_formats::tfrag::TfragGeometry;
|
|
use gc_formats::tfrag_light::{DirLightSet, LightBank, NormalTable};
|
|
|
|
use crate::ps2;
|
|
|
|
/// A vector of four PS2 float bit patterns.
|
|
type V4 = [u32; 4];
|
|
|
|
fn bits(v: [f32; 4]) -> V4 {
|
|
v.map(f32::to_bits)
|
|
}
|
|
|
|
/// `vmul`: lane-wise product.
|
|
fn vmul(a: V4, b: V4) -> V4 {
|
|
[0, 1, 2, 3].map(|k| ps2::mul(a[k], b[k]))
|
|
}
|
|
|
|
/// `vmulbc`: every lane times one scalar.
|
|
fn vmuls(a: V4, s: u32) -> V4 {
|
|
a.map(|x| ps2::mul(x, s))
|
|
}
|
|
|
|
/// `vadda` / `vmadd`: `acc + a * s`, lane-wise.
|
|
fn vmadds(acc: V4, a: V4, s: u32) -> V4 {
|
|
[0, 1, 2, 3].map(|k| ps2::add(acc[k], ps2::mul(a[k], s)))
|
|
}
|
|
|
|
/// `(ax*bx + ay*by) + 1.0*(az*bz)`.
|
|
///
|
|
/// The `1.0 *` is in the microprogram: `vmaddz.x r, vf21(1.0), t`. It matters
|
|
/// only for overflow behaviour, but dropping it changes the flush case.
|
|
fn dot3(a: V4, b: V4) -> u32 {
|
|
let p = vmul(a, b);
|
|
ps2::add(ps2::add(p[0], p[1]), ps2::mul(ps2::ONE, p[2]))
|
|
}
|
|
|
|
/// A colour byte as the float the microprogram builds: `65536 + c / 128`,
|
|
/// formed by `padduw` against `0x47800000`.
|
|
fn color_float(c: [u8; 4]) -> V4 {
|
|
c.map(|b| 0x4780_0000 + b as u32)
|
|
}
|
|
|
|
/// Clamp to a byte with `vminibcx` against `0x478000ff`, then take the low
|
|
/// byte of each lane.
|
|
fn pack_color(v: V4) -> [u8; 4] {
|
|
let clamp = 0x4780_00ff;
|
|
[
|
|
ps2::min(v[0], clamp) as u8,
|
|
ps2::min(v[1], clamp) as u8,
|
|
ps2::min(v[2], clamp) as u8,
|
|
v[3] as u8,
|
|
]
|
|
}
|
|
|
|
/// PEXT5 of the 5:5:5:1 base colour.
|
|
pub fn pext5(c: u16) -> [u8; 4] {
|
|
[
|
|
((c & 0x1f) << 3) as u8,
|
|
((c >> 5 & 0x1f) << 3) as u8,
|
|
((c >> 10 & 0x1f) << 3) as u8,
|
|
((c >> 15) << 7) as u8,
|
|
]
|
|
}
|
|
|
|
/// The light set a vertex uses, with the back-face factors pulled out of the
|
|
/// colour w lanes and those lanes cleared.
|
|
struct ActiveSet {
|
|
ca: V4,
|
|
da: V4,
|
|
cb: V4,
|
|
db: V4,
|
|
wa: u32,
|
|
wb: u32,
|
|
}
|
|
|
|
/// `select` names one light set, or blends two.
|
|
///
|
|
/// Bits 8..15 clear means "just set bits 0..3". Otherwise set bits 0..3 is
|
|
/// blended with set bits 4..7 by `t = bits 8..15 / 256`.
|
|
fn active_set(bank: &LightBank, select: u16) -> Option<ActiveSet> {
|
|
let set = |i: u16| -> Option<DirLightSet> { bank.sets.get((i & 0xf) as usize).copied() };
|
|
let (mut ca, da, mut cb, db);
|
|
if select & 0xff00 == 0 {
|
|
let s = set(select)?;
|
|
(ca, da, cb, db) = (
|
|
bits(s.color_a),
|
|
bits(s.dir_a),
|
|
bits(s.color_b),
|
|
bits(s.dir_b),
|
|
);
|
|
} else {
|
|
let t = ps2::itof12(((select as u32 >> 4) & 0xff0) as i32);
|
|
let w = ps2::sub(ps2::ONE, t);
|
|
let a = set(select)?;
|
|
let b = set(select >> 4)?;
|
|
let blend = |x: [f32; 4], y: [f32; 4]| -> V4 {
|
|
let (p, q) = (vmuls(bits(x), w), vmuls(bits(y), t));
|
|
[0, 1, 2, 3].map(|k| ps2::add(p[k], q[k]))
|
|
};
|
|
ca = blend(a.color_a, b.color_a);
|
|
cb = blend(a.color_b, b.color_b);
|
|
// Blended directions are re-normalised, which the unblended path does
|
|
// not need because the stored ones are already unit.
|
|
let norm = |v: V4| -> V4 {
|
|
let q = ps2::rsqrt(ps2::ONE, dot3(v, v));
|
|
[
|
|
ps2::mul(v[0], q),
|
|
ps2::mul(v[1], q),
|
|
ps2::mul(v[2], q),
|
|
v[3],
|
|
]
|
|
};
|
|
da = norm(blend(a.dir_a, b.dir_a));
|
|
db = norm(blend(a.dir_b, b.dir_b));
|
|
}
|
|
// `vaddw.x/.y vf31 = 0 + colour.w`, then `vsubw.w colour = 1 - 1`.
|
|
let (wa, wb) = (ps2::add(0, ca[3]), ps2::add(0, cb[3]));
|
|
ca[3] = 0;
|
|
cb[3] = 0;
|
|
Some(ActiveSet {
|
|
ca,
|
|
da,
|
|
cb,
|
|
db,
|
|
wa,
|
|
wb,
|
|
})
|
|
}
|
|
|
|
/// The pass's terms for one vertex.
|
|
///
|
|
/// This exists because a colour mismatch on its own cannot be attributed: the
|
|
/// final byte is a sum of five things, and the PCSX2 comparison -- the only
|
|
/// check on this pass there is -- is worthless without knowing *which* of them
|
|
/// the game disagrees with. It is a snapshot of what the pass computed, not a
|
|
/// second implementation: `light_vertex_n` and this share one body.
|
|
#[derive(Debug, Clone, Copy, PartialEq)]
|
|
pub struct VertexTerms {
|
|
/// The record's azimuth and elevation indices.
|
|
pub azimuth: u8,
|
|
pub elevation: u8,
|
|
/// The record's base colour, PEXT5-expanded.
|
|
pub base: [u8; 4],
|
|
/// The `select` word, the set it named, and whether it blended two.
|
|
pub select: u16,
|
|
pub set_index: Option<usize>,
|
|
pub blended: bool,
|
|
/// The vertex's normal.
|
|
pub normal: [f32; 4],
|
|
/// The active set's colours and directions, with the w lanes cleared as
|
|
/// the pass clears them.
|
|
pub color_a: [f32; 4],
|
|
pub color_b: [f32; 4],
|
|
pub dir_a: [f32; 4],
|
|
pub dir_b: [f32; 4],
|
|
/// The back-face factors taken from the colour w lanes.
|
|
pub wa: f32,
|
|
pub wb: f32,
|
|
/// The two dot products, and each after its `max(d, d*w)`.
|
|
pub dot_a: f32,
|
|
pub dot_b: f32,
|
|
pub max_a: f32,
|
|
pub max_b: f32,
|
|
/// The final colour.
|
|
pub out: [u8; 4],
|
|
}
|
|
|
|
/// Light one vertex: `base + A.rgb * max(dA, dA*wA) + B.rgb * max(dB, dB*wB)`.
|
|
pub fn light_vertex(
|
|
bank: &LightBank,
|
|
n: V4,
|
|
light: &gc_formats::tfrag::TfragLight,
|
|
) -> Option<[u8; 4]> {
|
|
light_vertex_n(bank, n, light, light.select)
|
|
}
|
|
|
|
fn light_vertex_n(
|
|
bank: &LightBank,
|
|
n: V4,
|
|
light: &gc_formats::tfrag::TfragLight,
|
|
select: u16,
|
|
) -> Option<[u8; 4]> {
|
|
Some(light_vertex_terms(bank, n, light, select)?.out)
|
|
}
|
|
|
|
/// Light one vertex, keeping the terms for the comparison.
|
|
pub fn light_vertex_terms(
|
|
bank: &LightBank,
|
|
n: V4,
|
|
light: &gc_formats::tfrag::TfragLight,
|
|
select: u16,
|
|
) -> Option<VertexTerms> {
|
|
let s = active_set(bank, select)?;
|
|
let da = dot3(n, s.da);
|
|
let db = dot3(n, s.db);
|
|
let da_max = ps2::max(da, ps2::mul(da, s.wa));
|
|
let db_max = ps2::max(db, ps2::mul(db, s.wb));
|
|
let base = pext5(light.color as u16);
|
|
let acc = vmuls(color_float(base), ps2::ONE);
|
|
let acc = vmadds(acc, s.ca, da_max);
|
|
let out = pack_color(vmadds(acc, s.cb, db_max));
|
|
let unbits = |v: V4| v.map(f32::from_bits);
|
|
Some(VertexTerms {
|
|
azimuth: light.azimuth_index(),
|
|
elevation: light.elevation_index(),
|
|
base,
|
|
select,
|
|
set_index: if select & 0xff00 == 0 {
|
|
bank.sets
|
|
.get((select & 0xf) as usize)
|
|
.map(|_| (select & 0xf) as usize)
|
|
} else {
|
|
let low = bank.sets.get((select & 0xf) as usize);
|
|
let high = bank.sets.get(((select >> 4) & 0xf) as usize);
|
|
low.and(high).map(|_| (select & 0xf) as usize)
|
|
},
|
|
blended: select & 0xff00 != 0,
|
|
normal: unbits(n),
|
|
color_a: unbits(s.ca),
|
|
color_b: unbits(s.cb),
|
|
dir_a: unbits(s.da),
|
|
dir_b: unbits(s.db),
|
|
wa: f32::from_bits(s.wa),
|
|
wb: f32::from_bits(s.wb),
|
|
dot_a: f32::from_bits(da),
|
|
dot_b: f32::from_bits(db),
|
|
max_a: f32::from_bits(da_max),
|
|
max_b: f32::from_bits(db_max),
|
|
out,
|
|
})
|
|
}
|
|
|
|
/// The normal a vertex lights with, as raw PS2 floats.
|
|
///
|
|
/// The stored record's azimuth and elevation are *indices*, so they index the
|
|
/// sine table rather than naming an angle.
|
|
///
|
|
/// All three components are negated: the stored normals point into the
|
|
/// surface. The z sign is what the PCSX2 comparison settled -- negating only x
|
|
/// and y still produces a plausible level, and it cost 48 percentage points of
|
|
/// agreement on level 0 (95.88% vs 100.00%) before it was measured.
|
|
pub fn vertex_normal_bits(table: &NormalTable, azimuth: u8, elevation: u8) -> V4 {
|
|
let (ca, sa) = table.get(azimuth);
|
|
let (ce, se) = table.get(elevation);
|
|
let x = ps2::mul(ca.to_bits(), ce.to_bits());
|
|
let y = ps2::mul(sa.to_bits(), ce.to_bits());
|
|
[
|
|
ps2::sub(0, x),
|
|
ps2::sub(0, y),
|
|
ps2::sub(0, se.to_bits()),
|
|
se.to_bits(),
|
|
]
|
|
}
|
|
|
|
/// Run the lighting pass over a patch's light records.
|
|
///
|
|
/// Returns one colour per light record, in the same order. A record whose
|
|
/// `select` names a light set the bank does not have is left at its base
|
|
/// colour, which is what the game writes: the PCSX2 comparison shows those
|
|
/// records' RAM colours equal to their PEXT5 base exactly. Transparent black
|
|
/// was the first guess and it was wrong -- the vertex is unlit, not invisible.
|
|
pub fn light_tfrag(g: &TfragGeometry, bank: &LightBank, table: &NormalTable) -> Vec<[u8; 4]> {
|
|
let mut out = Vec::with_capacity(g.lights.len());
|
|
for l in &g.lights {
|
|
let n = vertex_normal_bits(table, l.azimuth_index(), l.elevation_index());
|
|
out.push(light_vertex_n(bank, n, l, l.select).unwrap_or(pext5(l.color as u16)));
|
|
}
|
|
out
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
use gc_formats::tfrag::TfragLight;
|
|
|
|
fn table() -> NormalTable {
|
|
let mut cos_sin = Vec::with_capacity(256);
|
|
for i in 0..256usize {
|
|
let a = i as f64 * std::f64::consts::PI / 128.0;
|
|
cos_sin.push([(a.cos() as f32).to_bits(), (a.sin() as f32).to_bits()]);
|
|
}
|
|
NormalTable { cos_sin }
|
|
}
|
|
|
|
fn set(color: [f32; 4], dir: [f32; 4]) -> DirLightSet {
|
|
DirLightSet {
|
|
color_a: color,
|
|
dir_a: dir,
|
|
color_b: color,
|
|
dir_b: dir,
|
|
}
|
|
}
|
|
|
|
fn bank() -> LightBank {
|
|
// One set: a white light pointing along -X. The table's azimuth 0
|
|
// normal is -X too, so a vertex with that normal faces the light and
|
|
// one with azimuth 128 faces away.
|
|
LightBank {
|
|
sets: vec![set([1.0, 1.0, 1.0, 1.0], [-1.0, 0.0, 0.0, 0.0])],
|
|
point_lights: Vec::new(),
|
|
}
|
|
}
|
|
|
|
fn light(azimuth: i8) -> TfragLight {
|
|
TfragLight {
|
|
unknown_0: 0,
|
|
intensity: 0,
|
|
azimuth,
|
|
elevation: 0,
|
|
color: 0,
|
|
select: 0,
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn pext5_expands_five_bits_to_eight() {
|
|
assert_eq!(pext5(0x0000), [0, 0, 0, 0]);
|
|
assert_eq!(pext5(0xffff), [248, 248, 248, 128]);
|
|
assert_eq!(pext5(0x001f), [248, 0, 0, 0]);
|
|
}
|
|
|
|
#[test]
|
|
fn a_facing_vertex_is_lit_and_a_back_facing_one_is_not() {
|
|
let bank = bank();
|
|
let t = table();
|
|
// Azimuth 0 gives the normal -X, which faces the light. Azimuth 128
|
|
// gives +X, which does not.
|
|
let lit = light_tfrag(&patch(vec![light(0), light(-128)]), &bank, &t);
|
|
assert_eq!(lit.len(), 2);
|
|
assert!(
|
|
lit[0][0] > 200,
|
|
"facing vertex should be bright: {:?}",
|
|
lit[0]
|
|
);
|
|
assert_eq!(
|
|
lit[1][0], 0,
|
|
"back-facing vertex should be dark: {:?}",
|
|
lit[1]
|
|
);
|
|
}
|
|
|
|
/// A patch with one light record, wired to `table` and `bank`.
|
|
fn patch(lights: Vec<TfragLight>) -> TfragGeometry {
|
|
TfragGeometry {
|
|
lights,
|
|
..TfragGeometry::default()
|
|
}
|
|
}
|
|
|
|
#[test]
|
|
fn an_empty_bank_leaves_the_base_colour() {
|
|
// The game's own answer, over PINE: a record whose select names a set
|
|
// the bank does not have keeps its base colour. Transparent black was
|
|
// the first guess, and the comparison read it as a 4.4% mismatch on
|
|
// level 0 before this was measured.
|
|
let bank = LightBank::default();
|
|
let t = table();
|
|
let mut l = light(0);
|
|
// 5:5:5:1 with the top red step and the alpha bit set: (8, 0, 0,
|
|
// 128). Stored as the i16 the record carries, expanded by PEXT5.
|
|
l.color = 0x8001u16.cast_signed();
|
|
let out = light_tfrag(&patch(vec![l]), &bank, &t);
|
|
// PEXT5 of 1 is 1 << 3 = 8, with the alpha bit at 0x80.
|
|
assert_eq!(out, vec![[8, 0, 0, 128]]);
|
|
}
|
|
|
|
#[test]
|
|
fn normals_are_negated_and_unit_length() {
|
|
let t = table();
|
|
let n = vertex_normal_bits(&t, 0, 0);
|
|
// Azimuth 0, elevation 0 is (1, 0, 0) before negation.
|
|
assert_eq!(n[0], f32::to_bits(-1.0));
|
|
assert_eq!(n[1], 0);
|
|
assert_eq!(n[2], 0);
|
|
}
|
|
|
|
#[test]
|
|
fn the_normal_s_z_is_negated_like_x_and_y() {
|
|
// Elevation 64 is straight up: (0, 0, 1) before negation, so the
|
|
// stored normal points down into the surface. Getting the z sign
|
|
// wrong still produces a plausible level -- it flips which vertices
|
|
// a light reaches -- and it cost 48 percentage points of agreement
|
|
// before the PCSX2 comparison caught it.
|
|
let t = table();
|
|
let n = vertex_normal_bits(&t, 0, 64);
|
|
let f = |b: u32| f32::from_bits(b);
|
|
// The table's cos(pi/2) is 1.5e-8 rather than exactly zero, which is
|
|
// what the game's own table holds, so x and y are ~0 rather than 0.
|
|
assert!(f(n[0]).abs() < 1e-6, "x should be ~0, got {}", f(n[0]));
|
|
assert!(f(n[1]).abs() < 1e-6, "y should be ~0, got {}", f(n[1]));
|
|
assert_eq!(n[2], f32::to_bits(-1.0));
|
|
}
|
|
}
|