Parse tfrag topology: vertex info, strips, indices and triangles
Turns tfrag positions into drawable geometry. Every unpack in a patch's VIF lists is now classified by format *and* destination VU address, using the VU header at the head of the common list. That table is what tells the strips from the indices from the parent arrays: they are all V4_8 and only the address distinguishes them. Strip indices address the vertex-info array, not the position array. Several vertex-info entries can share a position, so an index resolves through vertex_info[i].vertex / 2 to reach a position. Vertex-info also carries UVs and the LOD parent link. A strip run of n vertices is a GS triangle strip of n - 2 triangles with alternating winding. The vertex count is biased by -128 when negative, and the first record of a list is always biased even when positive -- real patches open with -116 or -124, never a small positive number. tri_count lives in the patch header and cannot be derived from the lists without walking them, so reproducing it is a strong check: all 27 levels, 18,102 patches, 809,879 triangles, every patch matching its header, every triangle index inside its LOD's vertex-info range. Four wrong turns got there, each of which failed quietly rather than loudly: reading the element size from the immediate instead of the command, 2-byte payload padding instead of 4, ignoring the inline payload of STROW/STCOL/STMASK/MPG/DIRECT, and banking strips by arrival order instead of by LOD. Every one produced parseable, plausible geometry. Only the header's own triangle count caught them. Still open: ad-gif texture bindings, per-vertex RGBA and light records, and the LOD switch and morph rules.
This commit is contained in:
@@ -487,3 +487,96 @@ fn tfrag_vertices_decode_for_every_level() {
|
||||
);
|
||||
eprintln!("decoded {total_verts} vertices from {total_tfrags} tfrags across {levels} levels");
|
||||
}
|
||||
|
||||
/// Terrain topology: strips and index arrays must expand to exactly the
|
||||
/// triangle count each patch's header claims.
|
||||
///
|
||||
/// `tri_count` is stored in the patch header but is not derivable from the
|
||||
/// lists directly, so reproducing it on all 18,102 patches is a strong check
|
||||
/// that the strip walk, the index bank and the LOD assignment are all right.
|
||||
/// Triangle indices must also stay inside the vertex-info range for the LOD.
|
||||
#[test]
|
||||
fn tfrag_topology_reproduces_the_header_triangle_count() {
|
||||
require_disc!();
|
||||
gc_formats::iso9660::IsoImage::open(std::path::Path::new(
|
||||
&std::env::var("GC_DISC").expect("GC_DISC is set by require_disc!"),
|
||||
))
|
||||
.expect("ISO should open");
|
||||
let data = std::env::var_os("GC_EXTRACTED")
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| {
|
||||
let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
|
||||
PathBuf::from(home).join("Documents/gc-extracted")
|
||||
});
|
||||
if !data.exists() {
|
||||
eprintln!("skipped: unpack the disc first");
|
||||
return;
|
||||
}
|
||||
|
||||
let mut patches = 0usize;
|
||||
let mut triangles = 0usize;
|
||||
let mut levels = 0usize;
|
||||
|
||||
for n in 0..27 {
|
||||
let dir = data.join(format!("level{n}"));
|
||||
let core = match std::fs::read(dir.join("core_data.bin")) {
|
||||
Ok(c) => c,
|
||||
Err(_) => continue,
|
||||
};
|
||||
let index = match std::fs::read(dir.join("core_index.bin")) {
|
||||
Ok(i) => i,
|
||||
Err(_) => continue,
|
||||
};
|
||||
let region =
|
||||
|off: usize| i32::from_le_bytes(index[off..off + 4].try_into().unwrap()) as usize;
|
||||
let start = region(0x08);
|
||||
let end = [0x0c_usize, 0x10, 0x14]
|
||||
.iter()
|
||||
.map(|&o| region(o))
|
||||
.filter(|&v| v > 0)
|
||||
.min()
|
||||
.unwrap_or(core.len());
|
||||
let block = &core[start..end];
|
||||
|
||||
let header = gc_formats::tfrag::TfragsBlock::parse(block).expect("tfrags block header");
|
||||
let table = header.table_offset as usize;
|
||||
for i in 0..header.tfrag_count as usize {
|
||||
let th = header.header(block, i).expect("tfrag header");
|
||||
let data_base = table.wrapping_add(th.data as usize);
|
||||
if data_base >= block.len() {
|
||||
continue;
|
||||
}
|
||||
let g = gc_formats::tfrag::parse_tfrag(&block[data_base..], &th)
|
||||
.unwrap_or_else(|e| panic!("level{n} patch {i}: {e}"));
|
||||
let tris = gc_formats::tfrag::tfrag_triangles(&g, 0)
|
||||
.unwrap_or_else(|e| panic!("level{n} patch {i}: {e}"));
|
||||
assert_eq!(
|
||||
tris.len(),
|
||||
th.tri_count as usize,
|
||||
"level{n} patch {i}: {} triangles, header says {}",
|
||||
tris.len(),
|
||||
th.tri_count
|
||||
);
|
||||
let vinfo = g.lod_vinfo_count(0);
|
||||
for t in &tris {
|
||||
let idx = [t.a as usize, t.b as usize, t.c as usize];
|
||||
assert!(
|
||||
idx.iter().all(|&v| v < vinfo),
|
||||
"level{n} patch {i}: triangle index outside {vinfo} vertex-info entries"
|
||||
);
|
||||
assert!((t.ad_gif as usize) < g.ad_gif_count);
|
||||
}
|
||||
patches += 1;
|
||||
triangles += tris.len();
|
||||
}
|
||||
levels += 1;
|
||||
}
|
||||
|
||||
assert!(levels > 0, "no unpacked levels were found");
|
||||
assert!(patches > 18_000, "expected 18k+ patches, got {patches}");
|
||||
assert!(
|
||||
triangles > 700_000,
|
||||
"expected 700k+ triangles, got {triangles}"
|
||||
);
|
||||
eprintln!("reproduced {triangles} triangles across {patches} patches in {levels} levels");
|
||||
}
|
||||
|
||||
+676
-60
@@ -27,12 +27,17 @@
|
||||
//!
|
||||
//! # What is recovered here
|
||||
//!
|
||||
//! Vertex **positions**: a V3_16 signed unpack inside the tfrag's VIF command
|
||||
//! list, giving 6 bytes per vertex as three `s16` world coordinates.
|
||||
//! Vertex **positions**, **texture coordinates** and **triangles**. Positions
|
||||
//! are a V3_16 signed unpack; texture coordinates and LOD parenting are a V4_16
|
||||
//! unpack; triangles come from the strip command lists and index arrays. All of
|
||||
//! it is classified by unpack format *and* destination VU address, using the VU
|
||||
//! header at the head of the common list.
|
||||
//!
|
||||
//! Not yet recovered: the vertex-info array (texture coordinates), the strip and
|
||||
//! index arrays that define the triangles, and the ad-gif texture bindings. See
|
||||
//! `docs/plan/tfrag.md`.
|
||||
//! Strip indices address the vertex-info array rather than the position array —
|
||||
//! several vertex-info entries can share a position.
|
||||
//!
|
||||
//! Not yet recovered: the ad-gif texture bindings themselves, and the LOD
|
||||
//! switch and morph rules in `TfragProc`. See `docs/plan/tfrag.md`.
|
||||
|
||||
use crate::buf::{Buf, Error, ErrorKind, Result};
|
||||
|
||||
@@ -250,6 +255,16 @@ impl VifCode {
|
||||
((32usize >> self.vl()) * (self.vn() as usize + 1)) / 8
|
||||
}
|
||||
|
||||
/// Destination quadword address in VU memory, bits 9..0 of the immediate.
|
||||
pub fn vu_address(&self) -> u16 {
|
||||
self.imm & 0x3ff
|
||||
}
|
||||
|
||||
/// Unpack USN bit: 1 = unsigned.
|
||||
pub fn usn(&self) -> bool {
|
||||
(self.imm >> 14) & 1 != 0
|
||||
}
|
||||
|
||||
/// The PS2 vector format name, for diagnostics.
|
||||
pub fn format_name(&self) -> String {
|
||||
const NAMES: [&str; 4] = ["V1", "V2", "V3", "V4"];
|
||||
@@ -315,77 +330,435 @@ pub const DIRECT: u8 = 0x50;
|
||||
pub const DIRECTHL: u8 = 0x51;
|
||||
|
||||
/// The VIF lists that make up a tfrag's data block, in the order the game
|
||||
/// transfers them.
|
||||
/// transfers them, each paired with the LOD it contributes to (or `None`).
|
||||
///
|
||||
/// The four offsets do not run in ascending order: the LOD-2 list sits *before*
|
||||
/// the common list. Each is a byte range from the start of the tfrag's data
|
||||
/// block. Positions appear in the common list and the two LOD-0 lists only.
|
||||
fn position_lists(h: &TfragHeader) -> [(usize, usize); 3] {
|
||||
// The LOD-0-only list starts here rather than at an offset of its own.
|
||||
/// The offsets do not run in ascending order: the LOD-2 list sits *before* the
|
||||
/// common list. The LOD-0-only list has no offset of its own, starting instead
|
||||
/// at `shared_ofs + lod_1_size * 16`. Strips and index arrays are collected as
|
||||
/// the lists are walked and banked when the LOD that owns them is complete.
|
||||
/// The byte range of one VIF list within a tfrag's data block.
|
||||
type ListRange = (usize, usize);
|
||||
|
||||
/// The VIF lists that make up a tfrag's data block, in the order the game
|
||||
/// transfers them, each paired with the LOD it completes (or `None`).
|
||||
///
|
||||
/// The offsets do not run in ascending order: the LOD-2 list sits *before* the
|
||||
/// common list. The LOD-0-only list has no offset of its own, starting instead
|
||||
/// at `shared_ofs + lod_1_size * 16`. Strips and index arrays accumulate as
|
||||
/// the lists are walked and are banked when their LOD completes.
|
||||
fn list_ranges(h: &TfragHeader) -> [(Option<usize>, ListRange); 5] {
|
||||
let lod0_start = h.shared_ofs as usize + h.lod_1_size as usize * 0x10;
|
||||
[
|
||||
(h.shared_ofs as usize, h.lod_1_ofs as usize),
|
||||
(h.lod_0_ofs as usize, lod0_start),
|
||||
(lod0_start, h.rgba_ofs as usize),
|
||||
(None, (h.shared_ofs as usize, h.lod_1_ofs as usize)),
|
||||
(Some(2), (h.lod_2_ofs as usize, h.shared_ofs as usize)),
|
||||
(Some(1), (h.lod_1_ofs as usize, h.lod_0_ofs as usize)),
|
||||
(None, (h.lod_0_ofs as usize, lod0_start)),
|
||||
(Some(0), (lod0_start, h.rgba_ofs as usize)),
|
||||
]
|
||||
}
|
||||
|
||||
/// Collect the V3_16 position payloads from one byte range.
|
||||
fn positions_in_list(block: &[u8], start: usize, end: usize, out: &mut Vec<[i32; 3]>) -> bool {
|
||||
/// The VU header: 20 `u16` describing where each kind of data is unpacked.
|
||||
///
|
||||
/// Every unpack in a tfrag's lists is classified by format *and* destination
|
||||
/// VU address, so this table is what tells the strips from the indices from
|
||||
/// the parent arrays.
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub struct TfragVuHeader {
|
||||
pub positions_common_count: u16,
|
||||
pub positions_lod_01_count: u16,
|
||||
pub positions_lod_0_count: u16,
|
||||
pub positions_common_addr: u16,
|
||||
pub vertex_info_common_addr: u16,
|
||||
pub vertex_info_lod_01_addr: u16,
|
||||
pub vertex_info_lod_0_addr: u16,
|
||||
pub indices_addr: u16,
|
||||
pub strips_addr: u16,
|
||||
}
|
||||
|
||||
impl TfragVuHeader {
|
||||
fn parse(bytes: &[u8]) -> Result<TfragVuHeader> {
|
||||
let mut b = Buf::new(bytes);
|
||||
let mut w = [0u16; 20];
|
||||
for slot in w.iter_mut() {
|
||||
*slot = b.u16("tfrag VU header")?;
|
||||
}
|
||||
Ok(TfragVuHeader {
|
||||
positions_common_count: w[0],
|
||||
positions_lod_01_count: w[2],
|
||||
positions_lod_0_count: w[4],
|
||||
positions_common_addr: w[6],
|
||||
vertex_info_common_addr: w[7],
|
||||
vertex_info_lod_01_addr: w[9],
|
||||
vertex_info_lod_0_addr: w[11],
|
||||
indices_addr: w[13],
|
||||
strips_addr: w[18],
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// Per-vertex texture and LOD data (V4_16 signed, 8 bytes).
|
||||
///
|
||||
/// Strip indices address *this* array, not the position array: several
|
||||
/// vertex-info entries can share one position.
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub struct TfragVertexInfo {
|
||||
/// Texture S, signed 1/4096 fixed point.
|
||||
pub s: i16,
|
||||
/// Texture T, signed 1/4096 fixed point.
|
||||
pub t: i16,
|
||||
/// VU quadword offset of this entry's LOD parent's position.
|
||||
pub parent: i16,
|
||||
/// VU quadword offset of this entry's own position; index = `vertex / 2`.
|
||||
pub vertex: i16,
|
||||
}
|
||||
|
||||
/// One strip command record (V4_8 signed, 4 bytes).
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub struct TfragStrip {
|
||||
/// Vertex count, biased by -128 when the record also carries an action.
|
||||
/// Zero ends the list.
|
||||
pub vertex_count_and_flag: i8,
|
||||
/// >= 0 loads an ad-gif; negative is an XGKICK ending the GS packet.
|
||||
pub end_of_packet_flag: i8,
|
||||
/// Quadword offset into the ad-gif array (ad-gif index = `z / 5`); on a
|
||||
/// kick record, -1 keeps the current texture.
|
||||
pub ad_gif_offset: i8,
|
||||
}
|
||||
|
||||
/// Strips and index bytes of one level of detail.
|
||||
#[derive(Debug, Clone, Default, PartialEq, Eq)]
|
||||
pub struct TfragLod {
|
||||
pub strips: Vec<TfragStrip>,
|
||||
/// Indices into [`TfragGeometry::vertex_info`].
|
||||
pub indices: Vec<u8>,
|
||||
}
|
||||
|
||||
/// One triangle, as vertex-info indices plus the texture primitive in force.
|
||||
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
|
||||
pub struct TfragTriangle {
|
||||
pub a: u16,
|
||||
pub b: u16,
|
||||
pub c: u16,
|
||||
pub ad_gif: u16,
|
||||
}
|
||||
|
||||
/// Everything one tfrag contributes to a draw call.
|
||||
#[derive(Debug, Clone, Default, PartialEq)]
|
||||
pub struct TfragGeometry {
|
||||
pub positions: Vec<[i32; 3]>,
|
||||
pub vertex_info: Vec<TfragVertexInfo>,
|
||||
pub origin: [i32; 4],
|
||||
/// Positions per tier, concatenated common || lod01 || lod0.
|
||||
pub positions_tier: [usize; 3],
|
||||
/// Vertex-info entries per tier, same order.
|
||||
pub vinfo_tier: [usize; 3],
|
||||
/// Index 0 is highest detail.
|
||||
pub lod: [TfragLod; 3],
|
||||
pub ad_gif_count: usize,
|
||||
}
|
||||
|
||||
impl TfragGeometry {
|
||||
/// Vertex-info entries a LOD draws from: LOD 2 = common, LOD 1 adds LOD 0+1,
|
||||
/// LOD 0 adds LOD 0.
|
||||
pub fn lod_vinfo_count(&self, lod: usize) -> usize {
|
||||
match lod {
|
||||
0 => self.vinfo_tier[0] + self.vinfo_tier[1] + self.vinfo_tier[2],
|
||||
1 => self.vinfo_tier[0] + self.vinfo_tier[1],
|
||||
_ => self.vinfo_tier[0],
|
||||
}
|
||||
}
|
||||
|
||||
/// Position index a vertex-info entry uses, as the engine resolves it.
|
||||
pub fn position_index(&self, vinfo: usize) -> usize {
|
||||
let idx = self
|
||||
.vertex_info
|
||||
.get(vinfo)
|
||||
.map_or(0, |v| v.vertex as usize / 2);
|
||||
if idx >= self.positions.len() {
|
||||
0
|
||||
} else {
|
||||
idx
|
||||
}
|
||||
}
|
||||
|
||||
/// World-space position of a vertex-info entry, `(origin + local) / 1024`.
|
||||
pub fn world_position(&self, vinfo: usize) -> [f32; 3] {
|
||||
let local = self.positions[self.position_index(vinfo)];
|
||||
[
|
||||
(self.origin[0] + local[0]) as f32 / 1024.0,
|
||||
(self.origin[1] + local[1]) as f32 / 1024.0,
|
||||
(self.origin[2] + local[2]) as f32 / 1024.0,
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse one tfrag's data block.
|
||||
///
|
||||
/// `block` starts at the tfrag's own `data` offset; every offset in `header`
|
||||
/// is relative to that.
|
||||
pub fn parse_tfrag(block: &[u8], header: &TfragHeader) -> Result<TfragGeometry> {
|
||||
let mut g = TfragGeometry::default();
|
||||
let mut vu = TfragVuHeader::default();
|
||||
let mut have_vu = false;
|
||||
let mut have_origin = false;
|
||||
let mut strips: Vec<TfragStrip> = Vec::new();
|
||||
let mut indices: Vec<u8> = Vec::new();
|
||||
|
||||
for (lod, (start, end)) in list_ranges(header) {
|
||||
// Inverted ranges are normal: RAC2 leaves the LOD-2 list empty, and a
|
||||
// level may have no LOD 0+1 geometry.
|
||||
if start >= end || end > block.len() {
|
||||
// Empty lists and inverted ranges (a level with no LOD 2) are normal.
|
||||
return start >= end || end > block.len();
|
||||
continue;
|
||||
}
|
||||
let mut pos = start;
|
||||
while pos + 4 <= end {
|
||||
let code = u32::from_le_bytes([block[pos], block[pos + 1], block[pos + 2], block[pos + 3]]);
|
||||
let packet = VifCode::decode(code);
|
||||
if pos + packet.packet_size() > end {
|
||||
return false;
|
||||
let code =
|
||||
u32::from_le_bytes([block[pos], block[pos + 1], block[pos + 2], block[pos + 3]]);
|
||||
let v = VifCode::decode(code);
|
||||
if pos + v.packet_size() > end {
|
||||
return Err(Error {
|
||||
offset: 0,
|
||||
context: "tfrag VIF packet runs past its list",
|
||||
kind: ErrorKind::OutOfBounds {
|
||||
needed: 0,
|
||||
available: 0,
|
||||
},
|
||||
});
|
||||
}
|
||||
if packet.is_unpack() && packet.vl() == 1 && packet.vn() == 2 {
|
||||
let base = pos + 4;
|
||||
for v in 0..packet.count() {
|
||||
let o = base + v * 6;
|
||||
if o + 6 > end {
|
||||
return false;
|
||||
let payload = &block[pos + 4..pos + v.packet_size()];
|
||||
|
||||
if v.cmd == STROW {
|
||||
// The origin row is the last STROW that is not the vertex-info
|
||||
// row, and it precedes every position unpack.
|
||||
if payload.len() >= 16
|
||||
&& u32::from_le_bytes([payload[0], payload[1], payload[2], payload[3]])
|
||||
!= 0x4500_0000
|
||||
{
|
||||
for (i, slot) in g.origin.iter_mut().enumerate() {
|
||||
*slot = i32::from_le_bytes(payload[i * 4..i * 4 + 4].try_into().unwrap());
|
||||
}
|
||||
out.push([
|
||||
i16::from_le_bytes([block[o], block[o + 1]]) as i32,
|
||||
i16::from_le_bytes([block[o + 2], block[o + 3]]) as i32,
|
||||
i16::from_le_bytes([block[o + 4], block[o + 5]]) as i32,
|
||||
]);
|
||||
have_origin = true;
|
||||
}
|
||||
} else if v.is_unpack() {
|
||||
let addr = v.vu_address();
|
||||
match (v.vn(), v.vl()) {
|
||||
(3, 1) if v.usn() && addr == 0 => {
|
||||
vu = TfragVuHeader::parse(payload)?;
|
||||
have_vu = true;
|
||||
}
|
||||
// Ad-gifs: 5 quadwords each.
|
||||
(3, 0) => g.ad_gif_count = v.count() / 5,
|
||||
(2, 1) => {
|
||||
let tier = if addr == vu.positions_common_addr {
|
||||
0
|
||||
} else if g.positions_tier[1] == 0
|
||||
&& g.positions_tier[2] == 0
|
||||
&& addr
|
||||
== vu
|
||||
.positions_common_addr
|
||||
.wrapping_add(2 * g.positions_tier[0] as u16)
|
||||
{
|
||||
1
|
||||
} else {
|
||||
2
|
||||
};
|
||||
g.positions_tier[tier] = v.count();
|
||||
for p3 in read_i16x3(payload, v.count())? {
|
||||
g.positions.push([p3[0] as i32, p3[1] as i32, p3[2] as i32]);
|
||||
}
|
||||
}
|
||||
pos += packet.packet_size();
|
||||
(3, 1) => {
|
||||
let tier = if addr == vu.vertex_info_common_addr {
|
||||
0
|
||||
} else if addr == vu.vertex_info_lod_01_addr {
|
||||
1
|
||||
} else {
|
||||
2
|
||||
};
|
||||
g.vinfo_tier[tier] = v.count();
|
||||
for w in read_i16(payload, v.count())? {
|
||||
g.vertex_info.push(TfragVertexInfo {
|
||||
s: w[0],
|
||||
t: w[1],
|
||||
parent: w[2],
|
||||
vertex: w[3],
|
||||
});
|
||||
}
|
||||
true
|
||||
}
|
||||
(3, 2) => {
|
||||
if !v.usn() && addr == vu.strips_addr {
|
||||
strips = read_strips(payload, v.count())?;
|
||||
} else if addr == vu.indices_addr {
|
||||
indices = payload[..v.count() * 4].to_vec();
|
||||
}
|
||||
}
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
pos += v.packet_size();
|
||||
}
|
||||
if let Some(lod) = lod {
|
||||
g.lod[lod] = TfragLod {
|
||||
strips: std::mem::take(&mut strips),
|
||||
indices: std::mem::take(&mut indices),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
// An empty tfrag legitimately has no VU header, so this only bites when
|
||||
// there is geometry that needed classifying.
|
||||
if !have_vu && header.vert_count != 0 {
|
||||
return Err(Error {
|
||||
offset: 0,
|
||||
context: "tfrag has no VU header",
|
||||
kind: ErrorKind::Invalid("tfrag has no VU header"),
|
||||
});
|
||||
}
|
||||
if header.vert_count != 0 && g.positions.len() != header.vert_count as usize {
|
||||
return Err(Error {
|
||||
offset: 0,
|
||||
context: "tfrag position count does not match its header",
|
||||
kind: ErrorKind::Invalid("tfrag position count does not match its header"),
|
||||
});
|
||||
}
|
||||
let _ = have_origin;
|
||||
Ok(g)
|
||||
}
|
||||
|
||||
/// Read `count` V3_16 elements: three `i16` each, 6 bytes per element.
|
||||
fn read_i16x3(bytes: &[u8], count: usize) -> Result<Vec<[i16; 3]>> {
|
||||
if bytes.len() < count * 6 {
|
||||
return Err(Error {
|
||||
offset: 0,
|
||||
context: "unpack payload smaller than element count",
|
||||
kind: ErrorKind::OutOfBounds {
|
||||
needed: count * 6,
|
||||
available: bytes.len(),
|
||||
},
|
||||
});
|
||||
}
|
||||
Ok((0..count)
|
||||
.map(|i| {
|
||||
let o = i * 6;
|
||||
[
|
||||
i16::from_le_bytes([bytes[o], bytes[o + 1]]),
|
||||
i16::from_le_bytes([bytes[o + 2], bytes[o + 3]]),
|
||||
i16::from_le_bytes([bytes[o + 4], bytes[o + 5]]),
|
||||
]
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
/// Read `count` V4_16 elements: four `i16` each, 8 bytes per element.
|
||||
fn read_i16(bytes: &[u8], count: usize) -> Result<Vec<[i16; 4]>> {
|
||||
if bytes.len() < count * 8 {
|
||||
return Err(Error {
|
||||
offset: 0,
|
||||
context: "unpack payload smaller than element count",
|
||||
kind: ErrorKind::OutOfBounds {
|
||||
needed: count * 8,
|
||||
available: bytes.len(),
|
||||
},
|
||||
});
|
||||
}
|
||||
Ok((0..count)
|
||||
.map(|i| {
|
||||
let o = i * 8;
|
||||
[
|
||||
i16::from_le_bytes([bytes[o], bytes[o + 1]]),
|
||||
i16::from_le_bytes([bytes[o + 2], bytes[o + 3]]),
|
||||
i16::from_le_bytes([bytes[o + 4], bytes[o + 5]]),
|
||||
i16::from_le_bytes([bytes[o + 6], bytes[o + 7]]),
|
||||
]
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
fn read_strips(bytes: &[u8], count: usize) -> Result<Vec<TfragStrip>> {
|
||||
if bytes.len() < count * 4 {
|
||||
return Err(Error {
|
||||
offset: 0,
|
||||
context: "strip array shorter than its count",
|
||||
kind: ErrorKind::OutOfBounds {
|
||||
needed: count * 4,
|
||||
available: bytes.len(),
|
||||
},
|
||||
});
|
||||
}
|
||||
Ok((0..count)
|
||||
.map(|i| TfragStrip {
|
||||
vertex_count_and_flag: bytes[i * 4] as i8,
|
||||
end_of_packet_flag: bytes[i * 4 + 1] as i8,
|
||||
ad_gif_offset: bytes[i * 4 + 2] as i8,
|
||||
})
|
||||
.collect())
|
||||
}
|
||||
|
||||
/// Expand a LOD's strips and indices into triangles.
|
||||
///
|
||||
/// Runs are GS triangle strips of `n` vertices, so each contributes `n - 2`
|
||||
/// triangles with alternating winding. Reproduces the header's `tri_count` on
|
||||
/// every retail tfrag, which is the check that the strip walk is correct.
|
||||
pub fn tfrag_triangles(g: &TfragGeometry, lod: usize) -> Result<Vec<TfragTriangle>> {
|
||||
let mut out = Vec::new();
|
||||
let l = &g.lod[lod];
|
||||
let mut cursor = 0usize;
|
||||
let mut ad_gif = 0u16;
|
||||
for (k, s) in l.strips.iter().enumerate() {
|
||||
let mut n = s.vertex_count_and_flag as i32;
|
||||
if k == 0 || n <= 0 {
|
||||
if k != 0 && n == 0 {
|
||||
break;
|
||||
}
|
||||
if k == 0 || s.end_of_packet_flag >= 0 || s.ad_gif_offset >= 0 {
|
||||
let z = s.ad_gif_offset as i32;
|
||||
if z < 0 || z % 5 != 0 || (z / 5) as usize >= g.ad_gif_count {
|
||||
return Err(Error {
|
||||
offset: 0,
|
||||
context: "strip ad-gif offset outside the ad-gif array",
|
||||
kind: ErrorKind::Invalid("strip ad-gif offset outside the ad-gif array"),
|
||||
});
|
||||
}
|
||||
ad_gif = (z / 5) as u16;
|
||||
}
|
||||
n += 128;
|
||||
}
|
||||
let n = n as usize;
|
||||
if cursor + n > l.indices.len() {
|
||||
return Err(Error {
|
||||
offset: 0,
|
||||
context: "strip runs past the index array",
|
||||
kind: ErrorKind::OutOfBounds {
|
||||
needed: 0,
|
||||
available: 0,
|
||||
},
|
||||
});
|
||||
}
|
||||
for i in 2..n {
|
||||
let (a, b, c) = (
|
||||
l.indices[cursor + i - 2] as u16,
|
||||
l.indices[cursor + i - 1] as u16,
|
||||
l.indices[cursor + i] as u16,
|
||||
);
|
||||
out.push(if i & 1 != 0 {
|
||||
TfragTriangle {
|
||||
a: b,
|
||||
b: a,
|
||||
c,
|
||||
ad_gif,
|
||||
}
|
||||
} else {
|
||||
TfragTriangle { a, b, c, ad_gif }
|
||||
});
|
||||
}
|
||||
cursor += n;
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Vertex positions of a tfrag, in game order.
|
||||
///
|
||||
/// `block` is the tfrag's data block, starting at its own `data` offset;
|
||||
/// the offsets in `header` are relative to that. `vert_count` covers all three
|
||||
/// position tiers (common, LOD 0+1 and LOD 0), so positions are gathered from
|
||||
/// each list in transfer order.
|
||||
///
|
||||
/// Returns `None` if the lists cannot be walked cleanly or do not yield
|
||||
/// exactly `vert_count` positions: a wrong count means the walk desynchronised,
|
||||
/// and returning partial geometry would hide that.
|
||||
pub fn tfrag_positions(block: &[u8], header: &TfragHeader) -> Option<Vec<[i32; 3]>> {
|
||||
if header.vert_count == 0 {
|
||||
return Some(Vec::new());
|
||||
}
|
||||
let mut out = Vec::with_capacity(header.vert_count as usize);
|
||||
for (start, end) in position_lists(header) {
|
||||
if !positions_in_list(block, start, end, &mut out) {
|
||||
return None;
|
||||
}
|
||||
}
|
||||
if out.len() != header.vert_count as usize {
|
||||
return None;
|
||||
}
|
||||
Some(out)
|
||||
parse_tfrag(block, header).ok().map(|g| g.positions)
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -490,18 +863,43 @@ mod tests {
|
||||
assert_eq!(VifCode::decode(0x1400_0000).payload_size(), 0); // MSCAL
|
||||
}
|
||||
|
||||
/// A 20-word VU header placing the positions at VU address 2.
|
||||
fn vu_header_bytes(positions: u16) -> Vec<u8> {
|
||||
let mut w = [0u16; 20];
|
||||
w[0] = positions;
|
||||
w[6] = 2; // positions_common_addr
|
||||
w[7] = 4; // vertex_info_common_addr
|
||||
let mut out = Vec::new();
|
||||
// 5 V4_16 vectors = 20 u16, unsigned (bit 14) and FLG (bit 15), to address 0.
|
||||
out.extend_from_slice(
|
||||
&VifCode {
|
||||
irq: false,
|
||||
cmd: 0x6d,
|
||||
num: 5,
|
||||
imm: 0xc000,
|
||||
}
|
||||
.to_word()
|
||||
.to_le_bytes(),
|
||||
);
|
||||
for x in w {
|
||||
out.extend_from_slice(&x.to_le_bytes());
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Build a data block holding a single V3_16 unpack in the common list.
|
||||
fn synthetic_tfrag(verts: &[[i16; 3]]) -> (Vec<u8>, TfragHeader) {
|
||||
let mut block = Vec::new();
|
||||
let n = verts.len() as u16;
|
||||
// NUM is 8 bits, so a 256-element unpack has to encode zero.
|
||||
let num = if n == 256 { 0u16 } else { n };
|
||||
block.extend_from_slice(&vu_header_bytes(n));
|
||||
block.extend_from_slice(
|
||||
&VifCode {
|
||||
irq: false,
|
||||
cmd: 0x69,
|
||||
num,
|
||||
imm: 0x8009,
|
||||
imm: 0x8000 | 2,
|
||||
}
|
||||
.to_word()
|
||||
.to_le_bytes(),
|
||||
@@ -537,7 +935,11 @@ mod tests {
|
||||
[1123, -3072, -5203],
|
||||
];
|
||||
let (block, header) = synthetic_tfrag(&verts);
|
||||
let pos = tfrag_positions(&block, &header).expect("positions should be found");
|
||||
let g = match parse_tfrag(&block, &header) {
|
||||
Ok(g) => g,
|
||||
Err(e) => panic!("parse error: {e:?}"),
|
||||
};
|
||||
let pos = g.positions.clone();
|
||||
assert_eq!(pos.len(), 4);
|
||||
for (got, want) in pos.iter().zip(verts.iter()) {
|
||||
assert_eq!(*got, [want[0] as i32, want[1] as i32, want[2] as i32]);
|
||||
@@ -550,7 +952,13 @@ mod tests {
|
||||
// +24 rather than +22.
|
||||
let verts = [[1, 2, 3], [4, 5, 6], [7, 8, 9]];
|
||||
let (block, header) = synthetic_tfrag(&verts);
|
||||
let c = VifCode::decode(u32::from_le_bytes(block[0..4].try_into().unwrap()));
|
||||
// The VU header now leads the list, so find the V3_16 word rather than
|
||||
// assuming it is first.
|
||||
let c = block
|
||||
.chunks_exact(4)
|
||||
.map(|w| VifCode::decode(u32::from_le_bytes(w.try_into().unwrap())))
|
||||
.find(|c| c.cmd == 0x69)
|
||||
.expect("fixture should hold a V3_16 unpack");
|
||||
assert_eq!(c.payload_size(), 20);
|
||||
let pos = tfrag_positions(&block, &header).expect("positions should be found");
|
||||
assert_eq!(pos.len(), 3);
|
||||
@@ -586,3 +994,211 @@ mod tests {
|
||||
assert_eq!(tfrag_positions(&block, &header), Some(Vec::new()));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod topology_tests {
|
||||
use super::*;
|
||||
|
||||
/// A common list holding only a V3_16 position unpack, plus the LOD-0
|
||||
/// strips and indices that reference it.
|
||||
fn fixture() -> (Vec<u8>, TfragHeader) {
|
||||
const INDICES: u16 = 6;
|
||||
const STRIPS: u16 = 8;
|
||||
let verts: [[i16; 3]; 6] = [
|
||||
[99, -3795, -5203],
|
||||
[1123, -3795, -5203],
|
||||
[-715, -3072, -5203],
|
||||
[1123, -3072, -5203],
|
||||
[99, -3072, -2601],
|
||||
[1123, -3072, -2601],
|
||||
];
|
||||
let mut block = Vec::new();
|
||||
|
||||
// The VU header: positions at 2, common vertex info at 4, indices at 6,
|
||||
// strips at 8. Everything else is classified against these addresses.
|
||||
let mut w = [0u16; 20];
|
||||
w[0] = 6; // positions_common_count
|
||||
w[6] = 2; // positions_common_addr
|
||||
w[7] = 4; // vertex_info_common_addr
|
||||
w[13] = INDICES;
|
||||
w[18] = STRIPS;
|
||||
block.extend_from_slice(
|
||||
&VifCode {
|
||||
irq: false,
|
||||
cmd: 0x6d,
|
||||
num: 5,
|
||||
imm: 0xc000,
|
||||
}
|
||||
.to_word()
|
||||
.to_le_bytes(),
|
||||
);
|
||||
for x in w {
|
||||
block.extend_from_slice(&x.to_le_bytes());
|
||||
}
|
||||
|
||||
// Two ad-gifs (5 quadwords each), so strip offsets 0 and 5 are valid.
|
||||
block.extend_from_slice(
|
||||
&VifCode {
|
||||
irq: false,
|
||||
cmd: 0x6c,
|
||||
num: 10,
|
||||
imm: 0x8000,
|
||||
}
|
||||
.to_word()
|
||||
.to_le_bytes(),
|
||||
);
|
||||
block.resize(block.len() + 160, 0);
|
||||
|
||||
// Common vertex info: 6 entries, each pointing at its own position.
|
||||
block.extend_from_slice(
|
||||
&VifCode {
|
||||
irq: false,
|
||||
cmd: 0x6d,
|
||||
num: 6,
|
||||
imm: 0x8000 | 4,
|
||||
}
|
||||
.to_word()
|
||||
.to_le_bytes(),
|
||||
);
|
||||
for i in 0..6u16 {
|
||||
for c in [i * 8, i * 8, 0, i * 2] {
|
||||
block.extend_from_slice(&c.to_le_bytes());
|
||||
}
|
||||
}
|
||||
|
||||
// Positions.
|
||||
block.extend_from_slice(
|
||||
&VifCode {
|
||||
irq: false,
|
||||
cmd: 0x69,
|
||||
num: 6,
|
||||
imm: 0x8000 | 2,
|
||||
}
|
||||
.to_word()
|
||||
.to_le_bytes(),
|
||||
);
|
||||
for v in verts {
|
||||
for c in v {
|
||||
block.extend_from_slice(&c.to_le_bytes());
|
||||
}
|
||||
}
|
||||
// Pad the common list to a quadword boundary so the LOD-0 ranges are
|
||||
// well formed, as they are on disc.
|
||||
let common_end = block.len().div_ceil(16) * 16;
|
||||
block.resize(common_end, 0);
|
||||
|
||||
// LOD-0 list: strips then indices.
|
||||
block.extend_from_slice(
|
||||
&VifCode {
|
||||
irq: false,
|
||||
cmd: 0x6e,
|
||||
num: 3,
|
||||
imm: STRIPS,
|
||||
}
|
||||
.to_word()
|
||||
.to_le_bytes(),
|
||||
);
|
||||
for s in [
|
||||
// The first record always carries the -128 bias, as on disc.
|
||||
TfragStrip {
|
||||
vertex_count_and_flag: -124,
|
||||
end_of_packet_flag: 0,
|
||||
ad_gif_offset: 0,
|
||||
},
|
||||
TfragStrip {
|
||||
vertex_count_and_flag: -124,
|
||||
end_of_packet_flag: 0,
|
||||
ad_gif_offset: 5,
|
||||
},
|
||||
TfragStrip {
|
||||
vertex_count_and_flag: 0,
|
||||
end_of_packet_flag: -1,
|
||||
ad_gif_offset: -1,
|
||||
},
|
||||
] {
|
||||
block.extend_from_slice(&[
|
||||
s.vertex_count_and_flag as u8,
|
||||
s.end_of_packet_flag as u8,
|
||||
s.ad_gif_offset as u8,
|
||||
0,
|
||||
]);
|
||||
}
|
||||
let indices_at = block.len();
|
||||
block.extend_from_slice(
|
||||
&VifCode {
|
||||
irq: false,
|
||||
cmd: 0x6e,
|
||||
num: 8,
|
||||
imm: 0x4000 | INDICES,
|
||||
}
|
||||
.to_word()
|
||||
.to_le_bytes(),
|
||||
);
|
||||
block.extend_from_slice(&[0u8, 1, 2, 3, 4, 5, 4, 3]);
|
||||
// 8 V4_8 elements is a 32-byte payload; the rest is padding.
|
||||
block.resize(indices_at + 4 + 32, 0);
|
||||
let lod0_end = block.len().div_ceil(4) * 4;
|
||||
block.resize(lod0_end, 0);
|
||||
|
||||
let mut h = TfragHeader::parse(&[0u8; TFRAG_HEADER_SIZE]).unwrap();
|
||||
h.vert_count = 6;
|
||||
h.tri_count = 4; // two runs of 4 vertices
|
||||
h.shared_ofs = 0;
|
||||
h.lod_1_ofs = common_end as u16;
|
||||
h.lod_0_ofs = common_end as u16;
|
||||
h.lod_1_size = (common_end / 16) as u8;
|
||||
h.rgba_ofs = lod0_end as u16;
|
||||
(block, h)
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn expands_strips_into_triangles() {
|
||||
let (block, header) = fixture();
|
||||
let g = match parse_tfrag(&block, &header) {
|
||||
Ok(g) => g,
|
||||
Err(e) => panic!("parse error: {e:?}"),
|
||||
};
|
||||
assert_eq!(g.positions.len(), 6);
|
||||
assert_eq!(g.positions[0], [99, -3795, -5203]);
|
||||
assert_eq!(g.vertex_info.len(), 6);
|
||||
assert_eq!(g.vertex_info[3].vertex, 6);
|
||||
let tris = tfrag_triangles(&g, 0).expect("triangles should expand");
|
||||
// 4-vertex runs give 2 triangles each; the -124 record is 4 after bias.
|
||||
assert_eq!(tris.len(), 4);
|
||||
assert_eq!((tris[0].a, tris[0].b, tris[0].c), (0, 1, 2));
|
||||
assert_eq!((tris[1].a, tris[1].b, tris[1].c), (2, 1, 3));
|
||||
assert_eq!((tris[2].a, tris[2].b, tris[2].c), (4, 5, 4));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_vertex_info_entry_resolves_to_a_position() {
|
||||
let (block, header) = fixture();
|
||||
let g = parse_tfrag(&block, &header).unwrap();
|
||||
assert_eq!(g.position_index(3), 3);
|
||||
assert_eq!(g.lod_vinfo_count(0), 6);
|
||||
assert_eq!(
|
||||
g.world_position(0),
|
||||
[99.0 / 1024.0, -3795.0 / 1024.0, -5203.0 / 1024.0]
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_truncated_list_is_an_error() {
|
||||
let (mut block, header) = fixture();
|
||||
// Claim the common list runs past the block.
|
||||
let mut bad = header;
|
||||
bad.lod_1_ofs = 0x7ff0;
|
||||
assert!(parse_tfrag(&block, &bad).is_err());
|
||||
block.clear();
|
||||
assert!(parse_tfrag(&block, &bad).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_strip_past_its_index_array_is_an_error() {
|
||||
let (block, header) = fixture();
|
||||
let mut g = parse_tfrag(&block, &header).unwrap();
|
||||
// Strip declares more vertices than the index array holds.
|
||||
g.lod[0].indices.truncate(2);
|
||||
assert!(tfrag_triangles(&g, 0).is_err());
|
||||
}
|
||||
}
|
||||
|
||||
@@ -313,7 +313,52 @@ mismatch means the walk desynchronised, so `tfrag_positions` returns `None`
|
||||
rather than partial geometry. Across all 27 levels: 18,102 patches,
|
||||
821,439 vertices, zero mismatches.
|
||||
|
||||
### Topology: strips, indices and triangles
|
||||
|
||||
Also settled. `parse_tfrag` walks each list and classifies every unpack by
|
||||
format *and* destination VU address, using the VU header at the head of the
|
||||
common list (5 V4_16 vectors, unsigned, to address 0). That table is what
|
||||
separates the strips from the indices from the parent arrays — they are all
|
||||
V4_8, and only the address tells them apart.
|
||||
|
||||
| `(vn, vl)` | Meaning |
|
||||
|---|---|
|
||||
| (3, 1), unsigned, addr 0 | the VU header itself |
|
||||
| (3, 0) | ad-gifs, 5 quadwords each |
|
||||
| (2, 1) | positions, V3_16 |
|
||||
| (3, 1) | vertex info: UV, LOD parent, own position |
|
||||
| (3, 2) | strips (signed) and indices (unsigned) |
|
||||
|
||||
**Strip indices address the vertex-info array, not the position array.** Several
|
||||
vertex-info entries can share a position, so a strip index resolves through
|
||||
`vertex_info[i].vertex / 2` to reach a position.
|
||||
|
||||
A strip record is four bytes: a vertex count, a packet flag and an ad-gif
|
||||
offset. A negative count is biased by −128, and *the first record of a list is
|
||||
always biased* even when its count is positive — a real patch opens with −116 or
|
||||
−124, never a small positive number. A zero count ends the list. A run of `n`
|
||||
vertices is a GS triangle strip of `n − 2` triangles with alternating winding.
|
||||
|
||||
**Bank the LODs by index, not by arrival.** The lists are walked in transfer
|
||||
order but the strip and index arrays have to be stored by *which LOD they
|
||||
belong to*. Appending them as they are found fills the arrays LOD-2-first,
|
||||
which looks fine until the triangle counts come out low.
|
||||
|
||||
### Verification
|
||||
|
||||
`tri_count` is stored in the patch header and cannot be derived from the lists
|
||||
without walking them, so reproducing it is a strong check. All 27 levels:
|
||||
**18,102 patches, 809,879 triangles, every patch matching its header**, with
|
||||
every triangle index inside that LOD's vertex-info range and every ad-gif index
|
||||
inside the ad-gif array.
|
||||
|
||||
Getting there took four wrong turns, each of which failed quietly: reading the
|
||||
element size from the immediate, 2-byte payload padding, ignoring non-unpack
|
||||
payloads, and banking strips by arrival order. Every one produced parseable,
|
||||
plausible geometry; only the header's own triangle count caught them.
|
||||
|
||||
### Still open
|
||||
|
||||
Strips and triangle indices, the RGBA vertex-info arrays, texture binding, and
|
||||
the LOD/morph/cull rules in `TfragProc`. Positions alone are not drawable.
|
||||
The ad-gif texture bindings (which texture each primitive samples), the RGBA
|
||||
per-vertex colours and light records, and the LOD switch and morph rules in
|
||||
`TfragProc`. Positions and topology are drawable but not yet textured.
|
||||
|
||||
+9
-3
@@ -20,7 +20,7 @@ early, while they are still cheap to retire.
|
||||
header, and the `data` extent is parsed as a container of byte ranges. The
|
||||
geometry decompresses: level 0 yields 16,969,296 bytes of level content
|
||||
from a 9.7 MB stream.
|
||||
- [ ] Next: tfrag triangle topology, then the geometry directory's class tables.
|
||||
- [ ] Next: tfrag texture bindings, then the geometry directory's class tables.
|
||||
|
||||
The disc being the exact build the reference projects pin is the single
|
||||
biggest favourable fact so far: no disc conversion, no regional timing work,
|
||||
@@ -122,8 +122,14 @@ Phase 1 hands over the extents; this phase works out what each one contains.
|
||||
command byte and every element size follows from it. Verified on all 27
|
||||
levels: **18,102 patches, 821,439 vertices**, each patch yielding exactly
|
||||
its own `vert_count`. See `plan/level-assets.md` and `gc-formats::tfrag`.
|
||||
- [ ] tfrag topology: strips, triangle indices and the RGBA vertex-info arrays,
|
||||
which is what turns positions into drawable geometry.
|
||||
- [x] **tfrag topology: recovered.** Vertex-info (UVs plus LOD parenting), the
|
||||
strip command lists and the index arrays, classified by unpack format and
|
||||
destination VU address. Strip indices address the vertex-info array
|
||||
rather than positions. Every patch's strips expand to exactly its own
|
||||
`tri_count`: **18,102 patches, 809,879 triangles**, all indices in range.
|
||||
See `plan/level-assets.md` and `gc-formats::tfrag`.
|
||||
- [ ] tfrag shading inputs: the ad-gif texture bindings, per-vertex RGBA and
|
||||
light records, then the LOD switch and morph rules in `TfragProc`.
|
||||
- [ ] `gameplay` — begins `WAD` + a per-level byte, and does *not* decode with
|
||||
the WAD LZ77 decoder. All 27 levels fail identically, so the input is
|
||||
wrong rather than the decoder. Ruled out: bit order, header length, the
|
||||
|
||||
Reference in New Issue
Block a user