Decode tfrag texture bindings, vertex colours and light records

Adds level_core.rs (the uncompressed core index header and texture table)
and extends tfrag.rs with the data a draw call needs beyond geometry.

An ad-gif is five A+D quadwords holding packed fields rather than GS
registers; the level load rewrites them in place before drawing. On disc
TEX0_1.data_lo indexes the level's texture table, TEX1_1.data_lo is the
LOD bias K and data_hi the minification filter, and CLAMP_1 holds both
wrap modes.

Three things are easy to get wrong here:

* K is a signed 12-bit value in 1/16 units, sign-extended by shifting the
  field up 4 and back down. Retail values are -137..-89 mip-scaled;
  reading it unsigned gives 1911 instead of -137.
* TEX1_1.data_hi is MMIN, and 4 decodes as LinearMipmapNearest, not
  LinearMipmapLinear as the raw bit layout suggests.
* RGBA and the light/normal records are not in the VIF lists at all. They
  sit at fixed offsets in the patch data block: rgba_size*4 colours at
  rgba_ofs, vert_count light records at light_ofs+0x10. The tfrag origin
  lives there too -- if no STROW row supplied one, the quadword at
  light_ofs is it.

The texture table checks out on its own terms: 93 entries at index offset
0x2900 on level 0, and 0x2900 + 93*16 lands exactly on the next range.

Verification: an ad-gif index outside the level's table would mean the
geometry and the textures were misread independently, so every one is
range-checked against real entries. All 27 levels: 47,950 ad-gifs over
18,102 patches, all in range, referencing 1429 of 1595 level texture
entries (90%). Each patch's light array is exactly vert_count records and
its RGBA array exactly rgba_size*4.

Still open: LOD selection and morphing, and the texel data itself.
This commit is contained in:
2026-10-10 02:22:51 +02:00
parent 9459f15329
commit 48d9506cf1
6 changed files with 829 additions and 15 deletions
+125 -1
View File
@@ -564,7 +564,7 @@ fn tfrag_topology_reproduces_the_header_triangle_count() {
idx.iter().all(|&v| v < vinfo), idx.iter().all(|&v| v < vinfo),
"level{n} patch {i}: triangle index outside {vinfo} vertex-info entries" "level{n} patch {i}: triangle index outside {vinfo} vertex-info entries"
); );
assert!((t.ad_gif as usize) < g.ad_gif_count); assert!((t.ad_gif as usize) < g.ad_gifs.len());
} }
patches += 1; patches += 1;
triangles += tris.len(); triangles += tris.len();
@@ -580,3 +580,127 @@ fn tfrag_topology_reproduces_the_header_triangle_count() {
); );
eprintln!("reproduced {triangles} triangles across {patches} patches in {levels} levels"); eprintln!("reproduced {triangles} triangles across {patches} patches in {levels} levels");
} }
/// Texture bindings: every ad-gif a patch uses must index an entry that
/// actually exists in the level's texture table, and the light array must be
/// one record per vertex.
///
/// The texture table is the join between tfrags and the texel data, so an
/// out-of-range index here means the level's geometry and its textures have
/// been misread independently of each other.
#[test]
fn tfrag_ad_gifs_index_the_level_texture_table() {
require_disc!();
gc_formats::iso9660::IsoImage::open(std::path::Path::new(
&std::env::var("GC_DISC").expect("GC_DISC is set by require_disc!"),
))
.expect("ISO should open");
let data = std::env::var_os("GC_EXTRACTED")
.map(PathBuf::from)
.unwrap_or_else(|| {
let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
PathBuf::from(home).join("Documents/gc-extracted")
});
if !data.exists() {
eprintln!("skipped: unpack the disc first");
return;
}
let mut patches = 0usize;
let mut ad_gifs = 0usize;
let mut textures_seen = 0usize;
let mut textures_total = 0usize;
let mut levels = 0usize;
for n in 0..27 {
let dir = data.join(format!("level{n}"));
let core = match std::fs::read(dir.join("core_data.bin")) {
Ok(c) => c,
Err(_) => continue,
};
let index = match std::fs::read(dir.join("core_index.bin")) {
Ok(i) => i,
Err(_) => continue,
};
let ch = gc_formats::level_core::LevelCoreHeader::parse(&index).expect("core header");
let textures = gc_formats::level_core::texture_table(&index, ch.tfrag_textures)
.expect("texture table");
assert!(!textures.is_empty(), "level{n}: no tfrag textures");
// Indices are per-level, so the pair is what identifies a texture.
let mut level_seen = std::collections::BTreeSet::new();
let region =
|off: usize| i32::from_le_bytes(index[off..off + 4].try_into().unwrap()) as usize;
let start = region(0x08);
let end = [0x0c_usize, 0x10, 0x14]
.iter()
.map(|&o| region(o))
.filter(|&v| v > 0)
.min()
.unwrap_or(core.len());
let block = &core[start..end];
let header = gc_formats::tfrag::TfragsBlock::parse(block).expect("tfrags block header");
let table = header.table_offset as usize;
for i in 0..header.tfrag_count as usize {
let th = header.header(block, i).expect("tfrag header");
let data_base = table.wrapping_add(th.data as usize);
if data_base >= block.len() {
continue;
}
let g = gc_formats::tfrag::parse_tfrag(&block[data_base..], &th)
.unwrap_or_else(|e| panic!("level{n} patch {i}: {e}"));
assert_eq!(
g.lights.len(),
th.vert_count as usize,
"level{n} patch {i}: {} light records for {} vertices",
g.lights.len(),
th.vert_count
);
assert_eq!(
g.rgba.len(),
th.rgba_size as usize * 4,
"level{n} patch {i}: rgba"
);
for a in &g.ad_gifs {
let t = a.texture_index();
assert!(
t < textures.len(),
"level{n} patch {i}: texture {} outside the {} entry table",
t,
textures.len()
);
let tex = &textures[t];
assert!(
tex.width > 0 && tex.height > 0 && tex.width <= 256 && tex.height <= 256,
"level{n} patch {i}: texture {t} is {}x{}",
tex.width,
tex.height
);
level_seen.insert(t);
ad_gifs += 1;
}
patches += 1;
}
textures_seen += level_seen.len();
textures_total += textures.len();
levels += 1;
}
assert!(levels > 0, "no unpacked levels were found");
assert!(patches > 18_000, "expected 18k+ patches, got {patches}");
// Most of each level's texture table should be referenced by some tfrag. A
// handful genuinely go unused; if that share grows, the ad-gif walk has
// started missing primitives rather than the data having spare entries.
let covered = textures_seen as f64 / textures_total as f64;
assert!(
covered > 0.85,
"only {textures_seen} of {textures_total} level textures referenced ({:.1}%)",
covered * 100.0
);
eprintln!(
"{ad_gifs} ad-gifs over {patches} patches reference {textures_seen}/{textures_total} level textures"
);
}
+274
View File
@@ -0,0 +1,274 @@
//! The level core **index**: the uncompressed directory that sits alongside
//! `core_data` and points into it.
//!
//! Spec: `docs/formats/level-core.md`.
//!
//! The index is small (about 31 KB) and mostly a set of count/offset pairs. Two
//! things live here that the compressed data cannot supply: where each block of
//! `core_data` starts, and the per-level tables the geometry refers to by
//! index — most importantly the texture table that tfrag ad-gifs point into.
//!
//! Only the fields this crate actually reads are decoded. The class tables at
//! 0x18..0x30 resisted every field order and stride tried and are deliberately
//! left alone; see `docs/plan/level-assets.md`.
use crate::buf::{Buf, Error, ErrorKind, Result};
/// A count/offset pair naming an array in the index.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct ArrayRange {
pub count: i32,
pub offset: i32,
}
/// One texture in the level's texture table (16 bytes).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct TextureEntry {
/// Byte offset of the texel data within the level's texture base.
pub data_offset: i32,
pub width: i16,
pub height: i16,
/// Mip level count (3 or 4 on retail data).
pub ty: i16,
/// CLUT index within the level's palette space.
pub palette: i16,
/// Byte offset of mip level 1.
pub mipmap: i16,
/// Byte offset of the second mip chain.
pub pad: i16,
}
impl TextureEntry {
pub fn parse(bytes: &[u8]) -> Result<TextureEntry> {
let mut b = Buf::new(bytes);
Ok(TextureEntry {
data_offset: b.i32("texture data offset")?,
width: b.i16("texture width")?,
height: b.i16("texture height")?,
ty: b.i16("texture ty")?,
palette: b.i16("texture palette")?,
mipmap: b.i16("texture mipmap")?,
pad: b.i16("texture pad")?,
})
}
/// `ceil(log2(n))`, the GS TBW/TSB derivation.
///
/// PS2 texture widths are `width << 6` texels wide, so a 128 px texture is
/// 8192 texels and needs 13.
pub fn log2_size(&self) -> i32 {
let n = (self.width as i32) << 6;
let mut bits = 0;
while (1 << bits) < n.max(1) {
bits += 1;
}
bits
}
}
/// Parse a texture table.
pub fn texture_table(index: &[u8], range: ArrayRange) -> Result<Vec<TextureEntry>> {
let count = range.count.max(0) as usize;
let offset = range.offset.max(0) as usize;
// Bound the whole table before slicing, so an absurd count reports an error
// instead of panicking partway through.
let need = count
.checked_mul(16)
.and_then(|n| offset.checked_add(n))
.ok_or(Error {
offset,
context: "texture table range overflows",
kind: ErrorKind::InvalidLength("texture table"),
})?;
if need > index.len() {
return Err(Error {
offset,
context: "texture table runs past the index",
kind: ErrorKind::OutOfBounds {
needed: need,
available: index.len(),
},
});
}
let mut out = Vec::with_capacity(count);
for i in 0..count {
let at = offset + i * 16;
out.push(TextureEntry::parse(&index[at..at + 16])?);
}
Ok(out)
}
/// The level core index header.
///
/// Only fields this crate reads are decoded. Offsets are byte offsets into the
/// uncompressed index.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct LevelCoreHeader {
/// GS packet memory in use.
pub gs_ram: ArrayRange,
/// Byte offset of the tfrags block within `core_data`.
pub tfrags: i32,
/// Byte offset of the occlusion block.
pub occlusion: i32,
/// Byte offset of the sky block.
pub sky: i32,
/// Byte offset of the collision block.
pub collision: i32,
/// The table tfrag ad-gifs index into.
pub tfrag_textures: ArrayRange,
/// Byte offset of texture pixel data within `core_data`.
pub textures_base: i32,
}
impl LevelCoreHeader {
pub fn parse(bytes: &[u8]) -> Result<LevelCoreHeader> {
let mut b = Buf::new(bytes);
let pair = |b: &mut Buf, what: &'static str| -> Result<ArrayRange> {
let count = b.i32(what)?;
let offset = b.i32(what)?;
Ok(ArrayRange { count, offset })
};
let gs_ram = pair(&mut b, "gs_ram")?;
let tfrags = b.i32("tfrags")?;
let occlusion = b.i32("occlusion")?;
let sky = b.i32("sky")?;
let collision = b.i32("collision")?;
// 0x18..0x30: moby/tie/shrub/gadget class tables, not decoded yet.
b.skip(0x30 - 0x18, "class tables")?;
let tfrag_textures = pair(&mut b, "tfrag textures")?;
b.skip(0x60 - 0x38, "texture tables")?;
let textures_base = b.i32("textures base")?;
Ok(LevelCoreHeader {
gs_ram,
tfrags,
occlusion,
sky,
collision,
tfrag_textures,
textures_base,
})
}
}
#[cfg(test)]
mod tests {
use super::*;
/// A header as measured on level 0.
fn header() -> Vec<u8> {
let mut v = vec![0u8; 0x64];
v[0x00..0x04].copy_from_slice(&404i32.to_le_bytes());
v[0x04..0x08].copy_from_slice(&18_704i32.to_le_bytes());
v[0x08..0x0c].copy_from_slice(&0i32.to_le_bytes());
v[0x0c..0x10].copy_from_slice(&2_117_888i32.to_le_bytes());
v[0x10..0x14].copy_from_slice(&2_192_576i32.to_le_bytes());
v[0x14..0x18].copy_from_slice(&2_615_232i32.to_le_bytes());
v[0x30..0x34].copy_from_slice(&93i32.to_le_bytes());
v[0x34..0x38].copy_from_slice(&10_512i32.to_le_bytes());
v[0x60..0x64].copy_from_slice(&3_518_848i32.to_le_bytes());
v
}
#[test]
fn parses_the_measured_header() {
let h = LevelCoreHeader::parse(&header()).unwrap();
assert_eq!(
h.gs_ram,
ArrayRange {
count: 404,
offset: 18_704
}
);
assert_eq!(h.tfrags, 0);
assert_eq!(h.occlusion, 2_117_888);
assert_eq!(h.sky, 2_192_576);
assert_eq!(h.collision, 2_615_232);
assert_eq!(
h.tfrag_textures,
ArrayRange {
count: 93,
offset: 10_512
}
);
assert_eq!(h.textures_base, 3_518_848);
}
#[test]
fn a_short_header_is_an_error() {
assert!(LevelCoreHeader::parse(&[0u8; 0x20]).is_err());
}
#[test]
fn parses_a_texture_table() {
let mut index = vec![0u8; 10_512 + 2 * 16];
let mut put = |at: usize, e: TextureEntry| {
index[at..at + 4].copy_from_slice(&e.data_offset.to_le_bytes());
index[at + 4..at + 6].copy_from_slice(&e.width.to_le_bytes());
index[at + 6..at + 8].copy_from_slice(&e.height.to_le_bytes());
index[at + 8..at + 10].copy_from_slice(&e.ty.to_le_bytes());
index[at + 10..at + 12].copy_from_slice(&e.palette.to_le_bytes());
index[at + 12..at + 14].copy_from_slice(&e.mipmap.to_le_bytes());
};
put(
10_512,
TextureEntry {
data_offset: 0,
width: 128,
height: 128,
ty: 3,
palette: 8,
mipmap: 12,
pad: -1,
},
);
put(
10_528,
TextureEntry {
data_offset: 20_480,
width: 64,
height: 64,
ty: 3,
palette: 60,
mipmap: 64,
pad: -1,
},
);
let t = texture_table(
&index,
ArrayRange {
count: 2,
offset: 10_512,
},
)
.unwrap();
assert_eq!(t.len(), 2);
assert_eq!(t[0].width, 128);
assert_eq!(t[1].mipmap, 64);
// Widths are `width << 6` texels, so 128 px needs log2(8192) = 13.
assert_eq!(t[0].log2_size(), 13);
assert_eq!(t[1].log2_size(), 12);
}
#[test]
fn a_table_running_past_the_index_is_an_error() {
let index = vec![0u8; 64];
assert!(texture_table(
&index,
ArrayRange {
count: 8,
offset: 60
}
)
.is_err());
assert!(texture_table(
&index,
ArrayRange {
count: -1,
offset: 0
}
)
.unwrap()
.is_empty());
}
}
+1
View File
@@ -1,6 +1,7 @@
pub mod buf; pub mod buf;
pub mod global_wad; pub mod global_wad;
pub mod iso9660; pub mod iso9660;
pub mod level_core;
pub mod level_data; pub mod level_data;
pub mod level_wad; pub mod level_wad;
pub mod tfrag; pub mod tfrag;
+381 -8
View File
@@ -36,8 +36,12 @@
//! Strip indices address the vertex-info array rather than the position array — //! Strip indices address the vertex-info array rather than the position array —
//! several vertex-info entries can share a position. //! several vertex-info entries can share a position.
//! //!
//! Not yet recovered: the ad-gif texture bindings themselves, and the LOD //! Also recovered: the ad-gif texture primitives (texture index, wrap modes,
//! switch and morph rules in `TfragProc`. See `docs/plan/tfrag.md`. //! minification filter, LOD bias), the per-vertex colours and the per-vertex
//! light and normal records.
//!
//! Not yet recovered: the LOD switch and morph rules in `TfragProc`, and the
//! texel data the texture table points into. See `docs/plan/tfrag.md`.
use crate::buf::{Buf, Error, ErrorKind, Result}; use crate::buf::{Buf, Error, ErrorKind, Result};
@@ -442,6 +446,144 @@ pub struct TfragTriangle {
pub ad_gif: u16, pub ad_gif: u16,
} }
/// One GIF A+D quadword (16 bytes) from an ad-gif.
///
/// On disc the register values are packed fields, not GS registers; the level
/// load rewrites them in place before drawing.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct AdGif {
pub data_lo: i32,
pub data_hi: i32,
/// GS register address: 0x06 TEX0, 0x14 TEX1, 0x08 CLAMP, 0x34/0x36 MIPTBP.
pub address: u8,
}
/// A texture primitive: five A+D quadwords (0x50 bytes).
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct TfragAdGifs {
/// TEX0_1. On disc `data_lo` indexes the level's tfrag texture table.
pub tex0: AdGif,
/// TEX1_1. On disc `data_lo` is the LOD K in 1/16 units, `data_hi` the MMIN filter.
pub tex1: AdGif,
/// CLAMP_1. On disc `data_lo` is WMS and `data_hi` is WMT.
pub clamp: AdGif,
pub miptbp1: AdGif,
pub miptbp2: AdGif,
}
/// GS CLAMP_1 wrap mode. Retail tfrags only use Repeat and Clamp.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum GsWrap {
Repeat,
Clamp,
RegionClamp,
RegionRepeat,
}
impl GsWrap {
fn from_bits(v: i32) -> GsWrap {
match v & 3 {
0 => GsWrap::Repeat,
1 => GsWrap::Clamp,
2 => GsWrap::RegionClamp,
_ => GsWrap::RegionRepeat,
}
}
}
/// GS TEX1_1 magnification/minification filter.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum GsFilter {
Nearest,
Linear,
NearestMipmapNearest,
NearestMipmapLinear,
LinearMipmapNearest,
LinearMipmapLinear,
}
impl GsFilter {
fn from_bits(v: i32) -> GsFilter {
match v & 7 {
0 => GsFilter::Nearest,
1 => GsFilter::Linear,
2 => GsFilter::NearestMipmapNearest,
3 => GsFilter::NearestMipmapLinear,
4 => GsFilter::LinearMipmapNearest,
_ => GsFilter::LinearMipmapLinear,
}
}
}
impl TfragAdGifs {
fn parse(bytes: &[u8]) -> TfragAdGifs {
let q = |i: usize| AdGif {
data_lo: i32::from_le_bytes(bytes[i..i + 4].try_into().unwrap()),
data_hi: i32::from_le_bytes(bytes[i + 4..i + 8].try_into().unwrap()),
address: bytes[i + 8],
};
TfragAdGifs {
tex0: q(0),
tex1: q(16),
clamp: q(32),
miptbp1: q(48),
miptbp2: q(64),
}
}
/// Index into the level's tfrag texture table.
pub fn texture_index(&self) -> usize {
self.tex0.data_lo.max(0) as usize
}
/// Horizontal wrap mode.
pub fn wrap_s(&self) -> GsWrap {
GsWrap::from_bits(self.clamp.data_lo)
}
/// Vertical wrap mode.
pub fn wrap_t(&self) -> GsWrap {
GsWrap::from_bits(self.clamp.data_hi)
}
/// Minification filter, from the on-disc MMIN field.
pub fn min_filter(&self) -> GsFilter {
GsFilter::from_bits(self.tex1.data_hi)
}
/// TEX1_1 K field as the signed 12-bit raw value, 1/16 units.
pub fn lod_k_raw(&self) -> i16 {
((self.tex1.data_lo as u32 & 0xfff) << 4) as u16 as i16 >> 4
}
/// TEX1_1 K in mip levels. The GS computes `log2(1 / |Q|) + K`.
pub fn lod_k(&self) -> f32 {
self.lod_k_raw() as f32 / 16.0
}
}
/// Per-vertex colour (4 bytes), indexed by position index. 0x80 is 1.0.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct TfragRgba {
pub r: u8,
pub g: u8,
pub b: u8,
/// 0x80 is opaque.
pub a: u8,
}
/// Per-vertex light and normal record (8 bytes), indexed by position index.
#[derive(Debug, Clone, Copy, Default, PartialEq, Eq)]
pub struct TfragLight {
pub unknown_0: i8,
pub intensity: i8,
/// Normal azimuth, pi/128 radians per unit.
pub azimuth: i8,
/// Normal elevation, pi/128 radians per unit.
pub elevation: i8,
pub color: i16,
}
/// Everything one tfrag contributes to a draw call. /// Everything one tfrag contributes to a draw call.
#[derive(Debug, Clone, Default, PartialEq)] #[derive(Debug, Clone, Default, PartialEq)]
pub struct TfragGeometry { pub struct TfragGeometry {
@@ -454,7 +596,12 @@ pub struct TfragGeometry {
pub vinfo_tier: [usize; 3], pub vinfo_tier: [usize; 3],
/// Index 0 is highest detail. /// Index 0 is highest detail.
pub lod: [TfragLod; 3], pub lod: [TfragLod; 3],
pub ad_gif_count: usize, /// Texture primitives; strips select one by ad-gif index.
pub ad_gifs: Vec<TfragAdGifs>,
/// Per-vertex colours, indexed by position index.
pub rgba: Vec<TfragRgba>,
/// Per-vertex light and normal records, indexed by position index.
pub lights: Vec<TfragLight>,
} }
impl TfragGeometry { impl TfragGeometry {
@@ -547,7 +694,15 @@ pub fn parse_tfrag(block: &[u8], header: &TfragHeader) -> Result<TfragGeometry>
have_vu = true; have_vu = true;
} }
// Ad-gifs: 5 quadwords each. // Ad-gifs: 5 quadwords each.
(3, 0) => g.ad_gif_count = v.count() / 5, (3, 0) => {
for i in 0..v.count() / 5 {
let o = i * 0x50;
if payload.len() < o + 0x50 {
break;
}
g.ad_gifs.push(TfragAdGifs::parse(&payload[o..o + 0x50]));
}
}
(2, 1) => { (2, 1) => {
let tier = if addr == vu.positions_common_addr { let tier = if addr == vu.positions_common_addr {
0 0
@@ -614,6 +769,23 @@ pub fn parse_tfrag(block: &[u8], header: &TfragHeader) -> Result<TfragGeometry>
kind: ErrorKind::Invalid("tfrag has no VU header"), kind: ErrorKind::Invalid("tfrag has no VU header"),
}); });
} }
// These three arrays sit outside the VIF lists, at fixed offsets in the
// patch's data block.
g.rgba = read_rgba(
block,
header.rgba_ofs as usize,
header.rgba_size as usize * 4,
)?;
g.lights = read_lights(
block,
header.light_ofs as usize + 0x10,
header.vert_count as usize,
)?;
if !have_origin {
// Without an STROW origin the quadword at light_ofs carries it.
g.origin = read_i32x4(block, header.light_ofs as usize).unwrap_or([0; 4]);
}
if header.vert_count != 0 && g.positions.len() != header.vert_count as usize { if header.vert_count != 0 && g.positions.len() != header.vert_count as usize {
return Err(Error { return Err(Error {
offset: 0, offset: 0,
@@ -621,10 +793,70 @@ pub fn parse_tfrag(block: &[u8], header: &TfragHeader) -> Result<TfragGeometry>
kind: ErrorKind::Invalid("tfrag position count does not match its header"), kind: ErrorKind::Invalid("tfrag position count does not match its header"),
}); });
} }
let _ = have_origin;
Ok(g) Ok(g)
} }
/// Read `count` 4-byte colour records at `at`.
fn read_rgba(block: &[u8], at: usize, count: usize) -> Result<Vec<TfragRgba>> {
let end = at
.checked_add(count * 4)
.filter(|&e| e <= block.len())
.ok_or(Error {
offset: at,
context: "tfrag rgba array runs past the block",
kind: ErrorKind::OutOfBounds {
needed: count * 4,
available: block.len().saturating_sub(at),
},
})?;
Ok((at..end)
.step_by(4)
.map(|i| TfragRgba {
r: block[i],
g: block[i + 1],
b: block[i + 2],
a: block[i + 3],
})
.collect())
}
/// Read `count` 8-byte light records at `at`.
fn read_lights(block: &[u8], at: usize, count: usize) -> Result<Vec<TfragLight>> {
let end = at
.checked_add(count * 8)
.filter(|&e| e <= block.len())
.ok_or(Error {
offset: at,
context: "tfrag light array runs past the block",
kind: ErrorKind::OutOfBounds {
needed: count * 8,
available: block.len().saturating_sub(at),
},
})?;
Ok((at..end)
.step_by(8)
.map(|i| TfragLight {
unknown_0: block[i] as i8,
intensity: block[i + 1] as i8,
azimuth: block[i + 2] as i8,
elevation: block[i + 3] as i8,
color: i16::from_le_bytes([block[i + 4], block[i + 5]]),
})
.collect())
}
/// Read four `i32` at `at`, or `None` if out of range.
fn read_i32x4(block: &[u8], at: usize) -> Option<[i32; 4]> {
if at.checked_add(16).is_none_or(|e| e > block.len()) {
return None;
}
let mut out = [0i32; 4];
for (i, slot) in out.iter_mut().enumerate() {
*slot = i32::from_le_bytes(block[at + i * 4..at + i * 4 + 4].try_into().ok()?);
}
Some(out)
}
/// Read `count` V3_16 elements: three `i16` each, 6 bytes per element. /// Read `count` V3_16 elements: three `i16` each, 6 bytes per element.
fn read_i16x3(bytes: &[u8], count: usize) -> Result<Vec<[i16; 3]>> { fn read_i16x3(bytes: &[u8], count: usize) -> Result<Vec<[i16; 3]>> {
if bytes.len() < count * 6 { if bytes.len() < count * 6 {
@@ -712,7 +944,7 @@ pub fn tfrag_triangles(g: &TfragGeometry, lod: usize) -> Result<Vec<TfragTriangl
} }
if k == 0 || s.end_of_packet_flag >= 0 || s.ad_gif_offset >= 0 { if k == 0 || s.end_of_packet_flag >= 0 || s.ad_gif_offset >= 0 {
let z = s.ad_gif_offset as i32; let z = s.ad_gif_offset as i32;
if z < 0 || z % 5 != 0 || (z / 5) as usize >= g.ad_gif_count { if z < 0 || z % 5 != 0 || (z / 5) as usize >= g.ad_gifs.len() {
return Err(Error { return Err(Error {
offset: 0, offset: 0,
context: "strip ad-gif offset outside the ad-gif array", context: "strip ad-gif offset outside the ad-gif array",
@@ -1001,7 +1233,7 @@ mod topology_tests {
/// A common list holding only a V3_16 position unpack, plus the LOD-0 /// A common list holding only a V3_16 position unpack, plus the LOD-0
/// strips and indices that reference it. /// strips and indices that reference it.
fn fixture() -> (Vec<u8>, TfragHeader) { pub(super) fn fixture() -> (Vec<u8>, TfragHeader) {
const INDICES: u16 = 6; const INDICES: u16 = 6;
const STRIPS: u16 = 8; const STRIPS: u16 = 8;
let verts: [[i16; 3]; 6] = [ let verts: [[i16; 3]; 6] = [
@@ -1066,6 +1298,21 @@ mod topology_tests {
} }
} }
// The tfrag origin, as the STROW row that precedes the positions.
block.extend_from_slice(
&VifCode {
irq: false,
cmd: 0x30,
num: 0,
imm: 0,
}
.to_word()
.to_le_bytes(),
);
for c in [1024i32, 2048, 4096, 0] {
block.extend_from_slice(&c.to_le_bytes());
}
// Positions. // Positions.
block.extend_from_slice( block.extend_from_slice(
&VifCode { &VifCode {
@@ -1140,6 +1387,27 @@ mod topology_tests {
let lod0_end = block.len().div_ceil(4) * 4; let lod0_end = block.len().div_ceil(4) * 4;
block.resize(lod0_end, 0); block.resize(lod0_end, 0);
// Past the lists: the origin quadword and the light array.
let lights_at = block.len().div_ceil(16) * 16;
block.resize(lights_at, 0);
for c in [0i32, 0, 0, 0] {
block.extend_from_slice(&c.to_le_bytes());
}
for i in 0..6u8 {
let v = i as i8;
let colour = (i as u16) * 100;
block.extend_from_slice(&[
i,
v as u8,
(v * 8) as u8,
(v * 4) as u8,
colour as u8,
(colour >> 8) as u8,
0,
0,
]);
}
let mut h = TfragHeader::parse(&[0u8; TFRAG_HEADER_SIZE]).unwrap(); let mut h = TfragHeader::parse(&[0u8; TFRAG_HEADER_SIZE]).unwrap();
h.vert_count = 6; h.vert_count = 6;
h.tri_count = 4; // two runs of 4 vertices h.tri_count = 4; // two runs of 4 vertices
@@ -1148,6 +1416,8 @@ mod topology_tests {
h.lod_0_ofs = common_end as u16; h.lod_0_ofs = common_end as u16;
h.lod_1_size = (common_end / 16) as u8; h.lod_1_size = (common_end / 16) as u8;
h.rgba_ofs = lod0_end as u16; h.rgba_ofs = lod0_end as u16;
h.light_ofs = lights_at as u16;
h.light_end_ofs = (lights_at + 0x10 + 6 * 8) as u16;
(block, h) (block, h)
} }
@@ -1176,10 +1446,21 @@ mod topology_tests {
let g = parse_tfrag(&block, &header).unwrap(); let g = parse_tfrag(&block, &header).unwrap();
assert_eq!(g.position_index(3), 3); assert_eq!(g.position_index(3), 3);
assert_eq!(g.lod_vinfo_count(0), 6); assert_eq!(g.lod_vinfo_count(0), 6);
// Origin from the STROW row, in 1024ths of a world unit.
assert_eq!(g.origin, [1024, 2048, 4096, 0]);
assert_eq!( assert_eq!(
g.world_position(0), g.world_position(0),
[99.0 / 1024.0, -3795.0 / 1024.0, -5203.0 / 1024.0] [
(1024 + 99) as f32 / 1024.0,
(2048 - 3795) as f32 / 1024.0,
(4096 - 5203) as f32 / 1024.0
]
); );
// Six positions, so six light records.
assert_eq!(g.lights.len(), 6);
assert_eq!(g.lights[3].intensity, 3);
assert_eq!(g.lights[3].azimuth, 24);
assert_eq!(g.lights[3].color, 300);
} }
#[test] #[test]
@@ -1202,3 +1483,95 @@ mod topology_tests {
assert!(tfrag_triangles(&g, 0).is_err()); assert!(tfrag_triangles(&g, 0).is_err());
} }
} }
#[cfg(test)]
mod ad_gif_tests {
use super::*;
/// An ad-gif as measured in level 0: texture 12, K = -137/16, clamp, mip 4.
fn measured() -> TfragAdGifs {
TfragAdGifs {
tex0: AdGif {
data_lo: 12,
data_hi: 0,
address: 0x06,
},
tex1: AdGif {
data_lo: 0xf77,
data_hi: 4,
address: 0x14,
},
clamp: AdGif {
data_lo: 0,
data_hi: 1,
address: 0x08,
},
miptbp1: AdGif {
data_lo: 0,
data_hi: 0,
address: 0x34,
},
miptbp2: AdGif {
data_lo: 0,
data_hi: 0,
address: 0x36,
},
}
}
#[test]
fn decodes_the_texture_binding() {
let a = measured();
assert_eq!(a.texture_index(), 12);
assert_eq!(a.wrap_s(), GsWrap::Repeat);
assert_eq!(a.wrap_t(), GsWrap::Clamp);
assert_eq!(a.min_filter(), GsFilter::LinearMipmapNearest);
// K is a signed 12-bit value in 1/16 units.
assert_eq!(a.lod_k_raw(), -137);
assert_eq!(a.lod_k(), -137.0 / 16.0);
}
#[test]
fn a_negative_k_survives_the_round_trip() {
// The on-disc field is only 12 bits wide and is sign-extended through the
// shift, so 0xf77 in 1/16 units is -137 rather than +3959.
let a = TfragAdGifs {
tex1: AdGif {
data_lo: 0xf77,
data_hi: 4,
address: 0x14,
},
..measured()
};
assert_eq!(a.lod_k_raw(), -137);
}
#[test]
fn parses_five_quadwords_into_register_order() {
let mut bytes = Vec::new();
for (i, addr) in [0x06u8, 0x14, 0x08, 0x34, 0x36].iter().enumerate() {
bytes.extend_from_slice(&((i as i32 + 1) * 10).to_le_bytes());
bytes.extend_from_slice(&((i as i32 + 1) * 100).to_le_bytes());
bytes.push(*addr);
bytes.extend_from_slice(&[0u8; 7]);
}
let a = TfragAdGifs::parse(&bytes);
assert_eq!(a.tex0.data_lo, 10);
assert_eq!(a.tex0.address, 0x06);
assert_eq!(a.tex1.data_hi, 200);
assert_eq!(a.clamp.data_lo, 30);
assert_eq!(a.miptbp2.address, 0x36);
}
#[test]
fn the_fixture_exposes_two_ad_gifs() {
let (block, header) = super::topology_tests::fixture();
let g = parse_tfrag(&block, &header).unwrap();
assert_eq!(g.ad_gifs.len(), 2);
assert_eq!(g.ad_gifs[1].texture_index(), 0);
// Strip 1 selects ad-gif 1 via its quadword offset of 5.
let tris = tfrag_triangles(&g, 0).unwrap();
assert_eq!(tris[0].ad_gif, 0);
assert_eq!(tris[2].ad_gif, 1);
}
}
+40 -3
View File
@@ -357,8 +357,45 @@ element size from the immediate, 2-byte payload padding, ignoring non-unpack
payloads, and banking strips by arrival order. Every one produced parseable, payloads, and banking strips by arrival order. Every one produced parseable,
plausible geometry; only the header's own triangle count caught them. plausible geometry; only the header's own triangle count caught them.
### Texture bindings, colours and lights
Also settled. Adds `level_core.rs` for the index header and the texture table,
which is the join between tfrags and texel data.
A tfrag's **ad-gifs** are five A+D quadwords (0x50 bytes) holding packed
fields rather than GS registers; the level load rewrites them in place before
drawing. On disc `TEX0_1.data_lo` indexes the level's texture table,
`TEX1_1.data_lo` is the LOD bias K and `data_hi` the minification filter, and
`CLAMP_1` holds the two wrap modes. The texture table is a count/offset pair at
index offset 0x30, entries of 16 bytes.
The table checks out on its own terms: on level 0 it is 93 entries at 0x2900,
and 0x2900 + 93 x 16 lands exactly on the next range at 0x38.
Three details worth writing down:
* **K is a signed 12-bit value in 1/16 units**, sign-extended by shifting the
field up 4 and back down. Retail values are -137..-89, which is -8.56..-5.56
mip levels. Reading the field as unsigned gives 1911 instead of -137.
* **`data_hi` of TEX1_1 is MMIN, and 4 is `LinearMipmapNearest`**, not
`LinearMipmapLinear` as the bit layout alone suggests.
* **RGBA and the light/normal records are not in the VIF lists.** They sit at
fixed offsets in the patch's data block: `rgba_size * 4` four-byte colours at
`rgba_ofs`, and `vert_count` eight-byte records at `light_ofs + 0x10`. The
origin also lives there — if no STROW row supplied one, the quadword at
`light_ofs` is the origin.
### Verification
Every ad-gif must index a texture that exists, or the geometry and the texture
data have been misread independently of each other. Across all 27 levels:
**47,950 ad-gifs over 18,102 patches, all indices in range**, referencing
1429 of the 1595 level texture entries (90%). Each patch's light array is
exactly `vert_count` records and its RGBA array exactly `rgba_size * 4`.
### Still open ### Still open
The ad-gif texture bindings (which texture each primitive samples), the RGBA The LOD switch and morph rules in `TfragProc`, the texel data itself, and the
per-vertex colours and light records, and the LOD switch and morph rules in GS register values the level load builds from the texture entry. A tfrag can
`TfragProc`. Positions and topology are drawable but not yet textured. now be turned into positioned, UV-mapped, textured triangles; what is missing
is choosing which LOD to draw and decoding pixels.
+8 -3
View File
@@ -20,7 +20,7 @@ early, while they are still cheap to retire.
header, and the `data` extent is parsed as a container of byte ranges. The header, and the `data` extent is parsed as a container of byte ranges. The
geometry decompresses: level 0 yields 16,969,296 bytes of level content geometry decompresses: level 0 yields 16,969,296 bytes of level content
from a 9.7 MB stream. from a 9.7 MB stream.
- [ ] Next: tfrag texture bindings, then the geometry directory's class tables. - [ ] Next: tfrag LOD selection and morphing, then the first drawable frame.
The disc being the exact build the reference projects pin is the single The disc being the exact build the reference projects pin is the single
biggest favourable fact so far: no disc conversion, no regional timing work, biggest favourable fact so far: no disc conversion, no regional timing work,
@@ -128,8 +128,13 @@ Phase 1 hands over the extents; this phase works out what each one contains.
rather than positions. Every patch's strips expand to exactly its own rather than positions. Every patch's strips expand to exactly its own
`tri_count`: **18,102 patches, 809,879 triangles**, all indices in range. `tri_count`: **18,102 patches, 809,879 triangles**, all indices in range.
See `plan/level-assets.md` and `gc-formats::tfrag`. See `plan/level-assets.md` and `gc-formats::tfrag`.
- [ ] tfrag shading inputs: the ad-gif texture bindings, per-vertex RGBA and - [x] **tfrag texture bindings and per-vertex data.** The ad-gifs decode to a
light records, then the LOD switch and morph rules in `TfragProc`. texture index, wrap modes, minification filter and LOD bias; the level
index gained a texture table (`level_core.rs`) for those indices to land
in. Per-vertex RGBA and light/normal records recovered from their fixed
offsets. All 27 levels: 47,950 ad-gifs, every index in range, 90% of the
level texture tables referenced.
- [ ] tfrag LOD selection: the switch and morph rules in `TfragProc`.
- [ ] `gameplay` — begins `WAD` + a per-level byte, and does *not* decode with - [ ] `gameplay` — begins `WAD` + a per-level byte, and does *not* decode with
the WAD LZ77 decoder. All 27 levels fail identically, so the input is the WAD LZ77 decoder. All 27 levels fail identically, so the input is
wrong rather than the decoder. Ruled out: bit order, header length, the wrong rather than the decoder. Ruled out: bit order, header length, the