Decode level textures: 8-bit indices through a CSM1 palette

Adds texture.rs and makes the pipeline write gs_ram.bin, which it had
been skipping. Nothing could be decoded before this: the CLUTs live
there.

A level texture is not pixels, it is palette indices -- width*height
bytes at core_data[textures_base + data_offset], with colour coming from
a 256-entry CLUT at gs_ram[palette * 0x100]. gs_ram is a byte range in
the level's data extent header at offset 0x10, 518,656 bytes on level 0.

Two PS2 conventions have to be honoured and neither is a design choice:
within each 32-entry CLUT group the two middle 8-entry blocks are
swapped (bits 3 and 4 of the index are exchanged, an involution), and
alpha runs 0..=0x80 rather than 0..=255 with 0x80 fully opaque.

The mip chain splits across two buffers: levels 0 and 1 are in the
texture block with level 1 immediately after level 0, and levels 2 and 3
are resident in gs_ram at mipmap*0x100 and pad*0x100. Tfrag textures are
square so every level halves evenly.

Verification is two-layered, because every step here is a table lookup
and a wrong CLUT remap still produces a well-formed image of the wrong
thing. All 1,595 tfrag textures across all 27 levels decode; 96.8% are
fully opaque and 96.3% use more than eight colours. On top of that,
`gc-extract dump-texture` writes any texture or mip level as a PPM.
Level 0 texture 0 comes back as a clean bevelled metal panel with rivets
and specular highlights, and texture 74 as a vent grille whose punched
windows are fully transparent. That tool is worth keeping: it is the
cheapest way to tell a decode that is merely well-formed from one that
is right.

The hash table is regenerated for the 27 new gs_ram.bin files, now 5,186
files and 5,685,541,143 bytes.
This commit is contained in:
2026-10-10 02:30:56 +02:00
parent 48d9506cf1
commit 29fdf89f53
8 changed files with 693 additions and 15 deletions
+103
View File
@@ -704,3 +704,106 @@ fn tfrag_ad_gifs_index_the_level_texture_table() {
"{ad_gifs} ad-gifs over {patches} patches reference {textures_seen}/{textures_total} level textures"
);
}
/// Level textures decode: every tfrag texture's base level must resolve to a
/// plausible RGBA image.
///
/// This is the join between the texture table, the texel block and GS RAM, so
/// it is where a wrong CLUT remap, a wrong alpha scale or a wrong pixel base
/// would show up. The check is deliberately about *shape*, not specific
/// colours: a decoded image is plausible when it is fully opaque, is not
/// uniformly one colour, and uses more than a handful of distinct entries.
#[test]
fn tfrag_textures_decode_to_plausible_images() {
require_disc!();
gc_formats::iso9660::IsoImage::open(std::path::Path::new(
&std::env::var("GC_DISC").expect("GC_DISC is set by require_disc!"),
))
.expect("ISO should open");
let data = std::env::var_os("GC_EXTRACTED")
.map(PathBuf::from)
.unwrap_or_else(|| {
let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
PathBuf::from(home).join("Documents/gc-extracted")
});
if !data.exists() {
eprintln!("skipped: unpack the disc first");
return;
}
let mut decoded = 0usize;
let mut opaque = 0usize;
let mut multi_colour = 0usize;
let mut flat = 0usize;
for n in 0..27 {
let dir = data.join(format!("level{n}"));
let core = match std::fs::read(dir.join("core_data.bin")) {
Ok(c) => c,
Err(_) => continue,
};
let index = match std::fs::read(dir.join("core_index.bin")) {
Ok(i) => i,
Err(_) => continue,
};
let gs = match std::fs::read(dir.join("gs_ram.bin")) {
Ok(g) => g,
Err(_) => {
eprintln!("level{n}: no gs_ram.bin, skipping");
continue;
}
};
let ch = gc_formats::level_core::LevelCoreHeader::parse(&index).expect("core header");
let textures_at = ch.textures_base.max(0) as usize;
let texture_block = &core[textures_at..];
let table =
gc_formats::level_core::texture_table(&index, ch.tfrag_textures).expect("table");
for (i, e) in table.iter().enumerate() {
let img = gc_formats::texture::base_image(texture_block, &gs, e)
.unwrap_or_else(|err| panic!("level{n} texture {i}: {err}"));
let t = img.decode().unwrap();
let texels = (e.width as i32) * (e.height as i32);
assert_eq!(t.rgba.len(), (texels * 4) as usize);
assert_eq!(t.width, e.width as u32);
decoded += 1;
// Count distinct colours and how many texels are fully opaque.
let mut seen = std::collections::HashSet::new();
let mut op = 0usize;
for px in t.rgba.chunks_exact(4) {
seen.insert([px[0], px[1], px[2], px[3]]);
if px[3] == 0xff {
op += 1;
}
}
if seen.len() > 8 {
multi_colour += 1;
} else {
flat += 1;
}
if op == texels as usize {
opaque += 1;
}
}
}
assert!(decoded > 1_500, "only {decoded} textures decoded");
// Level art is mostly opaque and has real detail. A flat, transparent or
// single-colour result almost always means a CLUT or offset mistake.
let opaque_pct = opaque as f64 / decoded as f64;
let detail_pct = multi_colour as f64 / decoded as f64;
assert!(
opaque_pct > 0.9,
"only {opaque:.1}% of textures are fully opaque"
);
assert!(
detail_pct > 0.9,
"only {detail_pct:.1} of textures have >8 colours"
);
eprintln!(
"{decoded} textures decoded: {:.1}% fully opaque, {:.1}% with >8 colours ({flat} flat)",
opaque_pct * 100.0,
detail_pct * 100.0
);
}