Decode level textures: 8-bit indices through a CSM1 palette

Adds texture.rs and makes the pipeline write gs_ram.bin, which it had
been skipping. Nothing could be decoded before this: the CLUTs live
there.

A level texture is not pixels, it is palette indices -- width*height
bytes at core_data[textures_base + data_offset], with colour coming from
a 256-entry CLUT at gs_ram[palette * 0x100]. gs_ram is a byte range in
the level's data extent header at offset 0x10, 518,656 bytes on level 0.

Two PS2 conventions have to be honoured and neither is a design choice:
within each 32-entry CLUT group the two middle 8-entry blocks are
swapped (bits 3 and 4 of the index are exchanged, an involution), and
alpha runs 0..=0x80 rather than 0..=255 with 0x80 fully opaque.

The mip chain splits across two buffers: levels 0 and 1 are in the
texture block with level 1 immediately after level 0, and levels 2 and 3
are resident in gs_ram at mipmap*0x100 and pad*0x100. Tfrag textures are
square so every level halves evenly.

Verification is two-layered, because every step here is a table lookup
and a wrong CLUT remap still produces a well-formed image of the wrong
thing. All 1,595 tfrag textures across all 27 levels decode; 96.8% are
fully opaque and 96.3% use more than eight colours. On top of that,
`gc-extract dump-texture` writes any texture or mip level as a PPM.
Level 0 texture 0 comes back as a clean bevelled metal panel with rivets
and specular highlights, and texture 74 as a vent grille whose punched
windows are fully transparent. That tool is worth keeping: it is the
cheapest way to tell a decode that is merely well-formed from one that
is right.

The hash table is regenerated for the 27 new gs_ram.bin files, now 5,186
files and 5,685,541,143 bytes.
This commit is contained in:
2026-10-10 02:30:56 +02:00
parent 48d9506cf1
commit 29fdf89f53
8 changed files with 693 additions and 15 deletions
+47 -4
View File
@@ -393,9 +393,52 @@ data have been misread independently of each other. Across all 27 levels:
1429 of the 1595 level texture entries (90%). Each patch's light array is
exactly `vert_count` records and its RGBA array exactly `rgba_size * 4`.
### Texels: 8-bit indexed images and CSM1 palettes
Settled. Adds `texture.rs` and makes the pipeline write `gs_ram.bin`.
A level texture is not pixels, it is **palette indices**: `width * height`
bytes at `core_data[textures_base + data_offset]`. The colour comes from a
256-entry CLUT (1024 bytes) in GS RAM at `palette * 0x100`. `gs_ram` is a byte
range in the level's data extent header at offset 0x10 — 518,656 bytes on level
0 — and the extractor was not writing it out, so nothing could be decoded
before this.
Two PS2 conventions have to be honoured, and neither is a design choice:
* **CSM1 CLUT order.** Within each 32-entry group the two middle 8-entry
blocks are swapped: bits 3 and 4 of the index are exchanged. The remap is an
involution.
* **Alpha runs 0..=0x80, not 0..=255.** `0x80` is fully opaque and anything
above it clamps.
The mip chain splits across two buffers. Levels 0 and 1 are in the texture
block, with level 1 immediately after level 0 at `data_offset + w*h`; levels 2
and 3 are resident in GS RAM at `mipmap * 0x100` and `pad * 0x100`. Tfrag
textures are square, so every level halves evenly.
### Verification
Numbers alone do not settle this — every step is a table lookup, so a wrong
CLUT remap still yields a well-formed image of the wrong thing. Two layers:
* **Shape.** All 1,595 tfrag textures across all 27 levels decode; 96.8% are
fully opaque and 96.3% use more than eight distinct colours. A flat,
transparent or single-colour result would point at a CLUT or offset mistake.
* **Eyes.** `gc-extract dump-texture` writes any tfrag texture, or any of its
mip levels, as a binary PPM. Decoded level 0 texture 0 is a clean bevelled
metal panel with rivets and specular highlights; texture 74 is a vent grille
whose punched windows come back fully transparent. Both confirm the CLUT
remap, the alpha scale and the pixel base at once, which no assertion in the
suite does.
That tool is worth keeping: it is the cheapest way to tell a decode that is
merely well-formed from one that is right.
### Still open
The LOD switch and morph rules in `TfragProc`, the texel data itself, and the
GS register values the level load builds from the texture entry. A tfrag can
now be turned into positioned, UV-mapped, textured triangles; what is missing
is choosing which LOD to draw and decoding pixels.
The LOD switch and morph rules in `TfragProc`, and the GS register values the
level load builds from the texture entry (TBP0/TBP1 and so on) for a renderer
that emulates the GS rather than resolving addresses itself. A tfrag can now be
turned into positioned, UV-mapped, textured triangles; what is missing is
choosing which LOD to draw.
+8 -1
View File
@@ -14,7 +14,7 @@ early, while they are still cheap to retire.
- [x] Repository hygiene in place (pre-commit guard, `.gitignore`, licence
record)
- [x] **Phase 1 complete.** The disc pipeline runs end to end and is verified
against a committed hash table: 5,135 files, 4,500,106,960 bytes, every
against a committed hash table: 5,186 files, 5,685,541,143 bytes, every
one of the 92 asset containers parsing.
- [x] **Phase 2 in progress.** Level containers are parsed through their typed
header, and the `data` extent is parsed as a container of byte ranges. The
@@ -134,6 +134,13 @@ Phase 1 hands over the extents; this phase works out what each one contains.
in. Per-vertex RGBA and light/normal records recovered from their fixed
offsets. All 27 levels: 47,950 ad-gifs, every index in range, 90% of the
level texture tables referenced.
- [x] **Level textures decode.** 8-bit indexed texels plus a 256-entry CSM1
CLUT from `gs_ram`, honouring the PS2 CLUT block swap and the 0x80 alpha
scale. The pipeline now writes `gs_ram.bin`, which it had been skipping.
All 1,595 tfrag textures on all 27 levels decode; 96.8% fully opaque and
96.3% with real detail. Verified visually via the new
`gc-extract dump-texture`, because a wrong lookup still yields a
well-formed image of the wrong thing.
- [ ] tfrag LOD selection: the switch and morph rules in `TfragProc`.
- [ ] `gameplay` — begins `WAD` + a per-level byte, and does *not* decode with
the WAD LZ77 decoder. All 27 levels fail identically, so the input is