Files

T

ReaperOfVeriod b8b4a1a3d3 Consolidate: tidy level.rs, correct stale docs, pin hardware models

No behaviour change. This is the pass over the last stretch of work to
make sure the accumulated result hangs together rather than only each
commit in isolation.

Code. lit_color had grown a `let _ = (depth, lod_distances)` and a dead
branch from repeated edits. The directional pass genuinely does not
depend on distance, so the parameters are gone rather than discarded,
with a note that the point lights -- not implemented -- will need them.
lit_at folded into lit_color, which had become a wrapper around it.
Checked for leftover scaffolding: only two env vars remain, both real
(GC_ENGINE_SHOT and GC_ENGINE_DEBUG_BOUNDS); GC_ENGINE_UNLIT,
GC_ENGINE_DEBUG_COLOR and GC_ENGINE_ABL were never actually written,
because those edits silently failed to apply against rustfmt'd source.

AGENTS.md. The crate table did not say that gc-engine owns hardware
models. ps2.rs is the Emotion Engine's FPU and the lighting pass runs
on it, so the boundary is now stated: gc-formats decodes inputs,
gc-engine reproduces how the hardware computes with them. That is the
distinction rule 2 depends on, and it was implicit.

Docs. Three claims had been overtaken by what we learned and were
actively misleading:

* docs/formats/level-wad.md still said gameplay "does not decode with
  the WAD decoder". It does; it is a WAD stream.
* docs/plan/level-assets.md carried a whole section, "a compression we
  have not identified", with a ruled-out list including "bit order",
  "header length" and "the WAD format at all". All three were ruled out
  against the wrong input. Replaced, with the failure mode kept: a
  uniform failure across 27 inputs was read as "the decoder is wrong"
  when it meant "the data was handed over uncompressed", and the
  positive control -- decompress and look -- was never run.
* The same file still said the tfrag "record framing is not recovered".
  True, but it was a false lead that has since been superseded by the
  VIF1 framing. Kept as a recorded wrong turn rather than deleted.

Also refreshed "Still open" to name what is actually outstanding:
point lights, the GS register set, the guard-band clip test, the
unread core_index class tables, and -- stated plainly -- that lighting
magnitudes are unverified because there is still nothing to compare
against.

Verified after the changes: 148 tests pass, clippy and fmt clean, all
5,213 extracted files match the hash table, and level 0 still loads at
26,758 triangles with the same LOD split and fully lit.

2026-10-10 03:37:03 +02:00

5.8 KiB

Raw Permalink Blame History

The level container (`LEVEL<n>.WAD`)

Target: SCUS_972.68 v1.01. 27 containers.

The first typed structure in this project, and the one that turns the generic container table of global-wad.md into named, individually parseable assets. Implementation: crates/gc-formats/src/level_wad.rs.

Header

All 27 level containers begin with the same 0x60-byte header:

offset  size  field
0x00    4     header size, 0x60 on every level
0x04    4     zero on every level; written by the game at load time
0x08    4     level id
0x0c    4     reverb configuration index
0x10    8     data          geometry: tfrag, tie, shrub, moby, sky, textures
0x18    8     sound_bank
0x20    8     gameplay
0x28    8     occlusion
0x30    48    three {offset, size} chunk descriptors

Sizes here are sector counts. This is the opposite of the generic pair table in the G/ containers, where sizes are byte counts. Both appear on the same disc, so reading one as the other produces extents that are wrong by a factor of 2048 and that still look plausible. See "Two fields, two meanings" below.

Extents

The four named extents appear in the file in an order that is not the order of the fields in the header:

sound_bank   data   gameplay   occlusion

Level 0, measured:

Part	Offset	Sectors	Bytes	Unpacked to
sound_bank	1	734	1.50 MB	`level0/0002.bin`
data	735	6,484	13.28 MB	`level0/0001.bin`
gameplay	7,219	322	0.66 MB	`level0/0003.bin`
occlusion	7,541	12	0.02 MB	`level0/0004.bin`

The numbering of the unpacked files follows the header's field order, so the data extent is 0001.bin while it is physically second in the container. Both orderings are worth keeping straight; the typed header exists so callers never have to.

data is where the level's visible content lives and is the target of the next phase.

Two fields, two meanings

This is the trap in this format, and it is worth stating plainly because both readings type-check.

Where	Size field is
`G/*.WAD` generic pair table	bytes
`LEVEL<n>.WAD` named extents	sectors

Measured across all 92 indexed pair containers, the byte reading places every entry inside its file while the sector reading does not. Measured across all 27 level headers, the sector reading tiles each container exactly.

If the level header's sizes were read as bytes, sound_bank on level 0 would claim 734 bytes starting at sector 1, and data would start at byte offset 735 — inside the sound bank. The extents would overlap, which is the symptom to look for.

Invariants, as measured on all 27 levels

These are the checks the parser is validated against, and each holds everywhere rather than on one sample:

Extent order and tiling. Each extent's end is the next extent's start. 27 of 27.
The runtime sector field at 0x04 is zero. 27 of 27. It is not the container size and must not be used as one — the container's length comes from the ISO directory entry.
Extent coverage. sound_bank + data + gameplay + occlusion + chunks always reaches at least the end of the container. 21 of 27 land on exactly the container's sector length; 6 leave between 134 and 689 sectors of trailing data past the last named extent (levels 1, 2, 4, 11, 19, 20).

The 21-level case ends one sector past the file end. The header occupies sector 0 and the extents begin at sector 1, so this is ordinary sector rounding rather than an error. The correct assertion is therefore >=, not equality — asserting equality would truncate the tail of a level.
Level id equals container index on 26 of 27 levels. LEVEL21.WAD reports id 30.

Level ids are not file order

Point 4 matters enough to state separately. LEVEL21.WAD has id = 30, so the container's position in the directory is not the level's identity. Some level was cut from the disc and its number is not reused, leaving a gap.

Anything that keys off file order — a save slot, a planet select screen, the moby class table — would silently mislabel that planet. Use the header's id.

The ISO filesystem also sorts names alphabetically, so LEVEL10.WAD precedes LEVEL2.WAD. Tests and tools that care about order must sort numerically.

Chunk descriptors

0x30..0x60 holds three {offset, size} pairs in sector counts, following the named extents. They are populated on 6 levels (1, 2, 4, 7, 8, 11, 19, 20 — of which some are the trailing-data levels) and zero elsewhere. Their contents are not yet identified.

Not yet parsed

data — the geometry. Parsed through level-data.md; core_index, gs_ram and core_data are read, and core_data decompresses.
gameplay — a WAD LZ77 stream, exactly like core_data, and it decompresses. The magic is WADy\r\n: byte 3 is 0x79 on level 0 rather than 0x0d, which reads as a per-level variation but is just part of the 32-bit little-endian stream length. Decompressed it holds the level's light bank — 5 directional light sets on level 0. The earlier reading of "not a WAD" came from parsing the compressed bytes; see ../plan/level-assets.md.
sound_bank — begins with SBlk at +24 and DAW at +36; both signatures are present in all 27 levels.
occlusion — 2 to 25 sectors. Small enough to read fully.

Provenance

The header layout matches the GcUyaLevelWadHeader structure documented by the Wrench project (https://github.com/chaoticgd/wrench, GPLv3). No Wrench code is copied — see PROVENANCE.md. The structure here is written from the specification and every field is verified against all 27 levels on this disc. The two facts that the documentation does not state — the sector-count sizes and the LEVEL21 id divergence — were measured here.

5.8 KiB Raw Permalink Blame History

The level container (LEVEL<n>.WAD)