ReaperOfVeriod
b8b4a1a3d3
No behaviour change. This is the pass over the last stretch of work to make sure the accumulated result hangs together rather than only each commit in isolation. Code. lit_color had grown a `let _ = (depth, lod_distances)` and a dead branch from repeated edits. The directional pass genuinely does not depend on distance, so the parameters are gone rather than discarded, with a note that the point lights -- not implemented -- will need them. lit_at folded into lit_color, which had become a wrapper around it. Checked for leftover scaffolding: only two env vars remain, both real (GC_ENGINE_SHOT and GC_ENGINE_DEBUG_BOUNDS); GC_ENGINE_UNLIT, GC_ENGINE_DEBUG_COLOR and GC_ENGINE_ABL were never actually written, because those edits silently failed to apply against rustfmt'd source. AGENTS.md. The crate table did not say that gc-engine owns hardware models. ps2.rs is the Emotion Engine's FPU and the lighting pass runs on it, so the boundary is now stated: gc-formats decodes inputs, gc-engine reproduces how the hardware computes with them. That is the distinction rule 2 depends on, and it was implicit. Docs. Three claims had been overtaken by what we learned and were actively misleading: * docs/formats/level-wad.md still said gameplay "does not decode with the WAD decoder". It does; it is a WAD stream. * docs/plan/level-assets.md carried a whole section, "a compression we have not identified", with a ruled-out list including "bit order", "header length" and "the WAD format at all". All three were ruled out against the wrong input. Replaced, with the failure mode kept: a uniform failure across 27 inputs was read as "the decoder is wrong" when it meant "the data was handed over uncompressed", and the positive control -- decompress and look -- was never run. * The same file still said the tfrag "record framing is not recovered". True, but it was a false lead that has since been superseded by the VIF1 framing. Kept as a recorded wrong turn rather than deleted. Also refreshed "Still open" to name what is actually outstanding: point lights, the GS register set, the guard-band clip test, the unread core_index class tables, and -- stated plainly -- that lighting magnitudes are unverified because there is still nothing to compare against. Verified after the changes: 148 tests pass, clippy and fmt clean, all 5,213 extracted files match the hash table, and level 0 still loads at 26,758 triangles with the same LOD split and fully lit.
141 lines
5.8 KiB
Markdown
141 lines
5.8 KiB
Markdown
# The level container (`LEVEL<n>.WAD`)
|
|
|
|
Target: `SCUS_972.68` v1.01. 27 containers.
|
|
|
|
The first typed structure in this project, and the one that turns the generic
|
|
container table of [`global-wad.md`](global-wad.md) into named, individually
|
|
parseable assets. Implementation: `crates/gc-formats/src/level_wad.rs`.
|
|
|
|
## Header
|
|
|
|
All 27 level containers begin with the same 0x60-byte header:
|
|
|
|
```text
|
|
offset size field
|
|
0x00 4 header size, 0x60 on every level
|
|
0x04 4 zero on every level; written by the game at load time
|
|
0x08 4 level id
|
|
0x0c 4 reverb configuration index
|
|
0x10 8 data geometry: tfrag, tie, shrub, moby, sky, textures
|
|
0x18 8 sound_bank
|
|
0x20 8 gameplay
|
|
0x28 8 occlusion
|
|
0x30 48 three {offset, size} chunk descriptors
|
|
```
|
|
|
|
Sizes here are **sector counts**. This is the opposite of the generic pair table
|
|
in the `G/` containers, where sizes are byte counts. Both appear on the same
|
|
disc, so reading one as the other produces extents that are wrong by a factor of
|
|
2048 and that still *look* plausible. See "Two fields, two meanings" below.
|
|
|
|
## Extents
|
|
|
|
The four named extents appear in the file in an order that is **not** the order
|
|
of the fields in the header:
|
|
|
|
```text
|
|
sound_bank data gameplay occlusion
|
|
```
|
|
|
|
Level 0, measured:
|
|
|
|
| Part | Offset | Sectors | Bytes | Unpacked to |
|
|
|---|---:|---:|---:|---|
|
|
| sound_bank | 1 | 734 | 1.50 MB | `level0/0002.bin` |
|
|
| data | 735 | 6,484 | 13.28 MB | `level0/0001.bin` |
|
|
| gameplay | 7,219 | 322 | 0.66 MB | `level0/0003.bin` |
|
|
| occlusion | 7,541 | 12 | 0.02 MB | `level0/0004.bin` |
|
|
|
|
The numbering of the unpacked files follows the header's field order, so the
|
|
`data` extent is `0001.bin` while it is physically second in the container. Both
|
|
orderings are worth keeping straight; the typed header exists so callers never
|
|
have to.
|
|
|
|
`data` is where the level's visible content lives and is the target of the next
|
|
phase.
|
|
|
|
## Two fields, two meanings
|
|
|
|
This is the trap in this format, and it is worth stating plainly because both
|
|
readings type-check.
|
|
|
|
| Where | Size field is |
|
|
|---|---|
|
|
| `G/*.WAD` generic pair table | **bytes** |
|
|
| `LEVEL<n>.WAD` named extents | **sectors** |
|
|
|
|
Measured across all 92 indexed pair containers, the byte reading places every
|
|
entry inside its file while the sector reading does not. Measured across all 27
|
|
level headers, the sector reading tiles each container exactly.
|
|
|
|
If the level header's sizes were read as bytes, `sound_bank` on level 0 would
|
|
claim 734 bytes starting at sector 1, and `data` would start at byte offset 735
|
|
— inside the sound bank. The extents would overlap, which is the symptom to
|
|
look for.
|
|
|
|
## Invariants, as measured on all 27 levels
|
|
|
|
These are the checks the parser is validated against, and each holds everywhere
|
|
rather than on one sample:
|
|
|
|
1. **Extent order and tiling.** Each extent's end is the next extent's start.
|
|
27 of 27.
|
|
2. **The runtime sector field at `0x04` is zero.** 27 of 27. It is not the
|
|
container size and must not be used as one — the container's length comes
|
|
from the ISO directory entry.
|
|
3. **Extent coverage.** `sound_bank + data + gameplay + occlusion + chunks`
|
|
always reaches at least the end of the container. 21 of 27 land on exactly
|
|
the container's sector length; 6 leave between 134 and 689 sectors of trailing
|
|
data past the last named extent (levels 1, 2, 4, 11, 19, 20).
|
|
|
|
The 21-level case ends *one sector past* the file end. The header occupies
|
|
sector 0 and the extents begin at sector 1, so this is ordinary sector
|
|
rounding rather than an error. The correct assertion is therefore `>=`, not
|
|
equality — asserting equality would truncate the tail of a level.
|
|
|
|
4. **Level id equals container index on 26 of 27 levels.** `LEVEL21.WAD`
|
|
reports **id 30**.
|
|
|
|
## Level ids are not file order
|
|
|
|
Point 4 matters enough to state separately. `LEVEL21.WAD` has `id = 30`, so the
|
|
container's position in the directory is not the level's identity. Some level
|
|
was cut from the disc and its number is not reused, leaving a gap.
|
|
|
|
Anything that keys off file order — a save slot, a planet select screen, the
|
|
moby class table — would silently mislabel that planet. Use the header's `id`.
|
|
|
|
The ISO filesystem also sorts names alphabetically, so `LEVEL10.WAD` precedes
|
|
`LEVEL2.WAD`. Tests and tools that care about order must sort numerically.
|
|
|
|
## Chunk descriptors
|
|
|
|
`0x30..0x60` holds three `{offset, size}` pairs in sector counts, following the
|
|
named extents. They are populated on 6 levels (1, 2, 4, 7, 8, 11, 19, 20 — of
|
|
which some are the trailing-data levels) and zero elsewhere. Their contents are
|
|
not yet identified.
|
|
|
|
## Not yet parsed
|
|
|
|
- **`data`** — the geometry. Parsed through
|
|
[`level-data.md`](level-data.md); `core_index`, `gs_ram` and `core_data` are
|
|
read, and `core_data` decompresses.
|
|
- **`gameplay`** — a WAD LZ77 stream, exactly like `core_data`, and it
|
|
decompresses. The magic is `WADy\r\n`: byte 3 is `0x79` on level 0 rather
|
|
than `0x0d`, which reads as a per-level variation but is just part of the
|
|
32-bit little-endian stream length. Decompressed it holds the level's light
|
|
bank — 5 directional light sets on level 0. The earlier reading of "not a
|
|
WAD" came from parsing the *compressed* bytes; see
|
|
[`../plan/level-assets.md`](../plan/level-assets.md).
|
|
- **`sound_bank`** — begins with `SBlk` at `+24` and `DAW` at `+36`; both
|
|
signatures are present in all 27 levels.
|
|
- **`occlusion`** — 2 to 25 sectors. Small enough to read fully.
|
|
|
|
## Provenance
|
|
|
|
The header layout matches the `GcUyaLevelWadHeader` structure documented by the
|
|
Wrench project (<https://github.com/chaoticgd/wrench>, GPLv3). **No Wrench code
|
|
is copied** — see `PROVENANCE.md`. The structure here is written from the
|
|
specification and every field is verified against all 27 levels on this disc.
|
|
The two facts that the documentation does not state — the sector-count sizes and
|
|
the `LEVEL21` id divergence — were measured here. |