ReaperOfVeriod/Going-Commando
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Parse the level container header, naming each level's four extents

Browse Source 
The first typed structure read from real data. Every one of the 27 level
containers begins with the same 0x60-byte header, which turns the generic pair
table into named assets: data, sound_bank, gameplay and occlusion, plus three
chunk descriptors.

Two things about this header are worth stating because both are silent failures
rather than errors.

Sizes here are sector counts, not byte counts. The generic G/ pair table on the
same disc uses byte counts, so both readings type-check and the wrong one is off
by a factor of 2048. The tiling check settles it: read as sectors, the four
extents tile every one of the 27 containers exactly; read as bytes they overlap
each other.

Level id is not container order. LEVEL21.WAD reports id 30, so a level was cut
from the disc and its number left a gap. Anything keying off file order -- a
save slot, the planet select screen -- would mislabel that planet. There is also
a second trap in the same area: the ISO filesystem sorts names alphabetically, so
LEVEL10 precedes LEVEL2, and any tool needing order has to sort numerically.

The header's field at 0x04 is zero on all 27 levels. It is not the container
size despite looking like one, so the code reads it as a runtime field and takes
the container length from the ISO entry instead.

Measured across all 27 levels, three invariants now hold in tests: extents tile
in file order; extent coverage plus chunks reaches at least the end of the
container (21 levels land exactly on its length, 6 leave 134 to 689 sectors of
trailing data); and exactly one level's id differs from its container index. The
coverage assertion is >= rather than == on purpose -- 21 levels end one sector
past the file end because the header occupies sector 0, and asserting equality
would truncate a level's tail.

gameplay is left unparsed and that is recorded rather than hidden. It begins
"WAD" plus a per-level byte and does not decode with the WAD LZ77 decoder: all
27 levels produce plausible output for the first packet and then fail on a rule
the format's own encoder never breaks, which means the input is wrong rather
than the decoder. docs/plan/level-assets.md records what is ruled out and what
to try next.

66 unit tests, plus 7 integration tests against the real disc. The unpack hash
table is unchanged, so this adds parsing without altering the pipeline.
This commit is contained in:
ReaperOfVeriod
2026-10-09 16:20:25 +02:00
parent 027373e65b
commit a9e43e39c1
6 changed files with 753 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/formats/level-wad.md
+139
View File
@@ -0,0 +1,139 @@
# The level container (`LEVEL<n>.WAD`)
Target: `SCUS_972.68` v1.01. 27 containers.
The first typed structure in this project, and the one that turns the generic
container table of [`global-wad.md`](global-wad.md) into named, individually
parseable assets. Implementation: `crates/gc-formats/src/level_wad.rs`.
## Header
All 27 level containers begin with the same 0x60-byte header:
```text
offset size field
0x00 4 header size, 0x60 on every level
0x04 4 zero on every level; written by the game at load time
0x08 4 level id
0x0c 4 reverb configuration index
0x10 8 data geometry: tfrag, tie, shrub, moby, sky, textures
0x18 8 sound_bank
0x20 8 gameplay
0x28 8 occlusion
0x30 48 three {offset, size} chunk descriptors
```
Sizes here are **sector counts**. This is the opposite of the generic pair table
in the `G/` containers, where sizes are byte counts. Both appear on the same
disc, so reading one as the other produces extents that are wrong by a factor of
2048 and that still *look* plausible. See "Two fields, two meanings" below.
## Extents
The four named extents appear in the file in an order that is **not** the order
of the fields in the header:
```text
sound_bank data gameplay occlusion
```
Level 0, measured:
| Part | Offset | Sectors | Bytes | Unpacked to |
|---|---:|---:|---:|---|
| sound_bank | 1 | 734 | 1.50 MB | `level0/0002.bin` |
| data | 735 | 6,484 | 13.28 MB | `level0/0001.bin` |
| gameplay | 7,219 | 322 | 0.66 MB | `level0/0003.bin` |
| occlusion | 7,541 | 12 | 0.02 MB | `level0/0004.bin` |
The numbering of the unpacked files follows the header's field order, so the
`data` extent is `0001.bin` while it is physically second in the container. Both
orderings are worth keeping straight; the typed header exists so callers never
have to.
`data` is where the level's visible content lives and is the target of the next
phase.
## Two fields, two meanings
This is the trap in this format, and it is worth stating plainly because both
readings type-check.
| Where | Size field is |
|---|---|
| `G/*.WAD` generic pair table | **bytes** |
| `LEVEL<n>.WAD` named extents | **sectors** |
Measured across all 92 indexed pair containers, the byte reading places every
entry inside its file while the sector reading does not. Measured across all 27
level headers, the sector reading tiles each container exactly.
If the level header's sizes were read as bytes, `sound_bank` on level 0 would
claim 734 bytes starting at sector 1, and `data` would start at byte offset 735
— inside the sound bank. The extents would overlap, which is the symptom to
look for.
## Invariants, as measured on all 27 levels
These are the checks the parser is validated against, and each holds everywhere
rather than on one sample:
1. **Extent order and tiling.** Each extent's end is the next extent's start.
27 of 27.
2. **The runtime sector field at `0x04` is zero.** 27 of 27. It is not the
container size and must not be used as one — the container's length comes
from the ISO directory entry.
3. **Extent coverage.** `sound_bank + data + gameplay + occlusion + chunks`
always reaches at least the end of the container. 21 of 27 land on exactly
the container's sector length; 6 leave between 134 and 689 sectors of trailing
data past the last named extent (levels 1, 2, 4, 11, 19, 20).
The 21-level case ends *one sector past* the file end. The header occupies
sector 0 and the extents begin at sector 1, so this is ordinary sector
rounding rather than an error. The correct assertion is therefore `>=`, not
equality — asserting equality would truncate the tail of a level.
4. **Level id equals container index on 26 of 27 levels.** `LEVEL21.WAD`
reports **id 30**.
## Level ids are not file order
Point 4 matters enough to state separately. `LEVEL21.WAD` has `id = 30`, so the
container's position in the directory is not the level's identity. Some level
was cut from the disc and its number is not reused, leaving a gap.
Anything that keys off file order — a save slot, a planet select screen, the
moby class table — would silently mislabel that planet. Use the header's `id`.
The ISO filesystem also sorts names alphabetically, so `LEVEL10.WAD` precedes
`LEVEL2.WAD`. Tests and tools that care about order must sort numerically.
## Chunk descriptors
`0x30..0x60` holds three `{offset, size}` pairs in sector counts, following the
named extents. They are populated on 6 levels (1, 2, 4, 7, 8, 11, 19, 20 — of
which some are the trailing-data levels) and zero elsewhere. Their contents are
not yet identified.
## Not yet parsed
- **`data`** — the geometry. This is the next target and where the visible
level comes from: tfrag terrain, tie instances, shrub foliage, moby classes,
sky shells and textures.
- **`gameplay`** — begins `WAD` + a per-level id byte + a count, and behaves
like an LZ77 stream but does **not** decode with the WAD decoder (all 27
levels fail with "adjacent literal packets", which the format forbids). It is
a different compression, or a different framing. Recorded in
[`../plan/level-assets.md`](../plan/level-assets.md).
- **`sound_bank`** — begins with `SBlk` at `+24` and `DAW` at `+36`; both
signatures are present in all 27 levels.
- **`occlusion`** — 2 to 25 sectors. Small enough to read fully.
## Provenance
The header layout matches the `GcUyaLevelWadHeader` structure documented by the
Wrench project (<https://github.com/chaoticgd/wrench>, GPLv3). **No Wrench code
is copied** — see `PROVENANCE.md`. The structure here is written from the
specification and every field is verified against all 27 levels on this disc.
The two facts that the documentation does not state — the sector-count sizes and
the `LEVEL21` id divergence — were measured here.