ReaperOfVeriod
a9e43e39c1
The first typed structure read from real data. Every one of the 27 level containers begins with the same 0x60-byte header, which turns the generic pair table into named assets: data, sound_bank, gameplay and occlusion, plus three chunk descriptors. Two things about this header are worth stating because both are silent failures rather than errors. Sizes here are sector counts, not byte counts. The generic G/ pair table on the same disc uses byte counts, so both readings type-check and the wrong one is off by a factor of 2048. The tiling check settles it: read as sectors, the four extents tile every one of the 27 containers exactly; read as bytes they overlap each other. Level id is not container order. LEVEL21.WAD reports id 30, so a level was cut from the disc and its number left a gap. Anything keying off file order -- a save slot, the planet select screen -- would mislabel that planet. There is also a second trap in the same area: the ISO filesystem sorts names alphabetically, so LEVEL10 precedes LEVEL2, and any tool needing order has to sort numerically. The header's field at 0x04 is zero on all 27 levels. It is not the container size despite looking like one, so the code reads it as a runtime field and takes the container length from the ISO entry instead. Measured across all 27 levels, three invariants now hold in tests: extents tile in file order; extent coverage plus chunks reaches at least the end of the container (21 levels land exactly on its length, 6 leave 134 to 689 sectors of trailing data); and exactly one level's id differs from its container index. The coverage assertion is >= rather than == on purpose -- 21 levels end one sector past the file end because the header occupies sector 0, and asserting equality would truncate a level's tail. gameplay is left unparsed and that is recorded rather than hidden. It begins "WAD" plus a per-level byte and does not decode with the WAD LZ77 decoder: all 27 levels produce plausible output for the first packet and then fail on a rule the format's own encoder never breaks, which means the input is wrong rather than the decoder. docs/plan/level-assets.md records what is ruled out and what to try next. 66 unit tests, plus 7 integration tests against the real disc. The unpack hash table is unchanged, so this adds parsing without altering the pipeline.
5.7 KiB
The level container (LEVEL<n>.WAD)
Target: SCUS_972.68 v1.01. 27 containers.
The first typed structure in this project, and the one that turns the generic
container table of global-wad.md into named, individually
parseable assets. Implementation: crates/gc-formats/src/level_wad.rs.
Header
All 27 level containers begin with the same 0x60-byte header:
offset size field
0x00 4 header size, 0x60 on every level
0x04 4 zero on every level; written by the game at load time
0x08 4 level id
0x0c 4 reverb configuration index
0x10 8 data geometry: tfrag, tie, shrub, moby, sky, textures
0x18 8 sound_bank
0x20 8 gameplay
0x28 8 occlusion
0x30 48 three {offset, size} chunk descriptors
Sizes here are sector counts. This is the opposite of the generic pair table
in the G/ containers, where sizes are byte counts. Both appear on the same
disc, so reading one as the other produces extents that are wrong by a factor of
2048 and that still look plausible. See "Two fields, two meanings" below.
Extents
The four named extents appear in the file in an order that is not the order of the fields in the header:
sound_bank data gameplay occlusion
Level 0, measured:
| Part | Offset | Sectors | Bytes | Unpacked to |
|---|---|---|---|---|
| sound_bank | 1 | 734 | 1.50 MB | level0/0002.bin |
| data | 735 | 6,484 | 13.28 MB | level0/0001.bin |
| gameplay | 7,219 | 322 | 0.66 MB | level0/0003.bin |
| occlusion | 7,541 | 12 | 0.02 MB | level0/0004.bin |
The numbering of the unpacked files follows the header's field order, so the
data extent is 0001.bin while it is physically second in the container. Both
orderings are worth keeping straight; the typed header exists so callers never
have to.
data is where the level's visible content lives and is the target of the next
phase.
Two fields, two meanings
This is the trap in this format, and it is worth stating plainly because both readings type-check.
| Where | Size field is |
|---|---|
G/*.WAD generic pair table |
bytes |
LEVEL<n>.WAD named extents |
sectors |
Measured across all 92 indexed pair containers, the byte reading places every entry inside its file while the sector reading does not. Measured across all 27 level headers, the sector reading tiles each container exactly.
If the level header's sizes were read as bytes, sound_bank on level 0 would
claim 734 bytes starting at sector 1, and data would start at byte offset 735
— inside the sound bank. The extents would overlap, which is the symptom to
look for.
Invariants, as measured on all 27 levels
These are the checks the parser is validated against, and each holds everywhere rather than on one sample:
-
Extent order and tiling. Each extent's end is the next extent's start. 27 of 27.
-
The runtime sector field at
0x04is zero. 27 of 27. It is not the container size and must not be used as one — the container's length comes from the ISO directory entry. -
Extent coverage.
sound_bank + data + gameplay + occlusion + chunksalways reaches at least the end of the container. 21 of 27 land on exactly the container's sector length; 6 leave between 134 and 689 sectors of trailing data past the last named extent (levels 1, 2, 4, 11, 19, 20).The 21-level case ends one sector past the file end. The header occupies sector 0 and the extents begin at sector 1, so this is ordinary sector rounding rather than an error. The correct assertion is therefore
>=, not equality — asserting equality would truncate the tail of a level. -
Level id equals container index on 26 of 27 levels.
LEVEL21.WADreports id 30.
Level ids are not file order
Point 4 matters enough to state separately. LEVEL21.WAD has id = 30, so the
container's position in the directory is not the level's identity. Some level
was cut from the disc and its number is not reused, leaving a gap.
Anything that keys off file order — a save slot, a planet select screen, the
moby class table — would silently mislabel that planet. Use the header's id.
The ISO filesystem also sorts names alphabetically, so LEVEL10.WAD precedes
LEVEL2.WAD. Tests and tools that care about order must sort numerically.
Chunk descriptors
0x30..0x60 holds three {offset, size} pairs in sector counts, following the
named extents. They are populated on 6 levels (1, 2, 4, 7, 8, 11, 19, 20 — of
which some are the trailing-data levels) and zero elsewhere. Their contents are
not yet identified.
Not yet parsed
data— the geometry. This is the next target and where the visible level comes from: tfrag terrain, tie instances, shrub foliage, moby classes, sky shells and textures.gameplay— beginsWAD+ a per-level id byte + a count, and behaves like an LZ77 stream but does not decode with the WAD decoder (all 27 levels fail with "adjacent literal packets", which the format forbids). It is a different compression, or a different framing. Recorded in../plan/level-assets.md.sound_bank— begins withSBlkat+24andDAWat+36; both signatures are present in all 27 levels.occlusion— 2 to 25 sectors. Small enough to read fully.
Provenance
The header layout matches the GcUyaLevelWadHeader structure documented by the
Wrench project (https://github.com/chaoticgd/wrench, GPLv3). No Wrench code
is copied — see PROVENANCE.md. The structure here is written from the
specification and every field is verified against all 27 levels on this disc.
The two facts that the documentation does not state — the sector-count sizes and
the LEVEL21 id divergence — were measured here.