Fix level extents read as bytes, and extract the level geometry

A defect in the Phase 1 extractor, found by reading the data extent's contents:
level extents are declared in sector counts, the generic G/ pair table uses byte
counts, and level containers were being routed through the generic path. So
LEVEL0.WAD's data extent -- 6,484 sectors -- was taken as 6,484 bytes. Every
test still passed, because a truncated file is still a file.

The extent's own header gave it away: it declares an uncompressed total of
13,277,920 bytes, and 13,277,920 / 6,484 is 2047.7. A ratio landing on 2048 is
a unit error, not a compression ratio. Across 27 levels this had been discarding
roughly 300 MB of level content.

Level containers now go through the typed LevelHeader instead. Level 0 yields:

  data.bin         13,279,232   the data extent
  core_data.wad     9,738,678   the geometry, compressed
  core_data.bin    16,969,296   the geometry, decompressed
  core_index.bin       31,600   the geometry directory
  gameplay.bin       659,456
  sound_bank.bin   1,503,232

core_data is a WAD LZ77 stream -- the format already implemented for the game --
and it decompresses cleanly. The first real level content this project has
recovered.

New: LevelDataHeader parses the data extent's own header, a table of byte ranges
naming the level overlay executable, the geometry directory, the GS RAM image,
the HUD banks and the geometry. Note this is a third meaning for a "size" field
on the same disc, after sector counts in the level header and byte counts in the
generic table; all three are documented together in level-data.md so the next
parser does not rediscover this the hard way.

The geometry directory's offsets were checked against the geometry rather than
taken on trust: all twelve land inside the 16.9 MB of decompressed core_data,
which is what confirms the header is being read correctly. The class tables
themselves do not yet decode under the documented layout, and that is recorded
as open work in plan/level-assets.md rather than papered over.

75 unit tests, 9 integration tests against the real disc. Hash table regenerated
at 5,159 files and 5,671,915,799 bytes -- hashes only, no content. Extraction
now writes to ~/Documents/gc-extracted rather than /tmp, which is 16 GB and did
not have room for a 5.3 GB output.
This commit is contained in:
2026-10-09 16:28:26 +02:00
parent a9e43e39c1
commit 16c04db559
9 changed files with 868 additions and 147 deletions
+33
View File
@@ -3,6 +3,39 @@
Phase 2 work record. What has been established about the four extents inside
each `LEVEL<n>.WAD`, and what remains.
## A bug found and fixed
Reading the `data` extent's contents exposed a defect in the Phase 1 extractor.
Level extents are declared in **sector counts**; the generic `G/` pair table
uses **byte counts**. The extractor routed level containers through the generic
path, so `LEVEL0.WAD`'s `data` extent — 6,484 sectors — was taken as 6,484
*bytes*. Level 0 yielded a 6,484-byte "geometry" file instead of 13,279,232
bytes. Across 27 levels that silently discarded roughly 300 MB of level content
while every test still passed.
The tell was arithmetic, not a crash: the extent's own header declared an
uncompressed total of 13,277,920 bytes, and the ratio against the 6,484-byte
file was 2047.7× — one sector, to within rounding. A ratio that lands on 2048
is a unit error.
Fixed by extracting level containers through the typed `LevelHeader` instead of
the generic table. Level 0 now yields:
```text
data.bin 13,279,232 the level container's data extent
core_data.wad 9,738,678 the geometry, compressed
core_data.bin 16,969,296 the geometry, decompressed
core_index.bin 31,600 the geometry directory
gameplay.bin 659,456
sound_bank.bin 1,503,232
```
**Lesson:** a unit error is invisible to a type system and to a passing test. The
check that catches it is arithmetic against an independent number the file
itself declares. Any extent whose contents describe a larger structure should
have that structure's own total compared against the extent.
## Done: the container header
`crates/gc-formats/src/level_wad.rs` parses the 0x60-byte header on all 27