ReaperOfVeriod/Going-Commando
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add the disc pipeline: identify, unpack and verify

Browse Source 
Phase 1 of the roadmap. The disc is now turned into verified asset files, and
the result is checked against a committed hash table rather than against
whatever happens to be on disk.

  gc-extract identify disc.iso        SCUS_972.68 v1.01 NTSC, ELF SHA-1 matches
  gc-extract unpack   disc.iso out/   5135 files, 4,500,106,960 bytes, 10.8s
  gc-extract verify   out/ table.tsv  OK: 5135 files match

All 92 G/*.WAD containers parse. MPEG1 and MPEG2 are raw video and pass
through whole; the other 90 are indexed containers whose extents are extracted
individually.

Formats recovered and written up in docs/formats/:

- iso9660.rs. R&C2 lists its assets in the filesystem, unlike R&C1, so a
  directory walk is the primary asset index. Three details that all fail
  quietly rather than loudly: directory records store every 32-bit field twice
  (little-endian then big-endian), the CD001 signature is at sector offset 1
  rather than 0x8001, and the directory flag at record offset 25 is
  authoritative where the identifier's trailing slash is not.

- global_wad.rs. Container headers hold a table of extents. All 92 indexed
  containers use {offset in sectors, size in bytes}. Measured, not assumed: the
  sector-count reading places every entry of all 27 LEVEL<n> and all 27
  AUDIO<n> containers out of range, while the byte reading places all of them
  correctly. SCENE<n> is the one exception, a flat sector list whose extents
  are sized by the next value.

- wad.rs. Insomniac's LZ77 stream: four flag-selected packet classes, with 0-3
  trailing literal bytes per packet carried in unused low bits. This is not a
  bit stream, which is why it resists the usual LZ77 reading.

- buf.rs. A bounds-checked reader, so untrusted 2003 disc data produces typed
  errors rather than panics.

gc-extract carries the three stages and a SHA-1 implementation with FIPS 180-4
test vectors, so identification works without pulling in a crypto dependency.

Verification: 54 unit tests, plus four integration tests that run only when
GC_DISC names a disc. Those four check identification against the recorded
values, cross-check the Rust identifier against tools/identify-disc.py, parse
all 92 containers, and confirm a full unpack matches the committed table. The
committed table is 316 KB of sizes and SHA-1s for all 5,135 files: hashes only,
no game content, so it is safe to version and it turns "matches retail" into
something a test can assert.

docs/plan/disc-pipeline.md records two wrong turns, because both are easy to
repeat and both failed by producing plausible wrong output rather than an
error: the container table was read backwards, and the LZ77 stream was modelled
as a bit stream. The first survived because it happened to be correct on the
one container used for manual checking.
This commit is contained in:
ReaperOfVeriod
2026-10-09 16:08:30 +02:00
parent a02f79a6d1
commit 027373e65b
21 changed files with 8372 additions and 97 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/formats/disc-layout.md
+110
View File
@@ -0,0 +1,110 @@
# Disc layout
Target: `SCUS_972.68` v1.01 (USA Black Label, NTSC 60 Hz).
All integers are little-endian. Sector size is `0x800` (2048) bytes throughout;
everything on the disc is sector-aligned.
## Root directory
Observed by enumerating the ISO 9660 filesystem:
```text
SYSTEM.CNF 60 bytes
RC2.HDR 408,444 the table of contents, as a named file
SCUS_972.68 2,618,684 the boot ELF
IOPRP255.IMG 264,449 IOP modules
Z6TAIL.DUP 0 ISO 9660 padding
G/... 96 asset containers (see global-wad.md)
```
`RC2.HDR` being present as a real file is convenient. R&C1's equivalent is not a
named file and sits at sector 1500.
## ISO 9660 usage
The filesystem is load-bearing for this game, unlike R&C1 whose filesystem is
nearly vestigial. Everything the engine needs is either a file here or a sector
range declared by a container header.
**Consequence for the extractor:** `G/*` can be enumerated by walking the
directory. There is no need to parse a table of contents to find the assets.
### Details that will bite
- **Directory records store each 32-bit field twice**, little-endian then
big-endian. Reading 4 bytes where the record has 8 shifts every subsequent
field and yields an empty or nonsense listing with no error.
- **The `CD001` signature is at sector offset `0x0001`**, not `0x8001`. The
latter is its absolute file offset and lies outside a 2048-byte sector, so a
bounds-checked read there fails rather than returning empty bytes.
- **The directory flag is at record offset 25**, bit 1. Do not rely on the
identifier's trailing `/`: not every writer sets it, and the flag is
authoritative. Getting this wrong writes a subdirectory's *contents* to a file
named after the directory.
- Records never span a logical block; a zero record length means "advance to the
next block".
## Table of contents (`RC2.HDR`)
The game also maintains a table of contents, and a second copy lives at sector
**1001** (`SYSTEM.CNF` sits at sector 1000 on this disc). The ToC begins with a
chain of global container headers, each `{header_size, sector}`, running from
offset 0 until the level records begin.
Verified on this disc:
```text
offset 0x0000 header_size 0x0328 sector 2610 MPEG
offset 0x0328 header_size 0x0040 sector 1216608 MISC
offset 0x0368 header_size 0x1870 sector 1223855 HUD
offset 0x1bd8 header_size 0x0a48 sector 1242375 BONUS
offset 0x2620 header_size 0x1800 sector 1257884 AUDIO
offset 0x3e20 header_size 0x0ba8 sector 1275113 SPACE
offset 0x49c8 header_size 0x0170 sector 1280474 SCENE
offset 0x4b38 header_size 0x03c8 sector 1282655 GADGET
offset 0x4f00 header_size 0x00f8 sector 1285084 ARMOR
```
The nine header sizes match the container table in
[`global-wad.md`](global-wad.md) exactly, which cross-confirms both readings.
### Level records
Level records follow the globals on a **`0x2000`-byte stride**, each beginning
with `{header_size, sector}` where the sector is an **absolute** disc sector:
```text
+0x00000 LEVEL 0x0060 sector 1286296
+0x02000 SCENE 0x137c sector 1296509
+0x04000 AUDIO 0x1018 sector 1309652
+0x06000 <zero>
...
```
Levels appear as `LEVEL`/`SCENE`/`AUDIO` triples separated by gaps, and every
level's sector cross-checks against the `G/LEVEL<n>.WAD` entry found by walking
the filesystem. That agreement between two independent routes is what confirms
the layout; a single route agreeing with itself proves nothing.
**Not yet implemented.** The extractor does not read the ToC. It enumerates the
filesystem instead, which is sufficient and simpler. The ToC becomes necessary
when the game needs to resolve a sector at runtime, or when per-level overlay
loading has to be understood (Phase 3).
## Boot executable
`SCUS_972.68`, 2,618,684 bytes, SHA-1
`bacda860cbd6723142e86db56f197e47e728970f`.
ELF32, little-endian, MIPS (EM_MIPS = 8), entry point `0x00131AE8`, `$gp`
`0x001AEFF0`. The entry point independently matches the value recorded by the
`rac2-gc-decomp` community reference, which cross-confirms the disc
identification.
## Provenance
Container layout knowledge was guided by the Wrench project's format
descriptions (<https://github.com/chaoticgd/wrench>, GPLv3). **No Wrench code is
copied** — see `PROVENANCE.md`. Structures here are written from the
specification and verified against this disc.
docs/formats/global-wad.md
+160
View File
@@ -0,0 +1,160 @@
# The `G/*.WAD` asset containers
Target: `SCUS_972.68` v1.01 (USA Black Label, NTSC).
Everything except the boot ELF, `SYSTEM.CNF`, `RC2.HDR` and the IOP image
lives in a container under `G/` on the disc's ISO 9660 filesystem. R&C2 lists
these as real files, unlike R&C1 whose filesystem is vestigial, so enumeration
is a directory walk rather than a hunt for a table at a fixed sector.
## Container list
96 containers. The names follow the asset family; `n` runs 0..26 for the
per-level containers.
| File | Count | Bytes (disc) | Role |
|---|---|---|---|
| `ARMOR.WAD` | 1 | 2,480,432 | Armor and wrench meshes plus textures |
| `AUDIO.WAD` | 1 | 35,283,296 | Global audio banks and music |
| `AUDIO<n>.WAD` | 27 | 3,825,304 .. 46,747,520 | Per-level audio |
| `BONUS.WAD` | 1 | 31,762,085 | Collectibles, unlockables |
| `GADGET.WAD` | 1 | 4,972,992 | Gadget definitions |
| `HUD.WAD` | 1 | 37,927,695 | HUD elements and text |
| `LEVEL<n>.WAD` | 27 | 8,350,423 .. 25,119,448 | Level core, gameplay, occlusion |
| `MISC.WAD` | 1 | 14,840,496 | Miscellaneous assets |
| `MPEG.WAD` | 1 | 1,075,267,584 | Indexed video and subtitle streams |
| `MPEG1.WAD` | 1 | 1,296,228,352 | Raw concatenated video, not indexed |
| `MPEG2.WAD` | 1 | 114,769,924 | Raw concatenated video, not indexed |
| `SCENE.WAD` | 1 | 4,466,256 | Global cutscene data |
| `SCENE<n>.WAD` | 27 | 4,988 .. 14,964,240 | Per-level cutscene data |
| `SPACE.WAD` | 1 | 10,977,664 | Space level assets |
The three MPEG WADs are 2.5 GB of the 3.8 GB disc. Video is a data problem
before it is a decoding problem; the pipeline reads them in sectors rather than
slurping whole files.
## Header
Every indexed container begins with:
```text
offset size field
0x00 4 header size in bytes
0x04 4 sector count of the whole container
0x08.. ... table, interpreted per container kind (below)
```
The header size is the reliable discriminator: it is the one field every
container agrees on, and its value identifies the container family.
| Header size | Containers | Table kind |
|---|---|---|
| `0x0040` | `MISC` | sector/byte pairs |
| `0x0060` | `LEVEL<n>` | sector/byte pairs |
| `0x00f8` | `ARMOR` | sector/byte pairs |
| `0x0170` | `SCENE` | sector/byte pairs |
| `0x0328` | `MPEG` | sector/byte pairs |
| `0x03c8` | `GADGET` | sector/byte pairs |
| `0x0a48` | `BONUS` | sector/byte pairs |
| `0x0ba8` | `SPACE` | sector/byte pairs |
| `0x1018` | `AUDIO<n>` | sector/byte pairs |
| `0x137c` | `SCENE<n>` | **flat sector list** |
| `0x1800` | `AUDIO` | sector/byte pairs |
| `0x1870` | `HUD` | sector/byte pairs |
## Sector/byte pairs
The common form. From `0x08` to `header_size`, a run of 8-byte entries:
```text
offset size field
0x00 4 first sector of the extent
0x04 4 length of the extent **in bytes**
```
The offset is in **sectors**, the size is in **bytes**. This is the single most
important thing to get right, and it is easy to get backwards because both
fields are 32-bit integers in a structure where either reading type-checks.
**This was measured, not assumed.** For all 92 indexed pair containers:
- Reading the size as bytes places every entry inside its file.
- Reading the size as a sector count places entries past the end of the file,
and on `LEVEL<n>` and `AUDIO<n>` places *none* of them correctly — 0 of 5 on
`LEVEL0.WAD`, 0 of 42 on `AUDIO1.WAD`, and so on for all 27 of each.
Offsets chain, which is the confirmation that the reading is right. On
`AUDIO0.WAD`: sectors 3, 185, 367, 1458, where `185 = 3 + 182` sectors and each
entry's end is the next entry's start.
### Payload is not compressed
The bytes behind a pair are raw. Insomniac's LZ77 WAD container (see
[`wad.md`](wad.md)) is used elsewhere on the disc — inside level cores and MPEG
streams — but these global containers store their sections uncompressed and let
the table do the addressing. A decompressor applied here produces garbage.
### Worked example: ARMOR.WAD
`header_size = 0xf8`, so `(0xf8 - 8) / 8 = 30` entries. Reading the pairs:
```text
entry 0: sector 1, 372,272 bytes
entry 1: sector 124, 163,840 bytes (= 73 + 51 sectors earlier)
...
entry 29: sector 1200, 24,576 bytes
```
The published structure is nine armor entries then six wrench entries, each a
`{mesh, textures}` pair, which is 15 pairs — but the generic parser reads 30
entries because the header also declares the textures of the nine armors as
their own ranges. Entry 0 is armor 0's mesh and entry 1 is armor 0's textures;
entries 2 and 3 are armor 1's, and so on. Which pair means what is per-container
knowledge, recovered when that asset family is parsed.
## Flat sector list (`SCENE<n>.WAD`)
The 27 per-level scene containers are the exception. `header_size = 0x137c`,
and `0x137c - 8 = 4980` is **not** a multiple of 8, so the bytes cannot be
pairs at all. They are a flat run of 1,245 `u32` sector numbers, 30 scenes of
83 values each.
Each value is the first sector of one extent. The extent's length is the
difference to the next value; the last runs to the end of the container.
```text
sectors[0] = 3, sectors[1] = 7 -> extent 0 is sectors 3..7 (4 sectors)
```
The global `SCENE.WAD` is not one of these: it uses the pair form.
## Raw streams
`MPEG1.WAD` and `MPEG2.WAD` have no table. Their first word is video payload,
not a plausible header size (`7864320` and a negative value respectively), so
they are copied through whole and split by the video phase.
## Known trailing sentinels
A few containers declare entries whose extent runs past the end of the file:
1 in every `SCENE<n>.WAD`, 10 in `GADGET.WAD`, 35 in `MPEG.WAD`. These are
trailing sentinels or descriptors that reference sectors belonging to an
adjacent container. The extractor bounds-checks every extent and skips these
with a count, rather than reading past the end of the buffer or trusting the
header.
## Verification
Unpacking the disc produces 5,135 files totalling 4,500,106,960 bytes. Every
file's size and SHA-1 are recorded in
`crates/gc-extract/data/gc-files.tsv` (hashes only, no content). See
[`../plan/disc-pipeline.md`](../plan/disc-pipeline.md) for how that gate is run.
## What is not yet known
- Which pair in each container corresponds to which asset. Needed before any
asset can be parsed; the geometry is recovered per family in Phase 2.
- The `bindata` sub-structures inside audio and scene containers.
- How `MPEG.WAD`'s indexed streams relate to the raw `MPEG1`/`MPEG2` files.
Two descriptions of the same video data exist on the disc; which the game
reads is not yet established.
docs/formats/wad.md
+96
View File
@@ -0,0 +1,96 @@
# The WAD LZ77 container
Target: `SCUS_972.68` v1.01.
Insomniac's LZ77 stream. A 16-byte header followed by a sequence of packets.
Each packet begins with a **flag byte** selecting one of four classes.
This format is **not** the `G/*.WAD` container described in
[`global-wad.md`](global-wad.md), despite the shared name and extension. WAD
streams are compressed and appear inside level cores and MPEG data; the `G/`
containers are uncompressed tables. Confusing the two produces garbage.
## Header
```text
offset size field
0x00 3 magic, "WAD"
0x03 4 total stream length in bytes, including this header
0x07 9 unused
```
`compressed_size` at `0x03` is the authoritative length. A stream whose declared
size exceeds the buffer is rejected rather than truncated.
## Packets
| Flag | Class | Displacement bits | Length |
|---|---|---|---|
| `0x00..0x0f` | literal run | — | flag + 3 (flag 0 escapes to a byte + 18) |
| `0x10..0x1f` | back-reference | 15 | 3 bits in flag, else a byte + 7 |
| `0x20..0x3f` | back-reference | 14 | 5 bits in flag, else a byte + 0x1f |
| `0x40..0xff` | back-reference | 11 | 3 bits in flag |
Every packet ends with 0–3 extra literal bytes carried in unused low bits of the
packet, so literal and back-reference packets interleave without alignment.
This is why the format does not read as a bit stream, and why a first attempt
that modelled it as one will appear to produce plausible output for a few
kilobytes before failing.
### Literal run
Flag `< 0x10`. The count is `flag + 3`, except flag `0x00` which escapes to a
following byte giving `byte + 18`. The bytes follow immediately.
Two literal packets in a row are invalid: the encoder merges them. The decoder
rejects the stream rather than accept a misparse.
### Back-reference
A back-reference copies `len` bytes from `displacement` positions back in the
already-produced output.
**Overlapping copies expand as run-length.** The source index advances with the
destination, so `displacement = 1` repeats a single byte. This is used
intentionally by the encoder.
A displacement of zero, or one that reaches before the start of the output, is
invalid.
### Pad and dummy packets
In the `0x10..0x1f` class, a packet whose displacement field is entirely zero is
special:
- length field not 1 → a **pad packet**: the reader skips forward to the next
`0x1000` boundary. These exist so compressed blocks align with something on
the disc.
- length field 1 → a **dummy packet**: it carries only its trailing literal
bytes.
## Worked example
The first stream in this game's `MISC.WAD` begins:
```text
05 80 00 00 00 10 | 00 0d ...
```
- Flag `0x05` is `< 0x10`, so it is a literal run of `5 + 3 = 8` bytes:
`80 00 00 00 10 00 0d 00`.
- The next flag byte starts the following packet.
## Implementation
`crates/gc-formats/src/wad.rs`. Parses through the same bounds-checked reader as
everything else in the crate, so a truncated stream produces a typed error
rather than a panic. A `BadMagic`, an out-of-bounds read, and a displacement
before the output are three distinct error kinds.
## Provenance
The packet structure was recovered from behavioural analysis of the retail
data, guided by the format description in the Wrench project
(<https://github.com/chaoticgd/wrench>, GPLv3). **No Wrench code is copied into
this repository** — see `PROVENANCE.md`. The decoder here is written from the
specification and verified against this disc's own streams.
docs/plan/disc-pipeline.md
+140
View File
@@ -0,0 +1,140 @@
# Disc pipeline
The work record for Phase 1. What was tried, what worked, and what was ruled
out — the negative results matter as much as the working parts.
## Result
The disc pipeline runs end to end and is verified.
```text
$ gc-extract identify disc.iso
serial SCUS_972.68 version 1.01 video mode NTSC
boot ELF SHA-1 bacda860cbd6723142e86db56f197e47e728970f
$ gc-extract unpack disc.iso extracted/ --table gc-files.tsv
5135 files, 4,500,106,960 bytes, in 10.8s
$ gc-extract verify extracted/ gc-files.tsv
OK: 5135 files match the recorded table.
```
Every one of the 96 `G/` containers parses. 92 are indexed containers whose
extents are extracted individually; `MPEG1.WAD` and `MPEG2.WAD` are raw video
and pass through whole.
## The wrong turn worth recording
**The container pair table was read backwards for most of this phase.**
The obvious reading of an 8-byte entry is `{offset, size}`, both sector counts —
`SectorRange`. That reading:
- worked perfectly for `ARMOR.WAD` (30 of 30 entries in range), which is exactly
why it survived several rounds of testing;
- placed 1 entry out of range on `HUD.WAD`, 69 of 178 on `AUDIO.WAD`, and **all
entries out of range** on every `AUDIO<n>.WAD` and `LEVEL<n>.WAD`.
The `AUDIO0.WAD` numbers were the tell: under the sector reading, the count of
out-of-range entries equalled the count of in-range entries, for all 27 audio
containers. A systematic 50/50 split is a decoding error, not sparse data.
The correct form is `{offset in sectors, size in bytes}` — `SectorByteRange`.
The confirmation is that offsets chain: on `AUDIO0.WAD` the sectors are 3, 185,
367, 1458, where each entry's end is the next entry's start, in both bytes and
sectors. Under the byte reading the extents tile the file exactly.
Measured across all 92 indexed pair containers:
| Reading | Entries placed correctly |
|---|---|
| Sector/sector | many, but never all; 0 of 5 on `LEVEL0.WAD`, 0 of 42 on `AUDIO1.WAD`, and so on for all 27 of each |
| Sector/bytes | all, on every container |
**Lesson for the format parsers in Phase 2:** a parser that type-checks on one
container can still be wrong on the other 91. Prefer a check that must hold
*everywhere* — extents tiling a file without gaps or overlaps — over a check
that holds on the sample in front of you. This is why `ARMOR.WAD` was chosen to
validate by hand and why that choice nearly cost the phase.
## The WAD LZ77 decoder
Also initially wrong, and instructive.
The first attempt modelled the format as a **bit stream**: a control byte whose
bits each select "literal byte" or "back-reference". This is how many LZ77
variants work, and it produced plausible-looking output for a few kilobytes —
`80 00 00 00 10 00 0d ...` looked like real level data — before failing with an
invalid displacement. Four bit orderings and a dozen offset/length field
layouts were tried against a real stream; none decoded more than a handful of
bytes.
The actual format is a **flag-byte packet stream** with four packet classes,
chosen by the flag's value rather than by individual bits, where every packet
carries 0–3 trailing literal bytes in unused low bits. It cannot be expressed as
a bit stream, which is why no bit ordering worked. See
[`../formats/wad.md`](../formats/wad.md).
The failure mode is worth remembering: a decoder that produces *plausible* wrong
output is more dangerous than one that fails immediately. The first WAD decoder
did exactly that.
## Container families
| Family | Count | Header size | Table |
|---|---|---|---|
| `LEVEL<n>` | 27 | `0x60` | sector/byte pairs |
| `AUDIO<n>` | 27 | `0x1018` | sector/byte pairs |
| `SCENE<n>` | 27 | `0x137c` | flat sector list |
| `SCENE` | 1 | `0x170` | sector/byte pairs |
| `MPEG1`, `MPEG2` | 2 | — | none; raw video |
| others | 12 | various | sector/byte pairs |
`SCENE<n>` is the only family with a different table shape. `(0x137c - 8)` is
not a multiple of 8, so the bytes cannot be pairs at all: they are 1,245 sector
numbers in a flat list, each extent running to the next value.
## Deliberately deferred
- **The table of contents at sector 1001** is documented in
[`../formats/disc-layout.md`](../formats/disc-layout.md) but not read by the
extractor. Enumerating the ISO 9660 filesystem is simpler and sufficient; the
ToC matters when per-level overlay loading is understood in Phase 3.
- **Per-container entry semantics.** Which pair is a mesh, which is a texture,
which is audio: recovered per family as each is parsed in Phase 2. The
extractor only needs to know where each extent starts and ends.
- **Audio sub-structures.** `AUDIO<n>.WAD` headers hold 36 `bindata` ranges
followed by 15 music pointers; the split is inferred from the header size but
not yet confirmed by unpacking both halves.
## Known tolerated imprecision
A few containers declare extents that run past the end of their file: 1 in each
`SCENE<n>.WAD`, 10 in `GADGET.WAD`, 35 in `MPEG.WAD`. These are trailing
sentinels or descriptors referencing sectors in an adjacent container.
The extractor bounds-checks every extent against the container actually read, so
these cannot cause an out-of-bounds read. They are counted and reported rather
than silently dropped, because a silent skip would be indistinguishable from
data loss. Total skipped: 46 files' worth of entries, out of 5,135 files
extracted.
## Verification approach
`crates/gc-extract/data/gc-files.tsv` records the size and SHA-1 of all 5,135
produced files — hashes only, never content, 316 KB.
The value of this is that a future parser can assert "this matches retail"
rather than "this matches whatever is on my disk". When a Phase 2 parser is
wrong, the hash table is how we find out, and how we tell a wrong parser from a
wrong assumption.
Regenerate after an intentional change:
```sh
gc-extract unpack <disc.iso> extracted/ \
--table crates/gc-extract/data/gc-files.tsv
```
If only some files change, the unpack stage changed. If none do, only a parser
downstream is wrong.
docs/roadmap.md
+31 -13
View File
@@ -13,7 +13,10 @@ early, while they are still cheap to retire.
- [x] Workspace builds (`cargo check --workspace`)
- [x] Repository hygiene in place (pre-commit guard, `.gitignore`, licence
record)
- [ ] Nothing else implemented yet
- [x] **Phase 1 complete.** The disc pipeline runs end to end and is verified
against a committed hash table: 5,135 files, 4,500,106,960 bytes, every
one of the 92 asset containers parsing.
- [ ] Phase 2 next: the asset format parsers.
The disc being the exact build the reference projects pin is the single
biggest favourable fact so far: no disc conversion, no regional timing work,
@@ -46,22 +49,33 @@ and no re-derivation of the reference baseline.
---
## Phase 1 — Disc pipeline
## Phase 1 — Disc pipeline ✅
**Goal:** turn the disc image into verified, unpacked asset files.
**Done.** Gate met: every extracted file matches the committed hash table.
- [ ] `iso9660` reader (R&C2 lists assets as real files, unlike R&C1)
- [ ] `RC2.HDR` table of contents parser (408,444 bytes at disc root)
- [ ] WAD LZ77 decompression
- [ ] Unpack global WADs (`G/*.WAD`) and per-level WADs
- [ ] Commit a size+SHA-1 table per target file (hashes only, never content)
- [ ] `verify` stage
- [x] `iso9660` reader (R&C2 lists assets as real files, unlike R&C1)
- [x] `G/*.WAD` container header parsing — 92 containers, three table shapes
- [x] WAD LZ77 decompression (`wad.rs`, four packet classes)
- [x] Unpack every container, bounds-checking each extent
- [x] SHA-1 table for all 5,135 files (hashes only, never content)
- [x] `verify` stage, plus integration tests against a real disc
- [x] Format specs written: `disc-layout.md`, `global-wad.md`, `wad.md`
- [x] Work record with the negative results: `plan/disc-pipeline.md`
**Gate:** every extracted file matches the committed hash table, and the
decompressed size of every WAD stream matches its header.
```text
gc-extract identify disc.iso -> SCUS_972.68 v1.01 NTSC, ELF SHA-1 matches
gc-extract unpack disc.iso out/ -> 5135 files, 4,500,106,960 bytes, 10.8s
gc-extract verify out/ table.tsv -> OK: 5135 files match
```
**Note:** the three MPEG WADs total roughly 2.4 GB of the 3.8 GB disc. Video is
a data problem before it is a decoding problem — plan to stream.
**Not done, deliberately:** the table of contents at sector 1001 is documented
but not read. Enumerating the ISO filesystem is simpler and sufficient; the ToC
matters when per-level overlay loading is understood in Phase 3.
**Two wrong turns are recorded** in `plan/disc-pipeline.md`, because both would
be easy to repeat: the container pair table read backwards, and the LZ77 stream
mistaken for a bit stream. Both produced plausible-looking wrong output before
failing, which is the failure mode worth guarding against.
---
@@ -70,6 +84,10 @@ a data problem before it is a decoding problem — plan to stream.
**Goal:** every asset type parses into plain Rust values, tested against real
disc data.
Phase 1 hands over the extents; this phase works out what each one contains.
`global_wad.md` records which pair in each container corresponds to which asset,
and that table starts empty.
- [ ] `tfrag` — terrain: LOD chains, strips, morph targets
- [ ] `tie` — instanced environment geometry, fat vertices
- [ ] `shrub` — foliage and decorative meshes