52 Commits

Include renames

Author	SHA1	Message	Date
ReaperOfVeriod	a9e43e39c1	Parse the level container header, naming each level's four extents ... The first typed structure read from real data. Every one of the 27 level containers begins with the same 0x60-byte header, which turns the generic pair table into named assets: data, sound_bank, gameplay and occlusion, plus three chunk descriptors. Two things about this header are worth stating because both are silent failures rather than errors. Sizes here are sector counts, not byte counts. The generic G/ pair table on the same disc uses byte counts, so both readings type-check and the wrong one is off by a factor of 2048. The tiling check settles it: read as sectors, the four extents tile every one of the 27 containers exactly; read as bytes they overlap each other. Level id is not container order. LEVEL21.WAD reports id 30, so a level was cut from the disc and its number left a gap. Anything keying off file order -- a save slot, the planet select screen -- would mislabel that planet. There is also a second trap in the same area: the ISO filesystem sorts names alphabetically, so LEVEL10 precedes LEVEL2, and any tool needing order has to sort numerically. The header's field at 0x04 is zero on all 27 levels. It is not the container size despite looking like one, so the code reads it as a runtime field and takes the container length from the ISO entry instead. Measured across all 27 levels, three invariants now hold in tests: extents tile in file order; extent coverage plus chunks reaches at least the end of the container (21 levels land exactly on its length, 6 leave 134 to 689 sectors of trailing data); and exactly one level's id differs from its container index. The coverage assertion is >= rather than == on purpose -- 21 levels end one sector past the file end because the header occupies sector 0, and asserting equality would truncate a level's tail. gameplay is left unparsed and that is recorded rather than hidden. It begins "WAD" plus a per-level byte and does not decode with the WAD LZ77 decoder: all 27 levels produce plausible output for the first packet and then fail on a rule the format's own encoder never breaks, which means the input is wrong rather than the decoder. docs/plan/level-assets.md records what is ruled out and what to try next. 66 unit tests, plus 7 integration tests against the real disc. The unpack hash table is unchanged, so this adds parsing without altering the pipeline.	2026-10-09 16:20:25 +02:00
ReaperOfVeriod	027373e65b	Add the disc pipeline: identify, unpack and verify ... Phase 1 of the roadmap. The disc is now turned into verified asset files, and the result is checked against a committed hash table rather than against whatever happens to be on disk. gc-extract identify disc.iso SCUS_972.68 v1.01 NTSC, ELF SHA-1 matches gc-extract unpack disc.iso out/ 5135 files, 4,500,106,960 bytes, 10.8s gc-extract verify out/ table.tsv OK: 5135 files match All 92 G/*.WAD containers parse. MPEG1 and MPEG2 are raw video and pass through whole; the other 90 are indexed containers whose extents are extracted individually. Formats recovered and written up in docs/formats/: - iso9660.rs. R&C2 lists its assets in the filesystem, unlike R&C1, so a directory walk is the primary asset index. Three details that all fail quietly rather than loudly: directory records store every 32-bit field twice (little-endian then big-endian), the CD001 signature is at sector offset 1 rather than 0x8001, and the directory flag at record offset 25 is authoritative where the identifier's trailing slash is not. - global_wad.rs. Container headers hold a table of extents. All 92 indexed containers use {offset in sectors, size in bytes}. Measured, not assumed: the sector-count reading places every entry of all 27 LEVEL<n> and all 27 AUDIO<n> containers out of range, while the byte reading places all of them correctly. SCENE<n> is the one exception, a flat sector list whose extents are sized by the next value. - wad.rs. Insomniac's LZ77 stream: four flag-selected packet classes, with 0-3 trailing literal bytes per packet carried in unused low bits. This is not a bit stream, which is why it resists the usual LZ77 reading. - buf.rs. A bounds-checked reader, so untrusted 2003 disc data produces typed errors rather than panics. gc-extract carries the three stages and a SHA-1 implementation with FIPS 180-4 test vectors, so identification works without pulling in a crypto dependency. Verification: 54 unit tests, plus four integration tests that run only when GC_DISC names a disc. Those four check identification against the recorded values, cross-check the Rust identifier against tools/identify-disc.py, parse all 92 containers, and confirm a full unpack matches the committed table. The committed table is 316 KB of sizes and SHA-1s for all 5,135 files: hashes only, no game content, so it is safe to version and it turns "matches retail" into something a test can assert. docs/plan/disc-pipeline.md records two wrong turns, because both are easy to repeat and both failed by producing plausible wrong output rather than an error: the container table was read backwards, and the LZ77 stream was modelled as a bit stream. The first survived because it happened to be correct on the one container used for manual checking.	2026-10-09 16:08:30 +02:00