Add the disc pipeline: identify, unpack and verify
Phase 1 of the roadmap. The disc is now turned into verified asset files, and
the result is checked against a committed hash table rather than against
whatever happens to be on disk.
gc-extract identify disc.iso SCUS_972.68 v1.01 NTSC, ELF SHA-1 matches
gc-extract unpack disc.iso out/ 5135 files, 4,500,106,960 bytes, 10.8s
gc-extract verify out/ table.tsv OK: 5135 files match
All 92 G/*.WAD containers parse. MPEG1 and MPEG2 are raw video and pass
through whole; the other 90 are indexed containers whose extents are extracted
individually.
Formats recovered and written up in docs/formats/:
- iso9660.rs. R&C2 lists its assets in the filesystem, unlike R&C1, so a
directory walk is the primary asset index. Three details that all fail
quietly rather than loudly: directory records store every 32-bit field twice
(little-endian then big-endian), the CD001 signature is at sector offset 1
rather than 0x8001, and the directory flag at record offset 25 is
authoritative where the identifier's trailing slash is not.
- global_wad.rs. Container headers hold a table of extents. All 92 indexed
containers use {offset in sectors, size in bytes}. Measured, not assumed: the
sector-count reading places every entry of all 27 LEVEL<n> and all 27
AUDIO<n> containers out of range, while the byte reading places all of them
correctly. SCENE<n> is the one exception, a flat sector list whose extents
are sized by the next value.
- wad.rs. Insomniac's LZ77 stream: four flag-selected packet classes, with 0-3
trailing literal bytes per packet carried in unused low bits. This is not a
bit stream, which is why it resists the usual LZ77 reading.
- buf.rs. A bounds-checked reader, so untrusted 2003 disc data produces typed
errors rather than panics.
gc-extract carries the three stages and a SHA-1 implementation with FIPS 180-4
test vectors, so identification works without pulling in a crypto dependency.
Verification: 54 unit tests, plus four integration tests that run only when
GC_DISC names a disc. Those four check identification against the recorded
values, cross-check the Rust identifier against tools/identify-disc.py, parse
all 92 containers, and confirm a full unpack matches the committed table. The
committed table is 316 KB of sizes and SHA-1s for all 5,135 files: hashes only,
no game content, so it is safe to version and it turns "matches retail" into
something a test can assert.
docs/plan/disc-pipeline.md records two wrong turns, because both are easy to
repeat and both failed by producing plausible wrong output rather than an
error: the container table was read backwards, and the LZ77 stream was modelled
as a bit stream. The first survived because it happened to be correct on the
one container used for manual checking.
This commit is contained in:
@@ -1,38 +1,4 @@
|
||||
//! Asset format parsers for *Ratchet & Clank: Going Commando* (PS2).
|
||||
//!
|
||||
//! Target: `SCUS_972.68` v1.01, USA Black Label, NTSC 60 Hz. See
|
||||
//! `docs/disc-identification.md`.
|
||||
//!
|
||||
//! This crate reads data structures out of the game and produces plain Rust
|
||||
//! values. It performs no rendering, no simulation and no I/O beyond what the
|
||||
//! caller hands it, which keeps every parser testable against a byte slice.
|
||||
//!
|
||||
//! # What belongs here
|
||||
//!
|
||||
//! * `iso9660` - the disc filesystem. R&C2 lists its assets as real files,
|
||||
//! unlike R&C1, so enumeration works.
|
||||
//! * `hdr` - `RC2.HDR`, the table of contents (408,444 bytes at the
|
||||
//! disc root). The engine addresses assets by sector, and
|
||||
//! this is where those sector numbers come from.
|
||||
//! * `wad` - Insomniac's LZ77 container (`WAD` magic, 16-byte header).
|
||||
//! Carries level core data, audio and video streams.
|
||||
//! * `tfrag` - terrain fragments: LOD chains, strips, morph targets.
|
||||
//! * `tie` - instanced environment geometry ("ties"), with fat vertices.
|
||||
//! * `shrub` - instanced foliage and decorative meshes.
|
||||
//! * `moby` - dynamic actors, including Ratchet, enemies and crates.
|
||||
//! * `moby_anim` - skeletal animation: sequences, frames, blending.
|
||||
//! * `collision` - the collision world: cells, packed vertices, faces.
|
||||
//! * `occlusion` - the PVS grid used for visibility.
|
||||
//! * `sky`, `volumes`, `textures` - sky shells, fog volumes, CSM1 textures.
|
||||
//!
|
||||
//! # Rules
|
||||
//!
|
||||
//! **Bound every read.** Malformed data must produce a typed error, never a
|
||||
//! panic and never a read past the end of the buffer.
|
||||
//!
|
||||
//! **No game data in this repository.** Parsers are written against a copy of
|
||||
//! your own legally obtained disc, and the extracted output stays outside git.
|
||||
//!
|
||||
//! **Verify against the real thing.** Every parser gets a test that loads
|
||||
//! known-good bytes from your disc and checks the parsed result. See
|
||||
//! `docs/formats/` for the specs being implemented.
|
||||
pub mod buf;
|
||||
pub mod global_wad;
|
||||
pub mod iso9660;
|
||||
pub mod wad;
|
||||
|
||||
Reference in New Issue
Block a user