Add the disc pipeline: identify, unpack and verify
Phase 1 of the roadmap. The disc is now turned into verified asset files, and
the result is checked against a committed hash table rather than against
whatever happens to be on disk.
gc-extract identify disc.iso SCUS_972.68 v1.01 NTSC, ELF SHA-1 matches
gc-extract unpack disc.iso out/ 5135 files, 4,500,106,960 bytes, 10.8s
gc-extract verify out/ table.tsv OK: 5135 files match
All 92 G/*.WAD containers parse. MPEG1 and MPEG2 are raw video and pass
through whole; the other 90 are indexed containers whose extents are extracted
individually.
Formats recovered and written up in docs/formats/:
- iso9660.rs. R&C2 lists its assets in the filesystem, unlike R&C1, so a
directory walk is the primary asset index. Three details that all fail
quietly rather than loudly: directory records store every 32-bit field twice
(little-endian then big-endian), the CD001 signature is at sector offset 1
rather than 0x8001, and the directory flag at record offset 25 is
authoritative where the identifier's trailing slash is not.
- global_wad.rs. Container headers hold a table of extents. All 92 indexed
containers use {offset in sectors, size in bytes}. Measured, not assumed: the
sector-count reading places every entry of all 27 LEVEL<n> and all 27
AUDIO<n> containers out of range, while the byte reading places all of them
correctly. SCENE<n> is the one exception, a flat sector list whose extents
are sized by the next value.
- wad.rs. Insomniac's LZ77 stream: four flag-selected packet classes, with 0-3
trailing literal bytes per packet carried in unused low bits. This is not a
bit stream, which is why it resists the usual LZ77 reading.
- buf.rs. A bounds-checked reader, so untrusted 2003 disc data produces typed
errors rather than panics.
gc-extract carries the three stages and a SHA-1 implementation with FIPS 180-4
test vectors, so identification works without pulling in a crypto dependency.
Verification: 54 unit tests, plus four integration tests that run only when
GC_DISC names a disc. Those four check identification against the recorded
values, cross-check the Rust identifier against tools/identify-disc.py, parse
all 92 containers, and confirm a full unpack matches the committed table. The
committed table is 316 KB of sizes and SHA-1s for all 5,135 files: hashes only,
no game content, so it is safe to version and it turns "matches retail" into
something a test can assert.
docs/plan/disc-pipeline.md records two wrong turns, because both are easy to
repeat and both failed by producing plausible wrong output rather than an
error: the container table was read backwards, and the LZ77 stream was modelled
as a bit stream. The first survived because it happened to be correct on the
one container used for manual checking.
This commit is contained in:
@@ -12,3 +12,10 @@ anyhow.workspace = true
|
||||
thiserror.workspace = true
|
||||
gc-formats.workspace = true
|
||||
sha1.workspace = true
|
||||
|
||||
[[bin]]
|
||||
name = "gc-extract"
|
||||
path = "src/main.rs"
|
||||
|
||||
[dev-dependencies]
|
||||
gc-formats.workspace = true
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,388 @@
|
||||
//! Stage 1: identify the disc.
|
||||
//!
|
||||
//! Reads only the ISO 9660 structures, `SYSTEM.CNF` and the boot ELF — a few
|
||||
//! megabytes of seeks — so it is near-instant on a local image.
|
||||
//!
|
||||
//! The reference implementation of this stage is `tools/identify-disc.py`; the
|
||||
//! two are checked against each other in `tests/disc.rs`.
|
||||
|
||||
use std::path::Path;
|
||||
|
||||
use gc_formats::iso9660::IsoImage;
|
||||
|
||||
/// The build this project targets.
|
||||
pub const TARGET_SERIAL: &str = "SCUS_972.68";
|
||||
pub const TARGET_VERSION: &str = "1.01";
|
||||
/// Measured from a retail Black Label v1.01 image. See `docs/disc-identification.md`.
|
||||
pub const TARGET_BOOT_ELF_SHA1: &str = "bacda860cbd6723142e86db56f197e47e728970f";
|
||||
|
||||
/// What we learned about a disc.
|
||||
#[derive(Debug, Clone)]
|
||||
pub struct DiscInfo {
|
||||
pub path: String,
|
||||
pub image_size: u64,
|
||||
pub volume_id: String,
|
||||
/// Product serial, i.e. the boot executable's file name.
|
||||
pub serial: String,
|
||||
/// `VER` from `SYSTEM.CNF`.
|
||||
pub version: String,
|
||||
/// `VMODE` from `SYSTEM.CNF`: `NTSC` or `PAL`, possibly empty.
|
||||
pub vmode: String,
|
||||
pub boot_elf_size: u32,
|
||||
pub boot_elf_sha1: String,
|
||||
}
|
||||
|
||||
impl DiscInfo {
|
||||
/// Is this the exact build the reference material describes?
|
||||
pub fn is_target(&self) -> bool {
|
||||
self.serial == TARGET_SERIAL && self.version == TARGET_VERSION
|
||||
}
|
||||
|
||||
/// True when the video mode makes the game run at 50 Hz.
|
||||
///
|
||||
/// This matters more than a label difference: a PAL disc runs the game at
|
||||
/// 50 Hz, which puts speed-sensitive movement such as the chargeboots at
|
||||
/// 5/6 of their intended rate.
|
||||
pub fn is_pal(&self) -> bool {
|
||||
self.vmode.eq_ignore_ascii_case("PAL")
|
||||
}
|
||||
|
||||
/// A short human-readable summary of whether this disc can be used.
|
||||
///
|
||||
/// The video mode is reported for every disc, not just the target: a 50 Hz
|
||||
/// PAL disc changes movement timing, which is the kind of detail that is
|
||||
/// easy to forget and hard to notice later.
|
||||
pub fn verdict(&self) -> String {
|
||||
let rate = if self.is_pal() {
|
||||
" 50 Hz: speed-sensitive movement runs at 5/6 speed, so timing constants differ."
|
||||
} else if self.vmode.is_empty() {
|
||||
""
|
||||
} else {
|
||||
" 60 Hz."
|
||||
};
|
||||
|
||||
if self.is_target() && self.boot_elf_sha1 == TARGET_BOOT_ELF_SHA1 {
|
||||
return format!(
|
||||
"USA Black Label v1.01, NTSC 60 Hz: the build we target. Boot ELF SHA-1 matches.{rate}"
|
||||
);
|
||||
}
|
||||
if self.serial == "SCUS_972.68GH" {
|
||||
return format!(
|
||||
"Greatest Hits v2.00. This is a different build: the chargeboots were patched, \
|
||||
and reference material built for v1.01 will not match.{rate}"
|
||||
);
|
||||
}
|
||||
if self.serial == TARGET_SERIAL {
|
||||
return format!(
|
||||
"Serial matches but version is {} (we target {TARGET_VERSION}).{rate}",
|
||||
if self.version.is_empty() {
|
||||
"unknown"
|
||||
} else {
|
||||
&self.version
|
||||
}
|
||||
);
|
||||
}
|
||||
format!("Not the target build. We target {TARGET_SERIAL} v{TARGET_VERSION}.{rate}")
|
||||
}
|
||||
}
|
||||
|
||||
/// Extract `KEY = value` from `SYSTEM.CNF`, stopping at `;`, CR or LF.
|
||||
fn cnf_value(cnf: &str, key: &str) -> String {
|
||||
for line in cnf.lines() {
|
||||
if let Some((k, v)) = line.split_once('=') {
|
||||
if k.trim() == key {
|
||||
return v.trim().split(';').next().unwrap_or("").trim().to_string();
|
||||
}
|
||||
}
|
||||
}
|
||||
String::new()
|
||||
}
|
||||
|
||||
/// Identify a disc image, or explain why it cannot be used.
|
||||
pub fn identify(path: &Path) -> Result<DiscInfo, String> {
|
||||
let iso = IsoImage::open(path).map_err(|e| format!("cannot read {}: {e}", path.display()))?;
|
||||
|
||||
let cnf_entry = iso
|
||||
.find("SYSTEM.CNF")
|
||||
.ok_or_else(|| "no SYSTEM.CNF: not a PlayStation 2 game disc".to_string())?;
|
||||
let cnf = gc_formats::iso9660::read_at(path, cnf_entry.lba, cnf_entry.size)
|
||||
.map_err(|e| format!("cannot read SYSTEM.CNF: {e}"))?;
|
||||
let cnf = String::from_utf8_lossy(&cnf);
|
||||
|
||||
let boot = cnf_value(&cnf, "BOOT2");
|
||||
if boot.is_empty() {
|
||||
return Err("SYSTEM.CNF has no BOOT2 line: not a PlayStation 2 game disc".into());
|
||||
}
|
||||
let boot_path = boot
|
||||
.strip_prefix("cdrom0:")
|
||||
.unwrap_or(&boot)
|
||||
.replace('\\', "/")
|
||||
// SYSTEM.CNF writes the boot path rooted, e.g. `cdrom0:\SCUS_972.68`.
|
||||
// Strip any leading slash so it matches an ISO directory name.
|
||||
.trim_start_matches('/')
|
||||
.to_string();
|
||||
let serial = boot_path.rsplit('/').next().unwrap_or("").to_string();
|
||||
|
||||
let entry = iso
|
||||
.find(&boot_path)
|
||||
.ok_or_else(|| format!("SYSTEM.CNF boots {boot_path}, which is not on the disc"))?;
|
||||
let elf = gc_formats::iso9660::read_at(path, entry.lba, entry.size)
|
||||
.map_err(|e| format!("cannot read the boot executable {serial}: {e}"))?;
|
||||
|
||||
if elf.len() < 52 || &elf[..4] != b"\x7fELF" {
|
||||
return Err(format!("{serial} is not an ELF executable"));
|
||||
}
|
||||
|
||||
Ok(DiscInfo {
|
||||
path: path.display().to_string(),
|
||||
image_size: std::fs::metadata(path).map(|m| m.len()).unwrap_or(0),
|
||||
volume_id: iso.volume_id.clone(),
|
||||
serial,
|
||||
version: cnf_value(&cnf, "VER"),
|
||||
vmode: cnf_value(&cnf, "VMODE"),
|
||||
boot_elf_size: entry.size,
|
||||
boot_elf_sha1: sha1_hex(&elf),
|
||||
})
|
||||
}
|
||||
|
||||
/// SHA-1 of `data`, lowercase hex.
|
||||
///
|
||||
/// Implemented here rather than pulled in as a dependency so that `identify`
|
||||
/// stays usable as a standalone stage with a minimal dependency set.
|
||||
pub fn sha1_hex(data: &[u8]) -> String {
|
||||
let mut h = Sha1::new();
|
||||
h.update(data);
|
||||
h.hex()
|
||||
}
|
||||
|
||||
/// Minimal SHA-1 (FIPS 180-4).
|
||||
pub struct Sha1 {
|
||||
state: [u32; 5],
|
||||
buffer: [u8; 64],
|
||||
buffered: usize,
|
||||
length: u64,
|
||||
}
|
||||
|
||||
impl Default for Sha1 {
|
||||
fn default() -> Self {
|
||||
Self::new()
|
||||
}
|
||||
}
|
||||
|
||||
impl Sha1 {
|
||||
pub fn new() -> Self {
|
||||
Self {
|
||||
state: [
|
||||
0x6745_2301,
|
||||
0xEFCD_AB89,
|
||||
0x98BA_DCFE,
|
||||
0x1032_5476,
|
||||
0xC3D2_E1F0,
|
||||
],
|
||||
buffer: [0u8; 64],
|
||||
buffered: 0,
|
||||
length: 0,
|
||||
}
|
||||
}
|
||||
|
||||
pub fn update(&mut self, mut data: &[u8]) {
|
||||
self.length = self.length.wrapping_add(data.len() as u64);
|
||||
if self.buffered > 0 {
|
||||
let take = (64 - self.buffered).min(data.len());
|
||||
self.buffer[self.buffered..self.buffered + take].copy_from_slice(&data[..take]);
|
||||
self.buffered += take;
|
||||
data = &data[take..];
|
||||
if self.buffered == 64 {
|
||||
let block = self.buffer;
|
||||
self.compress(&block);
|
||||
self.buffered = 0;
|
||||
}
|
||||
}
|
||||
while data.len() >= 64 {
|
||||
let mut block = [0u8; 64];
|
||||
block.copy_from_slice(&data[..64]);
|
||||
self.compress(&block);
|
||||
data = &data[64..];
|
||||
}
|
||||
if !data.is_empty() {
|
||||
self.buffer[..data.len()].copy_from_slice(data);
|
||||
self.buffered = data.len();
|
||||
}
|
||||
}
|
||||
|
||||
fn compress(&mut self, block: &[u8; 64]) {
|
||||
let mut w = [0u32; 80];
|
||||
for i in 0..16 {
|
||||
w[i] = u32::from_be_bytes([
|
||||
block[i * 4],
|
||||
block[i * 4 + 1],
|
||||
block[i * 4 + 2],
|
||||
block[i * 4 + 3],
|
||||
]);
|
||||
}
|
||||
for i in 16..80 {
|
||||
w[i] = (w[i - 3] ^ w[i - 8] ^ w[i - 14] ^ w[i - 16]).rotate_left(1);
|
||||
}
|
||||
let (mut a, mut b, mut c, mut d, mut e) = (
|
||||
self.state[0],
|
||||
self.state[1],
|
||||
self.state[2],
|
||||
self.state[3],
|
||||
self.state[4],
|
||||
);
|
||||
for (i, &wi) in w.iter().enumerate() {
|
||||
let (f, k) = match i {
|
||||
0..=19 => ((b & c) | ((!b) & d), 0x5A82_7999u32),
|
||||
20..=39 => (b ^ c ^ d, 0x6ED9_EBA1),
|
||||
40..=59 => ((b & c) | (b & d) | (c & d), 0x8F1B_BCDC),
|
||||
_ => (b ^ c ^ d, 0xCA62_C1D6),
|
||||
};
|
||||
let temp = a
|
||||
.rotate_left(5)
|
||||
.wrapping_add(f)
|
||||
.wrapping_add(e)
|
||||
.wrapping_add(k)
|
||||
.wrapping_add(wi);
|
||||
e = d;
|
||||
d = c;
|
||||
c = b.rotate_left(30);
|
||||
b = a;
|
||||
a = temp;
|
||||
}
|
||||
self.state[0] = self.state[0].wrapping_add(a);
|
||||
self.state[1] = self.state[1].wrapping_add(b);
|
||||
self.state[2] = self.state[2].wrapping_add(c);
|
||||
self.state[3] = self.state[3].wrapping_add(d);
|
||||
self.state[4] = self.state[4].wrapping_add(e);
|
||||
}
|
||||
|
||||
pub fn hex(mut self) -> String {
|
||||
let bit_len = self.length.wrapping_mul(8);
|
||||
self.update_raw(&[0x80]);
|
||||
while self.buffered != 56 {
|
||||
self.update_raw(&[0x00]);
|
||||
}
|
||||
let len_bytes = bit_len.to_be_bytes();
|
||||
self.update_raw(&len_bytes);
|
||||
let mut out = String::with_capacity(40);
|
||||
for word in self.state {
|
||||
out.push_str(&format!("{word:08x}"));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Feed padding without changing the recorded message length.
|
||||
fn update_raw(&mut self, data: &[u8]) {
|
||||
for &byte in data {
|
||||
self.buffer[self.buffered] = byte;
|
||||
self.buffered += 1;
|
||||
if self.buffered == 64 {
|
||||
let block = self.buffer;
|
||||
self.compress(&block);
|
||||
self.buffered = 0;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn sha1_matches_known_vectors() {
|
||||
assert_eq!(sha1_hex(b""), "da39a3ee5e6b4b0d3255bfef95601890afd80709");
|
||||
assert_eq!(sha1_hex(b"abc"), "a9993e364706816aba3e25717850c26c9cd0d89d");
|
||||
assert_eq!(
|
||||
sha1_hex(b"abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"),
|
||||
"84983e441c3bd26ebaae4aa1f95129e5e54670f1"
|
||||
);
|
||||
// Exactly one block.
|
||||
assert_eq!(
|
||||
sha1_hex(&[b'a'; 64]),
|
||||
"0098ba824b5c16427bd7a1122a5a442a25ec644d"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sha1_handles_multi_block_input() {
|
||||
let million: Vec<u8> = std::iter::repeat_n(b'a', 1_000_000).collect();
|
||||
assert_eq!(
|
||||
sha1_hex(&million),
|
||||
"34aa973cd4c4daa4f61eeb2bdbad27316534016f"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn cnf_parsing() {
|
||||
let cnf = "BOOT2 = cdrom0:\\SCUS_972.68;1\r\nVER = 1.01\r\nVMODE = NTSC\r\n\r\n";
|
||||
assert_eq!(cnf_value(cnf, "BOOT2"), "cdrom0:\\SCUS_972.68");
|
||||
assert_eq!(cnf_value(cnf, "VER"), "1.01");
|
||||
assert_eq!(cnf_value(cnf, "VMODE"), "NTSC");
|
||||
assert_eq!(cnf_value(cnf, "MISSING"), "");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ntsc_verdict_mentions_sixty_hertz() {
|
||||
let d = DiscInfo {
|
||||
path: "x.iso".into(),
|
||||
image_size: 0,
|
||||
volume_id: String::new(),
|
||||
serial: TARGET_SERIAL.into(),
|
||||
version: TARGET_VERSION.into(),
|
||||
vmode: "NTSC".into(),
|
||||
boot_elf_size: 0,
|
||||
boot_elf_sha1: TARGET_BOOT_ELF_SHA1.into(),
|
||||
};
|
||||
assert!(d.verdict().contains("60 Hz"), "{}", d.verdict());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn target_verdict_for_the_supported_build() {
|
||||
let d = DiscInfo {
|
||||
path: "x.iso".into(),
|
||||
image_size: 0,
|
||||
volume_id: "RATCHETANDCLANK2".into(),
|
||||
serial: TARGET_SERIAL.into(),
|
||||
version: TARGET_VERSION.into(),
|
||||
vmode: "NTSC".into(),
|
||||
boot_elf_size: 0,
|
||||
boot_elf_sha1: TARGET_BOOT_ELF_SHA1.into(),
|
||||
};
|
||||
assert!(d.is_target());
|
||||
assert!(!d.is_pal());
|
||||
assert!(d.verdict().contains("the build we target"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn greatest_hits_is_called_out_as_a_different_build() {
|
||||
let d = DiscInfo {
|
||||
path: "x.iso".into(),
|
||||
image_size: 0,
|
||||
volume_id: String::new(),
|
||||
serial: "SCUS_972.68GH".into(),
|
||||
version: "2.00".into(),
|
||||
vmode: "NTSC".into(),
|
||||
boot_elf_size: 0,
|
||||
boot_elf_sha1: String::new(),
|
||||
};
|
||||
assert!(!d.is_target());
|
||||
assert!(d.verdict().contains("different build"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn pal_is_flagged_as_fifty_hertz() {
|
||||
let d = DiscInfo {
|
||||
path: "x.iso".into(),
|
||||
image_size: 0,
|
||||
volume_id: String::new(),
|
||||
serial: "SCES-516.07".into(),
|
||||
version: "1.00".into(),
|
||||
vmode: "PAL".into(),
|
||||
boot_elf_size: 0,
|
||||
boot_elf_sha1: String::new(),
|
||||
};
|
||||
assert!(d.is_pal());
|
||||
assert!(d.verdict().contains("50 Hz"), "{}", d.verdict());
|
||||
}
|
||||
}
|
||||
@@ -1,29 +1,23 @@
|
||||
//! Disc pipeline for *Ratchet & Clank: Going Commando* (PS2).
|
||||
//! The disc pipeline for *Ratchet & Clank: Going Commando*.
|
||||
//!
|
||||
//! Turns a legally obtained disc image into verified, unpacked asset files that
|
||||
//! the engine can read directly, and does nothing else.
|
||||
//! Three stages, in order:
|
||||
//!
|
||||
//! # Stages
|
||||
//!
|
||||
//! 1. **identify** - read `SYSTEM.CNF`, confirm serial/version/video mode and
|
||||
//! 1. `identify` — read `SYSTEM.CNF`, confirm serial, version and video mode,
|
||||
//! hash the boot ELF. Reports clearly when a disc is not the supported
|
||||
//! build. A Python reference implementation of this stage lives in
|
||||
//! `tools/identify-disc.py`.
|
||||
//! 2. **unpack** - read `RC2.HDR`, walk the table of contents, decompress WAD
|
||||
//! streams and write per-level and global asset directories.
|
||||
//! 3. **verify** - check every extracted file against a recorded size and
|
||||
//! SHA-1 table, so a corrupted or mismatched dump cannot quietly produce a
|
||||
//! subtly wrong game.
|
||||
//! build.
|
||||
//! 2. `unpack` — enumerate the ISO 9660 filesystem, decompress WAD streams and
|
||||
//! write asset files.
|
||||
//! 3. `verify` — check extracted files against a committed size and SHA-1
|
||||
//! table, so a wrong or damaged dump cannot quietly produce a subtly wrong
|
||||
//! game.
|
||||
//!
|
||||
//! # Output layout
|
||||
//!
|
||||
//! Extracted data is written outside the repository (see `.gitignore`). The
|
||||
//! pre-commit hook in `.githooks/` refuses disc images, asset directories and
|
||||
//! oversized files, so nothing here can end up in git history by accident.
|
||||
//!
|
||||
//! # Verification is not optional
|
||||
//!
|
||||
//! Every parser downstream is only as trustworthy as the bytes it was handed.
|
||||
//! A hash table committed to the repo (sizes and hashes only, never content)
|
||||
//! is what lets a test say "this parse matches the retail disc" rather than
|
||||
//! "this parse matches whatever happened to be on my disk".
|
||||
//! Everything here reads the user's own legally obtained disc and writes
|
||||
//! outside the repository. See `AGENTS.md`.
|
||||
|
||||
pub mod identify;
|
||||
pub mod unpack;
|
||||
pub mod verify;
|
||||
|
||||
pub use identify::{identify, DiscInfo};
|
||||
pub use unpack::unpack;
|
||||
pub use verify::{verify, VerifyReport};
|
||||
|
||||
@@ -0,0 +1,215 @@
|
||||
//! `gc-extract` — the disc pipeline command line.
|
||||
//!
|
||||
//! Three subcommands, in the order they are normally used:
|
||||
//!
|
||||
//! ```text
|
||||
//! gc-extract identify <disc.iso>
|
||||
//! gc-extract unpack <disc.iso> <out-dir> [--table <table.tsv>]
|
||||
//! gc-extract verify <out-dir> <table.tsv>
|
||||
//! ```
|
||||
//!
|
||||
//! Only your own legally obtained disc is read. Nothing is written inside the
|
||||
//! repository.
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::ExitCode;
|
||||
|
||||
use gc_extract::{identify, unpack, verify};
|
||||
|
||||
fn main() -> ExitCode {
|
||||
let args: Vec<String> = std::env::args().skip(1).collect();
|
||||
match args.first().map(String::as_str) {
|
||||
Some("identify") => run_identify(&args[1..]),
|
||||
Some("unpack") => run_unpack(&args[1..]),
|
||||
Some("verify") => run_verify(&args[1..]),
|
||||
Some("--help") | Some("-h") | Some("help") | None => {
|
||||
usage();
|
||||
ExitCode::SUCCESS
|
||||
}
|
||||
Some(other) => {
|
||||
eprintln!("unknown subcommand `{other}`\n");
|
||||
usage();
|
||||
ExitCode::FAILURE
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Group digits for readability. Rust's format syntax has no thousands
|
||||
/// separator, so do it here rather than sprinkle the logic through the prints.
|
||||
fn thousands(n: u64) -> String {
|
||||
let s = n.to_string();
|
||||
let mut out = String::with_capacity(s.len() + s.len() / 3);
|
||||
for (i, c) in s.chars().enumerate() {
|
||||
if i > 0 && (s.len() - i) % 3 == 0 {
|
||||
out.push('_');
|
||||
}
|
||||
out.push(c);
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn usage() {
|
||||
println!(
|
||||
"gc-extract — Ratchet & Clank: Going Commando disc pipeline
|
||||
|
||||
identify <disc.iso>
|
||||
Report serial, version and video mode, and hash the boot ELF.
|
||||
|
||||
unpack <disc.iso> <out-dir> [--table <file.tsv>]
|
||||
Decompress every WAD on the disc into <out-dir>.
|
||||
With --table, also write a SHA-1 table (sizes and hashes only).
|
||||
|
||||
verify <out-dir> <table.tsv>
|
||||
Check <out-dir> against a previously written table.
|
||||
|
||||
You supply the disc. Nothing here is redistributed with the project."
|
||||
);
|
||||
}
|
||||
|
||||
fn run_identify(args: &[String]) -> ExitCode {
|
||||
if args.is_empty() {
|
||||
eprintln!("usage: gc-extract identify <disc.iso>");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
let path = Path::new(&args[0]);
|
||||
match identify(path) {
|
||||
Ok(info) => {
|
||||
println!("Disc identification");
|
||||
println!("{}", "=".repeat(64));
|
||||
println!(" file {}", info.path);
|
||||
println!(" size {} bytes", thousands(info.image_size));
|
||||
println!(" volume id {}", info.volume_id);
|
||||
println!(" serial {}", info.serial);
|
||||
println!(
|
||||
" version {}",
|
||||
if info.version.is_empty() {
|
||||
"(absent)"
|
||||
} else {
|
||||
&info.version
|
||||
}
|
||||
);
|
||||
println!(
|
||||
" video mode {}",
|
||||
if info.vmode.is_empty() {
|
||||
"(absent)"
|
||||
} else {
|
||||
&info.vmode
|
||||
}
|
||||
);
|
||||
println!();
|
||||
println!("Boot executable");
|
||||
println!("{}", "-".repeat(64));
|
||||
println!(
|
||||
" size {} bytes",
|
||||
thousands(info.boot_elf_size as u64)
|
||||
);
|
||||
println!(" SHA-1 {}", info.boot_elf_sha1);
|
||||
println!();
|
||||
println!("{}", "=".repeat(64));
|
||||
println!(" {}", info.verdict());
|
||||
ExitCode::SUCCESS
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("error: {e}");
|
||||
ExitCode::FAILURE
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn run_unpack(args: &[String]) -> ExitCode {
|
||||
if args.len() < 2 {
|
||||
eprintln!("usage: gc-extract unpack <disc.iso> <out-dir> [--table <file.tsv>]");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
let iso = PathBuf::from(&args[0]);
|
||||
let out = PathBuf::from(&args[1]);
|
||||
let mut table_path: Option<PathBuf> = None;
|
||||
let mut i = 2;
|
||||
while i < args.len() {
|
||||
match args[i].as_str() {
|
||||
"--table" => {
|
||||
i += 1;
|
||||
match args.get(i) {
|
||||
Some(p) => table_path = Some(PathBuf::from(p)),
|
||||
None => {
|
||||
eprintln!("--table needs a path");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
}
|
||||
}
|
||||
other => {
|
||||
eprintln!("unexpected argument `{other}`");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
}
|
||||
i += 1;
|
||||
}
|
||||
|
||||
let ident = match identify::identify(&iso) {
|
||||
Ok(i) => i,
|
||||
Err(e) => {
|
||||
eprintln!("error: {e}");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
};
|
||||
println!("Disc: {} {} ({})", ident.serial, ident.version, ident.vmode);
|
||||
|
||||
let started = std::time::Instant::now();
|
||||
println!("Unpacking into {} ...", out.display());
|
||||
match unpack::unpack(&iso, &out) {
|
||||
Ok(files) => {
|
||||
let total: u64 = files.iter().map(|f| f.size).sum();
|
||||
println!(
|
||||
" {} files, {} bytes, in {:.1}s",
|
||||
files.len(),
|
||||
thousands(total),
|
||||
started.elapsed().as_secs_f64()
|
||||
);
|
||||
|
||||
if let Some(tp) = table_path {
|
||||
let entries: Vec<(String, u64, String)> = files
|
||||
.iter()
|
||||
.map(|f| (f.path.clone(), f.size, f.sha1.clone()))
|
||||
.collect();
|
||||
let text = verify::render_table(&entries);
|
||||
if let Err(e) = std::fs::write(&tp, text) {
|
||||
eprintln!("error: cannot write {}: {e}", tp.display());
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
println!(" table written to {}", tp.display());
|
||||
}
|
||||
ExitCode::SUCCESS
|
||||
}
|
||||
Err(e) => {
|
||||
eprintln!("error: {e}");
|
||||
ExitCode::FAILURE
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn run_verify(args: &[String]) -> ExitCode {
|
||||
if args.len() < 2 {
|
||||
eprintln!("usage: gc-extract verify <out-dir> <table.tsv>");
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
let dir = PathBuf::from(&args[0]);
|
||||
let table = match std::fs::read_to_string(&args[1]) {
|
||||
Ok(t) => t,
|
||||
Err(e) => {
|
||||
eprintln!("error: cannot read {}: {e}", args[1]);
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
};
|
||||
let expected = verify::parse_table(&table);
|
||||
if expected.is_empty() {
|
||||
eprintln!("error: {} contained no usable rows", args[1]);
|
||||
return ExitCode::FAILURE;
|
||||
}
|
||||
let report = verify::verify(&dir, &expected);
|
||||
println!("{}", report.summary());
|
||||
if report.ok() {
|
||||
ExitCode::SUCCESS
|
||||
} else {
|
||||
ExitCode::FAILURE
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,222 @@
|
||||
//! Stage 2: unpack the disc into asset files.
|
||||
//!
|
||||
//! R&C2 lists its assets in the ISO 9660 filesystem, unlike R&C1, so
|
||||
//! enumeration is straightforward: every entry under `G/` is a container whose
|
||||
//! header holds a table of sector ranges, and each range is written out as a
|
||||
//! file.
|
||||
//!
|
||||
//! Output goes outside the repository. `AGENTS.md` explains why, and the
|
||||
//! pre-commit hook enforces it.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
use gc_formats::global_wad::{GlobalWad, RangeKind};
|
||||
use gc_formats::iso9660::IsoImage;
|
||||
|
||||
/// One unpacked asset, with the identity we record for verification.
|
||||
pub struct Extracted {
|
||||
/// Path relative to the output directory, e.g. `armor/0000.bin`.
|
||||
pub path: String,
|
||||
pub size: u64,
|
||||
pub sha1: String,
|
||||
}
|
||||
|
||||
/// Unpack `iso_path` into `out_dir`.
|
||||
///
|
||||
/// Returns every file written, sorted by path, so the caller can build a hash
|
||||
/// table or verify against one.
|
||||
pub fn unpack(iso_path: &Path, out_dir: &Path) -> Result<Vec<Extracted>, String> {
|
||||
let iso = IsoImage::open(iso_path).map_err(|e| format!("cannot read the disc: {e}"))?;
|
||||
std::fs::create_dir_all(out_dir)
|
||||
.map_err(|e| format!("cannot create {}: {e}", out_dir.display()))?;
|
||||
|
||||
let mut written: BTreeMap<String, Extracted> = BTreeMap::new();
|
||||
let mut skipped: Vec<(String, String)> = Vec::new();
|
||||
|
||||
for entry in &iso.files {
|
||||
if entry.is_dir {
|
||||
continue;
|
||||
}
|
||||
// `Z6TAIL.DUP` is zero-length padding the ISO 9660 filesystem needs.
|
||||
if entry.name.starts_with("Z6TAIL") {
|
||||
continue;
|
||||
}
|
||||
|
||||
let relative = entry.name.trim_end_matches('/').to_string();
|
||||
|
||||
if !relative.starts_with("G/") {
|
||||
// Not a container: the boot ELF, SYSTEM.CNF, RC2.HDR and the IOP
|
||||
// image are copied through verbatim.
|
||||
let data = gc_formats::iso9660::read_at(iso_path, entry.lba, entry.size)
|
||||
.map_err(|e| format!("cannot read {}: {e}", entry.name))?;
|
||||
write_file(out_dir, &relative, &data, &mut written);
|
||||
continue;
|
||||
}
|
||||
|
||||
// A container: write out each range its header declares.
|
||||
let raw = gc_formats::iso9660::read_at(iso_path, entry.lba, entry.size)
|
||||
.map_err(|e| format!("cannot read {}: {e}", entry.name))?;
|
||||
|
||||
let stem = relative
|
||||
.trim_start_matches("G/")
|
||||
.trim_end_matches(".WAD")
|
||||
.to_lowercase();
|
||||
|
||||
// MPEG1 and MPEG2 are raw concatenated video, not indexed containers:
|
||||
// their first word is video payload, not a header size. Copy them
|
||||
// through; the video phase splits them properly.
|
||||
if stem.starts_with("mpeg1") || stem.starts_with("mpeg2") {
|
||||
write_file(out_dir, &format!("{stem}.bin"), &raw, &mut written);
|
||||
println!(
|
||||
" {:<16} {} bytes, raw video stream (not indexed)",
|
||||
stem,
|
||||
raw.len()
|
||||
);
|
||||
continue;
|
||||
}
|
||||
|
||||
let container = match GlobalWad::parse_with(&raw, RangeKind::for_container(&relative)) {
|
||||
Ok(c) => c,
|
||||
Err(e) => {
|
||||
// Some containers have a header this generic parser does not
|
||||
// model (the scene containers nest further). Copy the file
|
||||
// through untouched rather than failing the whole run: the
|
||||
// bytes are preserved and a typed parser can be added later.
|
||||
skipped.push((relative.clone(), e.to_string()));
|
||||
write_file(out_dir, &relative, &raw, &mut written);
|
||||
continue;
|
||||
}
|
||||
};
|
||||
let present = container.present().count();
|
||||
let mut out_of_range = 0usize;
|
||||
|
||||
for (index, offset, byte_len) in container.present() {
|
||||
let len = byte_len as usize;
|
||||
|
||||
// Bounds-check against the container we actually read, so a corrupt
|
||||
// header cannot make us read past the end of the file.
|
||||
if offset + len as u64 > raw.len() as u64 {
|
||||
out_of_range += 1;
|
||||
continue;
|
||||
}
|
||||
let data = &raw[offset as usize..offset as usize + len];
|
||||
write_file(
|
||||
out_dir,
|
||||
&format!("{stem}/{index:04}.bin"),
|
||||
data,
|
||||
&mut written,
|
||||
);
|
||||
}
|
||||
if out_of_range > 0 {
|
||||
eprintln!(
|
||||
" {:<16} {} bytes, {} entries -> {} present, {out_of_range} out of range (header describes \
|
||||
sectors beyond the file; see docs/plan/disc-pipeline.md)",
|
||||
stem, raw.len(), container.entries.len(), present
|
||||
);
|
||||
} else {
|
||||
println!(
|
||||
" {:<16} {} bytes, {} entries -> {} present",
|
||||
stem,
|
||||
raw.len(),
|
||||
container.entries.len(),
|
||||
present
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
for (name, why) in &skipped {
|
||||
eprintln!(" {name}: not an indexed container ({why}); copied verbatim");
|
||||
}
|
||||
|
||||
Ok(written.into_values().collect())
|
||||
}
|
||||
|
||||
fn write_file(
|
||||
out_dir: &Path,
|
||||
relative: &str,
|
||||
data: &[u8],
|
||||
written: &mut BTreeMap<String, Extracted>,
|
||||
) {
|
||||
use crate::identify::sha1_hex;
|
||||
|
||||
let dest = out_dir.join(relative);
|
||||
if let Some(parent) = dest.parent() {
|
||||
if let Err(e) = std::fs::create_dir_all(parent) {
|
||||
eprintln!("warning: cannot create {}: {e}", parent.display());
|
||||
return;
|
||||
}
|
||||
}
|
||||
if let Err(e) = std::fs::write(&dest, data) {
|
||||
eprintln!("warning: cannot write {}: {e}", dest.display());
|
||||
return;
|
||||
}
|
||||
written.insert(
|
||||
relative.to_string(),
|
||||
Extracted {
|
||||
path: relative.to_string(),
|
||||
size: data.len() as u64,
|
||||
sha1: sha1_hex(data),
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
/// Default output directory: `extracted/` beside the repository root.
|
||||
pub fn default_out_dir() -> PathBuf {
|
||||
PathBuf::from("extracted")
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
/// A container with three entries, the second of which is absent.
|
||||
fn container_bytes() -> Vec<u8> {
|
||||
let header_size = 8 + 3 * 8;
|
||||
let mut v = Vec::new();
|
||||
v.extend_from_slice(&(header_size as u32).to_le_bytes());
|
||||
v.extend_from_slice(&10u32.to_le_bytes());
|
||||
v.extend_from_slice(&1i32.to_le_bytes());
|
||||
v.extend_from_slice(&1i32.to_le_bytes());
|
||||
v.extend_from_slice(&2i32.to_le_bytes());
|
||||
v.extend_from_slice(&0i32.to_le_bytes()); // absent
|
||||
v.extend_from_slice(&2i32.to_le_bytes());
|
||||
v.extend_from_slice(&1i32.to_le_bytes());
|
||||
v.resize(header_size + 3 * 2048, 0xAB);
|
||||
v
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn parses_a_container_and_lists_present_entries() {
|
||||
let raw = container_bytes();
|
||||
let c = GlobalWad::parse(&raw).unwrap();
|
||||
assert_eq!(c.entries.len(), 3);
|
||||
let present: Vec<usize> = c.present().map(|(i, _, _)| i).collect();
|
||||
assert_eq!(present, vec![0, 2]);
|
||||
// The default reading treats the second field as a byte count, so an
|
||||
// entry of size 1 is one byte.
|
||||
assert_eq!(c.present().next().unwrap().2, 1);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_declared_range_is_inside_the_container() {
|
||||
let raw = container_bytes();
|
||||
let c = GlobalWad::parse(&raw).unwrap();
|
||||
for (_, offset, len) in c.present() {
|
||||
assert!(
|
||||
offset + len <= raw.len() as u64,
|
||||
"range at {offset} runs past the end"
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn a_range_past_the_end_would_be_caught() {
|
||||
let mut raw = container_bytes();
|
||||
// Push the first entry's offset far beyond the file.
|
||||
raw[8..12].copy_from_slice(&9999i32.to_le_bytes());
|
||||
let c = GlobalWad::parse(&raw).unwrap();
|
||||
let (_, offset, len) = c.present().next().unwrap();
|
||||
assert!(offset + len > raw.len() as u64);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,266 @@
|
||||
//! Stage 3: verify extracted files against a committed hash table.
|
||||
//!
|
||||
//! The table records sizes and SHA-1 values only — never content. Its purpose
|
||||
//! is to let a test assert "this parse matches retail" rather than "this parse
|
||||
//! matches whatever happened to be on my disk". See `AGENTS.md`.
|
||||
|
||||
use std::collections::BTreeMap;
|
||||
use std::path::Path;
|
||||
|
||||
/// One expected file: path, size in bytes, SHA-1.
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Expected {
|
||||
pub path: String,
|
||||
pub size: u64,
|
||||
pub sha1: String,
|
||||
}
|
||||
|
||||
/// Outcome of verifying a directory.
|
||||
#[derive(Debug, Default)]
|
||||
pub struct VerifyReport {
|
||||
pub checked: usize,
|
||||
pub missing: Vec<String>,
|
||||
pub size_mismatch: Vec<(String, u64, u64)>,
|
||||
pub hash_mismatch: Vec<(String, String, String)>,
|
||||
/// Files present in the directory but absent from the table.
|
||||
pub unexpected: Vec<String>,
|
||||
}
|
||||
|
||||
impl VerifyReport {
|
||||
pub fn ok(&self) -> bool {
|
||||
self.missing.is_empty()
|
||||
&& self.size_mismatch.is_empty()
|
||||
&& self.hash_mismatch.is_empty()
|
||||
&& self.unexpected.is_empty()
|
||||
}
|
||||
|
||||
/// A short human-readable summary.
|
||||
pub fn summary(&self) -> String {
|
||||
if self.ok() {
|
||||
return format!("OK: {} files match the recorded table.", self.checked);
|
||||
}
|
||||
let mut lines = vec![format!(
|
||||
"FAILED: {} files checked, {} problems",
|
||||
self.checked,
|
||||
self.missing.len()
|
||||
+ self.size_mismatch.len()
|
||||
+ self.hash_mismatch.len()
|
||||
+ self.unexpected.len()
|
||||
)];
|
||||
for p in self.missing.iter().take(10) {
|
||||
lines.push(format!(" missing: {p}"));
|
||||
}
|
||||
for (p, want, got) in self.size_mismatch.iter().take(10) {
|
||||
lines.push(format!(" size: {p}: expected {want}, got {got}"));
|
||||
}
|
||||
for (p, want, got) in self.hash_mismatch.iter().take(10) {
|
||||
lines.push(format!(" hash: {p}: expected {want}, got {got}"));
|
||||
}
|
||||
for p in self.unexpected.iter().take(10) {
|
||||
lines.push(format!(" unexpected: {p}"));
|
||||
}
|
||||
lines.join("\n")
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse a table from TSV lines of `sha1<TAB>size<TAB>path`.
|
||||
pub fn parse_table(text: &str) -> Vec<Expected> {
|
||||
text.lines()
|
||||
.filter(|l| !l.trim().is_empty() && !l.starts_with('#'))
|
||||
.filter_map(|line| {
|
||||
let mut parts = line.split('\t');
|
||||
let sha1 = parts.next()?.trim().to_string();
|
||||
let size: u64 = parts.next()?.trim().parse().ok()?;
|
||||
let path = parts.next()?.trim().to_string();
|
||||
if sha1.len() != 40 || path.is_empty() {
|
||||
return None;
|
||||
}
|
||||
Some(Expected { path, size, sha1 })
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Render entries as a TSV table, sorted by path.
|
||||
pub fn render_table(entries: &[(String, u64, String)]) -> String {
|
||||
let mut sorted: Vec<_> = entries.to_vec();
|
||||
sorted.sort_by(|a, b| a.0.cmp(&b.0));
|
||||
let mut out = String::new();
|
||||
for (path, size, sha1) in sorted {
|
||||
out.push_str(&format!("{sha1}\t{size}\t{path}\n"));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
/// Verify `dir` against `expected`.
|
||||
pub fn verify(dir: &Path, expected: &[Expected]) -> VerifyReport {
|
||||
let mut report = VerifyReport::default();
|
||||
|
||||
let present = walk(dir);
|
||||
|
||||
for e in expected {
|
||||
report.checked += 1;
|
||||
match std::fs::read(dir.join(&e.path)) {
|
||||
Ok(data) => {
|
||||
if data.len() as u64 != e.size {
|
||||
report
|
||||
.size_mismatch
|
||||
.push((e.path.clone(), e.size, data.len() as u64));
|
||||
continue;
|
||||
}
|
||||
let got = crate::identify::sha1_hex(&data);
|
||||
if got != e.sha1 {
|
||||
report
|
||||
.hash_mismatch
|
||||
.push((e.path.clone(), e.sha1.clone(), got));
|
||||
}
|
||||
}
|
||||
Err(_) => report.missing.push(e.path.clone()),
|
||||
}
|
||||
}
|
||||
|
||||
let expected_set: BTreeMap<&str, ()> = expected.iter().map(|e| (e.path.as_str(), ())).collect();
|
||||
for p in present {
|
||||
if !expected_set.contains_key(p.as_str()) {
|
||||
report.unexpected.push(p);
|
||||
}
|
||||
}
|
||||
report
|
||||
}
|
||||
|
||||
/// List files under `dir`, as `/`-separated relative paths, sorted.
|
||||
fn walk(dir: &Path) -> Vec<String> {
|
||||
let mut out = Vec::new();
|
||||
collect(dir, dir, &mut out);
|
||||
out.sort();
|
||||
out
|
||||
}
|
||||
|
||||
fn collect(root: &Path, dir: &Path, out: &mut Vec<String>) {
|
||||
let Ok(entries) = std::fs::read_dir(dir) else {
|
||||
return;
|
||||
};
|
||||
for entry in entries.flatten() {
|
||||
let path = entry.path();
|
||||
if path.is_dir() {
|
||||
collect(root, &path, out);
|
||||
} else if let Ok(rel) = path.strip_prefix(root) {
|
||||
out.push(rel.to_string_lossy().replace('\\', "/"));
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn parses_a_table_and_skips_comments() {
|
||||
let text = concat!(
|
||||
"# a comment\n",
|
||||
"da39a3ee5e6b4b0d3255bfef95601890afd80709\t0\tempty.bin\n",
|
||||
"a9993e364706816aba3e25717850c26c9cd0d89d\t3\tabc.bin\n",
|
||||
"\n",
|
||||
);
|
||||
let t = parse_table(text);
|
||||
assert_eq!(t.len(), 2);
|
||||
assert_eq!(t[0].path, "empty.bin");
|
||||
assert_eq!(t[0].size, 0);
|
||||
assert_eq!(t[1].path, "abc.bin");
|
||||
assert_eq!(t[1].size, 3);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ignores_malformed_rows() {
|
||||
let text = "notasha1\t3\tx.bin\n\
|
||||
a9993e364706816aba3e25717850c26c9cd0d89d\tnope\tx.bin\n\
|
||||
a9993e364706816aba3e25717850c26c9cd0d89d\t3\t";
|
||||
assert!(parse_table(text).is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn round_trips_a_rendered_table() {
|
||||
let entries = vec![
|
||||
(
|
||||
"b.bin".to_string(),
|
||||
2u64,
|
||||
"a9993e364706816aba3e25717850c26c9cd0d89d".to_string(),
|
||||
),
|
||||
(
|
||||
"a.bin".to_string(),
|
||||
1u64,
|
||||
"da39a3ee5e6b4b0d3255bfef95601890afd80709".to_string(),
|
||||
),
|
||||
];
|
||||
let text = render_table(&entries);
|
||||
assert!(
|
||||
text.lines().next().unwrap().starts_with("da39a3ee"),
|
||||
"sorted by path"
|
||||
);
|
||||
assert_eq!(parse_table(&text).len(), 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn verify_detects_a_missing_file() {
|
||||
let dir = std::env::temp_dir().join(format!("gc-verify-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
|
||||
let expected = vec![Expected {
|
||||
path: "nope.bin".to_string(),
|
||||
size: 3,
|
||||
sha1: "a9993e364706816aba3e25717850c26c9cd0d89d".to_string(),
|
||||
}];
|
||||
let report = verify(&dir, &expected);
|
||||
assert!(!report.ok());
|
||||
assert_eq!(report.missing, vec!["nope.bin".to_string()]);
|
||||
assert!(report.summary().contains("missing"));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn verify_detects_a_hash_mismatch() {
|
||||
let dir = std::env::temp_dir().join(format!("gc-verify-hash-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
std::fs::write(dir.join("a.bin"), b"xyz").unwrap();
|
||||
|
||||
let expected = vec![Expected {
|
||||
path: "a.bin".to_string(),
|
||||
size: 3,
|
||||
// SHA-1 of "abc", not "xyz".
|
||||
sha1: "a9993e364706816aba3e25717850c26c9cd0d89d".to_string(),
|
||||
}];
|
||||
let report = verify(&dir, &expected);
|
||||
assert_eq!(report.hash_mismatch.len(), 1);
|
||||
assert!(report.summary().contains("hash"));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn verify_detects_an_unexpected_file() {
|
||||
let dir = std::env::temp_dir().join(format!("gc-verify-extra-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
std::fs::write(dir.join("extra.bin"), b"q").unwrap();
|
||||
let report = verify(&dir, &[]);
|
||||
assert_eq!(report.unexpected, vec!["extra.bin".to_string()]);
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn verify_passes_a_correct_directory() {
|
||||
let dir = std::env::temp_dir().join(format!("gc-verify-ok-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
std::fs::create_dir_all(&dir).unwrap();
|
||||
std::fs::write(dir.join("a.bin"), b"abc").unwrap();
|
||||
|
||||
let expected = vec![Expected {
|
||||
path: "a.bin".to_string(),
|
||||
size: 3,
|
||||
sha1: "a9993e364706816aba3e25717850c26c9cd0d89d".to_string(),
|
||||
}];
|
||||
let report = verify(&dir, &expected);
|
||||
assert!(report.ok(), "{}", report.summary());
|
||||
let _ = std::fs::remove_dir_all(&dir);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,184 @@
|
||||
//! Integration tests against a real disc image.
|
||||
//!
|
||||
//! These are skipped unless `GC_DISC` points at a legally obtained disc, since
|
||||
//! the repository must not contain game data. When it is set they check the
|
||||
//! identification and unpack results against the committed hash table.
|
||||
//!
|
||||
//! ```sh
|
||||
//! GC_DISC="path/to/disc.iso" cargo test -p gc-extract --test disc
|
||||
//! ```
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// The disc under test, if the environment names one.
|
||||
fn disc() -> Option<PathBuf> {
|
||||
let p = std::env::var_os("GC_DISC")?;
|
||||
let p = PathBuf::from(p);
|
||||
p.is_file().then_some(p)
|
||||
}
|
||||
|
||||
/// Skip a test with a clear reason rather than failing.
|
||||
macro_rules! require_disc {
|
||||
() => {
|
||||
match disc() {
|
||||
Some(p) => p,
|
||||
None => {
|
||||
eprintln!("skipped: set GC_DISC to a Going Commando disc image to run this");
|
||||
return;
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn identifies_the_supported_build() {
|
||||
let path = require_disc!();
|
||||
let info = gc_extract::identify::identify(&path).expect("disc should identify");
|
||||
|
||||
assert_eq!(info.serial, "SCUS_972.68");
|
||||
assert_eq!(info.volume_id, "RATCHETANDCLANK2");
|
||||
assert!(
|
||||
info.is_target(),
|
||||
"expected the supported build, got: {}",
|
||||
info.verdict()
|
||||
);
|
||||
assert_eq!(
|
||||
info.boot_elf_sha1,
|
||||
gc_extract::identify::TARGET_BOOT_ELF_SHA1,
|
||||
"boot ELF SHA-1 differs from the recorded retail value"
|
||||
);
|
||||
assert_eq!(info.boot_elf_size, 2_618_684);
|
||||
// This disc is NTSC; the movement constants assume 60 Hz.
|
||||
assert!(!info.is_pal(), "test disc should be NTSC");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn agrees_with_the_python_reference_identifier() {
|
||||
let path = require_disc!();
|
||||
let info = gc_extract::identify::identify(&path).expect("disc should identify");
|
||||
|
||||
// tools/identify-disc.py is the reference implementation and prints these
|
||||
// two values. If they ever disagree, one of them has a bug.
|
||||
// Run from the workspace root so the relative path to the tool resolves.
|
||||
let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR"))
|
||||
.parent()
|
||||
.and_then(|p| p.parent())
|
||||
.expect("the crate should live two levels below the workspace root")
|
||||
.to_path_buf();
|
||||
let py = std::process::Command::new("python3")
|
||||
.arg(root.join("tools/identify-disc.py"))
|
||||
.arg(&path)
|
||||
.current_dir(&root)
|
||||
.output()
|
||||
.expect("python3 should be available to run the reference identifier");
|
||||
assert!(
|
||||
py.status.success(),
|
||||
"the reference identifier failed:\n{}",
|
||||
String::from_utf8_lossy(&py.stderr)
|
||||
);
|
||||
let text = String::from_utf8_lossy(&py.stdout);
|
||||
|
||||
assert!(
|
||||
text.contains(&format!("serial {}", info.serial)),
|
||||
"serial disagrees with the reference identifier:\n{text}"
|
||||
);
|
||||
assert!(
|
||||
text.contains(&format!("SHA-1 {}", info.boot_elf_sha1)),
|
||||
"boot ELF SHA-1 disagrees with the reference identifier:\n{text}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn unpacks_and_matches_the_committed_table() {
|
||||
let path = require_disc!();
|
||||
|
||||
let table_path = Path::new(env!("CARGO_MANIFEST_DIR")).join("data/gc-files.tsv");
|
||||
let table_text =
|
||||
std::fs::read_to_string(&table_path).expect("the hash table should be committed");
|
||||
let expected = gc_extract::verify::parse_table(&table_text);
|
||||
assert!(
|
||||
expected.len() > 5000,
|
||||
"expected a full table, got {} rows",
|
||||
expected.len()
|
||||
);
|
||||
|
||||
// Unpack into a temporary directory so the test leaves nothing behind.
|
||||
let tmp = std::env::temp_dir().join(format!("gc-unpack-{}", std::process::id()));
|
||||
let _ = std::fs::remove_dir_all(&tmp);
|
||||
std::fs::create_dir_all(&tmp).expect("should create a temp directory");
|
||||
|
||||
let files = gc_extract::unpack::unpack(&path, &tmp).expect("unpack should succeed");
|
||||
|
||||
// Every unpacked file must match the recorded size and hash.
|
||||
let report = gc_extract::verify::verify(&tmp, &expected);
|
||||
let _ = std::fs::remove_dir_all(&tmp);
|
||||
|
||||
assert!(
|
||||
report.ok(),
|
||||
"unpacked data does not match the committed table:\n{}",
|
||||
report.summary()
|
||||
);
|
||||
assert_eq!(
|
||||
files.len(),
|
||||
expected.len(),
|
||||
"unpack produced a different file count"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn every_container_parses() {
|
||||
let path = require_disc!();
|
||||
|
||||
let iso = gc_formats::iso9660::IsoImage::open(&path).expect("ISO should open");
|
||||
let containers: Vec<_> = iso
|
||||
.files
|
||||
.iter()
|
||||
.filter(|e| e.name.starts_with("G/") && e.name.ends_with(".WAD"))
|
||||
.collect();
|
||||
|
||||
// 92 `G/*.WAD` containers: 27 LEVEL, 27 AUDIO, 27 SCENE, 11 globals.
|
||||
assert_eq!(containers.len(), 92, "expected 92 asset containers");
|
||||
|
||||
// MPEG1 and MPEG2 are raw video streams with no table; the other 90 must
|
||||
// all parse as indexed containers.
|
||||
let raw_video: Vec<&str> = containers
|
||||
.iter()
|
||||
.map(|c| c.name.as_str())
|
||||
.filter(|n| *n == "G/MPEG1.WAD" || *n == "G/MPEG2.WAD")
|
||||
.collect();
|
||||
assert_eq!(
|
||||
raw_video.len(),
|
||||
2,
|
||||
"expected MPEG1 and MPEG2 to be the raw streams"
|
||||
);
|
||||
|
||||
// The per-level families must all be present, at 27 each.
|
||||
for family in ["LEVEL", "AUDIO", "SCENE"] {
|
||||
let n = containers
|
||||
.iter()
|
||||
.filter(|c| {
|
||||
let stem = c.name.trim_start_matches("G/").trim_end_matches(".WAD");
|
||||
stem.strip_prefix(family)
|
||||
.map(|rest| !rest.is_empty() && rest.chars().all(|ch| ch.is_ascii_digit()))
|
||||
.unwrap_or(false)
|
||||
})
|
||||
.count();
|
||||
assert_eq!(n, 27, "expected 27 {family}<n>.WAD containers, found {n}");
|
||||
}
|
||||
|
||||
for entry in &containers {
|
||||
if raw_video.contains(&entry.name.as_str()) {
|
||||
continue;
|
||||
}
|
||||
let data = gc_formats::iso9660::read_at(&path, entry.lba, entry.size)
|
||||
.expect("read should succeed");
|
||||
let kind = gc_formats::global_wad::RangeKind::for_container(&entry.name);
|
||||
let container = gc_formats::global_wad::GlobalWad::parse_with(&data, kind)
|
||||
.unwrap_or_else(|e| panic!("{} failed to parse: {e}", entry.name));
|
||||
assert!(
|
||||
!container.entries.is_empty(),
|
||||
"{} parsed but had no entries",
|
||||
entry.name
|
||||
);
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user