Phase 1 of the roadmap. The disc is now turned into verified asset files, and
the result is checked against a committed hash table rather than against
whatever happens to be on disk.
gc-extract identify disc.iso SCUS_972.68 v1.01 NTSC, ELF SHA-1 matches
gc-extract unpack disc.iso out/ 5135 files, 4,500,106,960 bytes, 10.8s
gc-extract verify out/ table.tsv OK: 5135 files match
All 92 G/*.WAD containers parse. MPEG1 and MPEG2 are raw video and pass
through whole; the other 90 are indexed containers whose extents are extracted
individually.
Formats recovered and written up in docs/formats/:
- iso9660.rs. R&C2 lists its assets in the filesystem, unlike R&C1, so a
directory walk is the primary asset index. Three details that all fail
quietly rather than loudly: directory records store every 32-bit field twice
(little-endian then big-endian), the CD001 signature is at sector offset 1
rather than 0x8001, and the directory flag at record offset 25 is
authoritative where the identifier's trailing slash is not.
- global_wad.rs. Container headers hold a table of extents. All 92 indexed
containers use {offset in sectors, size in bytes}. Measured, not assumed: the
sector-count reading places every entry of all 27 LEVEL<n> and all 27
AUDIO<n> containers out of range, while the byte reading places all of them
correctly. SCENE<n> is the one exception, a flat sector list whose extents
are sized by the next value.
- wad.rs. Insomniac's LZ77 stream: four flag-selected packet classes, with 0-3
trailing literal bytes per packet carried in unused low bits. This is not a
bit stream, which is why it resists the usual LZ77 reading.
- buf.rs. A bounds-checked reader, so untrusted 2003 disc data produces typed
errors rather than panics.
gc-extract carries the three stages and a SHA-1 implementation with FIPS 180-4
test vectors, so identification works without pulling in a crypto dependency.
Verification: 54 unit tests, plus four integration tests that run only when
GC_DISC names a disc. Those four check identification against the recorded
values, cross-check the Rust identifier against tools/identify-disc.py, parse
all 92 containers, and confirm a full unpack matches the committed table. The
committed table is 316 KB of sizes and SHA-1s for all 5,135 files: hashes only,
no game content, so it is safe to version and it turns "matches retail" into
something a test can assert.
docs/plan/disc-pipeline.md records two wrong turns, because both are easy to
repeat and both failed by producing plausible wrong output rather than an
error: the container table was read backwards, and the LZ77 stream was modelled
as a bit stream. The first survived because it happened to be correct on the
one container used for manual checking.
389 lines
12 KiB
Rust
389 lines
12 KiB
Rust
//! Stage 1: identify the disc.
|
|
//!
|
|
//! Reads only the ISO 9660 structures, `SYSTEM.CNF` and the boot ELF — a few
|
|
//! megabytes of seeks — so it is near-instant on a local image.
|
|
//!
|
|
//! The reference implementation of this stage is `tools/identify-disc.py`; the
|
|
//! two are checked against each other in `tests/disc.rs`.
|
|
|
|
use std::path::Path;
|
|
|
|
use gc_formats::iso9660::IsoImage;
|
|
|
|
/// The build this project targets.
|
|
pub const TARGET_SERIAL: &str = "SCUS_972.68";
|
|
pub const TARGET_VERSION: &str = "1.01";
|
|
/// Measured from a retail Black Label v1.01 image. See `docs/disc-identification.md`.
|
|
pub const TARGET_BOOT_ELF_SHA1: &str = "bacda860cbd6723142e86db56f197e47e728970f";
|
|
|
|
/// What we learned about a disc.
|
|
#[derive(Debug, Clone)]
|
|
pub struct DiscInfo {
|
|
pub path: String,
|
|
pub image_size: u64,
|
|
pub volume_id: String,
|
|
/// Product serial, i.e. the boot executable's file name.
|
|
pub serial: String,
|
|
/// `VER` from `SYSTEM.CNF`.
|
|
pub version: String,
|
|
/// `VMODE` from `SYSTEM.CNF`: `NTSC` or `PAL`, possibly empty.
|
|
pub vmode: String,
|
|
pub boot_elf_size: u32,
|
|
pub boot_elf_sha1: String,
|
|
}
|
|
|
|
impl DiscInfo {
|
|
/// Is this the exact build the reference material describes?
|
|
pub fn is_target(&self) -> bool {
|
|
self.serial == TARGET_SERIAL && self.version == TARGET_VERSION
|
|
}
|
|
|
|
/// True when the video mode makes the game run at 50 Hz.
|
|
///
|
|
/// This matters more than a label difference: a PAL disc runs the game at
|
|
/// 50 Hz, which puts speed-sensitive movement such as the chargeboots at
|
|
/// 5/6 of their intended rate.
|
|
pub fn is_pal(&self) -> bool {
|
|
self.vmode.eq_ignore_ascii_case("PAL")
|
|
}
|
|
|
|
/// A short human-readable summary of whether this disc can be used.
|
|
///
|
|
/// The video mode is reported for every disc, not just the target: a 50 Hz
|
|
/// PAL disc changes movement timing, which is the kind of detail that is
|
|
/// easy to forget and hard to notice later.
|
|
pub fn verdict(&self) -> String {
|
|
let rate = if self.is_pal() {
|
|
" 50 Hz: speed-sensitive movement runs at 5/6 speed, so timing constants differ."
|
|
} else if self.vmode.is_empty() {
|
|
""
|
|
} else {
|
|
" 60 Hz."
|
|
};
|
|
|
|
if self.is_target() && self.boot_elf_sha1 == TARGET_BOOT_ELF_SHA1 {
|
|
return format!(
|
|
"USA Black Label v1.01, NTSC 60 Hz: the build we target. Boot ELF SHA-1 matches.{rate}"
|
|
);
|
|
}
|
|
if self.serial == "SCUS_972.68GH" {
|
|
return format!(
|
|
"Greatest Hits v2.00. This is a different build: the chargeboots were patched, \
|
|
and reference material built for v1.01 will not match.{rate}"
|
|
);
|
|
}
|
|
if self.serial == TARGET_SERIAL {
|
|
return format!(
|
|
"Serial matches but version is {} (we target {TARGET_VERSION}).{rate}",
|
|
if self.version.is_empty() {
|
|
"unknown"
|
|
} else {
|
|
&self.version
|
|
}
|
|
);
|
|
}
|
|
format!("Not the target build. We target {TARGET_SERIAL} v{TARGET_VERSION}.{rate}")
|
|
}
|
|
}
|
|
|
|
/// Extract `KEY = value` from `SYSTEM.CNF`, stopping at `;`, CR or LF.
|
|
fn cnf_value(cnf: &str, key: &str) -> String {
|
|
for line in cnf.lines() {
|
|
if let Some((k, v)) = line.split_once('=') {
|
|
if k.trim() == key {
|
|
return v.trim().split(';').next().unwrap_or("").trim().to_string();
|
|
}
|
|
}
|
|
}
|
|
String::new()
|
|
}
|
|
|
|
/// Identify a disc image, or explain why it cannot be used.
|
|
pub fn identify(path: &Path) -> Result<DiscInfo, String> {
|
|
let iso = IsoImage::open(path).map_err(|e| format!("cannot read {}: {e}", path.display()))?;
|
|
|
|
let cnf_entry = iso
|
|
.find("SYSTEM.CNF")
|
|
.ok_or_else(|| "no SYSTEM.CNF: not a PlayStation 2 game disc".to_string())?;
|
|
let cnf = gc_formats::iso9660::read_at(path, cnf_entry.lba, cnf_entry.size)
|
|
.map_err(|e| format!("cannot read SYSTEM.CNF: {e}"))?;
|
|
let cnf = String::from_utf8_lossy(&cnf);
|
|
|
|
let boot = cnf_value(&cnf, "BOOT2");
|
|
if boot.is_empty() {
|
|
return Err("SYSTEM.CNF has no BOOT2 line: not a PlayStation 2 game disc".into());
|
|
}
|
|
let boot_path = boot
|
|
.strip_prefix("cdrom0:")
|
|
.unwrap_or(&boot)
|
|
.replace('\\', "/")
|
|
// SYSTEM.CNF writes the boot path rooted, e.g. `cdrom0:\SCUS_972.68`.
|
|
// Strip any leading slash so it matches an ISO directory name.
|
|
.trim_start_matches('/')
|
|
.to_string();
|
|
let serial = boot_path.rsplit('/').next().unwrap_or("").to_string();
|
|
|
|
let entry = iso
|
|
.find(&boot_path)
|
|
.ok_or_else(|| format!("SYSTEM.CNF boots {boot_path}, which is not on the disc"))?;
|
|
let elf = gc_formats::iso9660::read_at(path, entry.lba, entry.size)
|
|
.map_err(|e| format!("cannot read the boot executable {serial}: {e}"))?;
|
|
|
|
if elf.len() < 52 || &elf[..4] != b"\x7fELF" {
|
|
return Err(format!("{serial} is not an ELF executable"));
|
|
}
|
|
|
|
Ok(DiscInfo {
|
|
path: path.display().to_string(),
|
|
image_size: std::fs::metadata(path).map(|m| m.len()).unwrap_or(0),
|
|
volume_id: iso.volume_id.clone(),
|
|
serial,
|
|
version: cnf_value(&cnf, "VER"),
|
|
vmode: cnf_value(&cnf, "VMODE"),
|
|
boot_elf_size: entry.size,
|
|
boot_elf_sha1: sha1_hex(&elf),
|
|
})
|
|
}
|
|
|
|
/// SHA-1 of `data`, lowercase hex.
|
|
///
|
|
/// Implemented here rather than pulled in as a dependency so that `identify`
|
|
/// stays usable as a standalone stage with a minimal dependency set.
|
|
pub fn sha1_hex(data: &[u8]) -> String {
|
|
let mut h = Sha1::new();
|
|
h.update(data);
|
|
h.hex()
|
|
}
|
|
|
|
/// Minimal SHA-1 (FIPS 180-4).
|
|
pub struct Sha1 {
|
|
state: [u32; 5],
|
|
buffer: [u8; 64],
|
|
buffered: usize,
|
|
length: u64,
|
|
}
|
|
|
|
impl Default for Sha1 {
|
|
fn default() -> Self {
|
|
Self::new()
|
|
}
|
|
}
|
|
|
|
impl Sha1 {
|
|
pub fn new() -> Self {
|
|
Self {
|
|
state: [
|
|
0x6745_2301,
|
|
0xEFCD_AB89,
|
|
0x98BA_DCFE,
|
|
0x1032_5476,
|
|
0xC3D2_E1F0,
|
|
],
|
|
buffer: [0u8; 64],
|
|
buffered: 0,
|
|
length: 0,
|
|
}
|
|
}
|
|
|
|
pub fn update(&mut self, mut data: &[u8]) {
|
|
self.length = self.length.wrapping_add(data.len() as u64);
|
|
if self.buffered > 0 {
|
|
let take = (64 - self.buffered).min(data.len());
|
|
self.buffer[self.buffered..self.buffered + take].copy_from_slice(&data[..take]);
|
|
self.buffered += take;
|
|
data = &data[take..];
|
|
if self.buffered == 64 {
|
|
let block = self.buffer;
|
|
self.compress(&block);
|
|
self.buffered = 0;
|
|
}
|
|
}
|
|
while data.len() >= 64 {
|
|
let mut block = [0u8; 64];
|
|
block.copy_from_slice(&data[..64]);
|
|
self.compress(&block);
|
|
data = &data[64..];
|
|
}
|
|
if !data.is_empty() {
|
|
self.buffer[..data.len()].copy_from_slice(data);
|
|
self.buffered = data.len();
|
|
}
|
|
}
|
|
|
|
fn compress(&mut self, block: &[u8; 64]) {
|
|
let mut w = [0u32; 80];
|
|
for i in 0..16 {
|
|
w[i] = u32::from_be_bytes([
|
|
block[i * 4],
|
|
block[i * 4 + 1],
|
|
block[i * 4 + 2],
|
|
block[i * 4 + 3],
|
|
]);
|
|
}
|
|
for i in 16..80 {
|
|
w[i] = (w[i - 3] ^ w[i - 8] ^ w[i - 14] ^ w[i - 16]).rotate_left(1);
|
|
}
|
|
let (mut a, mut b, mut c, mut d, mut e) = (
|
|
self.state[0],
|
|
self.state[1],
|
|
self.state[2],
|
|
self.state[3],
|
|
self.state[4],
|
|
);
|
|
for (i, &wi) in w.iter().enumerate() {
|
|
let (f, k) = match i {
|
|
0..=19 => ((b & c) | ((!b) & d), 0x5A82_7999u32),
|
|
20..=39 => (b ^ c ^ d, 0x6ED9_EBA1),
|
|
40..=59 => ((b & c) | (b & d) | (c & d), 0x8F1B_BCDC),
|
|
_ => (b ^ c ^ d, 0xCA62_C1D6),
|
|
};
|
|
let temp = a
|
|
.rotate_left(5)
|
|
.wrapping_add(f)
|
|
.wrapping_add(e)
|
|
.wrapping_add(k)
|
|
.wrapping_add(wi);
|
|
e = d;
|
|
d = c;
|
|
c = b.rotate_left(30);
|
|
b = a;
|
|
a = temp;
|
|
}
|
|
self.state[0] = self.state[0].wrapping_add(a);
|
|
self.state[1] = self.state[1].wrapping_add(b);
|
|
self.state[2] = self.state[2].wrapping_add(c);
|
|
self.state[3] = self.state[3].wrapping_add(d);
|
|
self.state[4] = self.state[4].wrapping_add(e);
|
|
}
|
|
|
|
pub fn hex(mut self) -> String {
|
|
let bit_len = self.length.wrapping_mul(8);
|
|
self.update_raw(&[0x80]);
|
|
while self.buffered != 56 {
|
|
self.update_raw(&[0x00]);
|
|
}
|
|
let len_bytes = bit_len.to_be_bytes();
|
|
self.update_raw(&len_bytes);
|
|
let mut out = String::with_capacity(40);
|
|
for word in self.state {
|
|
out.push_str(&format!("{word:08x}"));
|
|
}
|
|
out
|
|
}
|
|
|
|
/// Feed padding without changing the recorded message length.
|
|
fn update_raw(&mut self, data: &[u8]) {
|
|
for &byte in data {
|
|
self.buffer[self.buffered] = byte;
|
|
self.buffered += 1;
|
|
if self.buffered == 64 {
|
|
let block = self.buffer;
|
|
self.compress(&block);
|
|
self.buffered = 0;
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
#[cfg(test)]
|
|
mod tests {
|
|
use super::*;
|
|
|
|
#[test]
|
|
fn sha1_matches_known_vectors() {
|
|
assert_eq!(sha1_hex(b""), "da39a3ee5e6b4b0d3255bfef95601890afd80709");
|
|
assert_eq!(sha1_hex(b"abc"), "a9993e364706816aba3e25717850c26c9cd0d89d");
|
|
assert_eq!(
|
|
sha1_hex(b"abcdbcdecdefdefgefghfghighijhijkijkljklmklmnlmnomnopnopq"),
|
|
"84983e441c3bd26ebaae4aa1f95129e5e54670f1"
|
|
);
|
|
// Exactly one block.
|
|
assert_eq!(
|
|
sha1_hex(&[b'a'; 64]),
|
|
"0098ba824b5c16427bd7a1122a5a442a25ec644d"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn sha1_handles_multi_block_input() {
|
|
let million: Vec<u8> = std::iter::repeat_n(b'a', 1_000_000).collect();
|
|
assert_eq!(
|
|
sha1_hex(&million),
|
|
"34aa973cd4c4daa4f61eeb2bdbad27316534016f"
|
|
);
|
|
}
|
|
|
|
#[test]
|
|
fn cnf_parsing() {
|
|
let cnf = "BOOT2 = cdrom0:\\SCUS_972.68;1\r\nVER = 1.01\r\nVMODE = NTSC\r\n\r\n";
|
|
assert_eq!(cnf_value(cnf, "BOOT2"), "cdrom0:\\SCUS_972.68");
|
|
assert_eq!(cnf_value(cnf, "VER"), "1.01");
|
|
assert_eq!(cnf_value(cnf, "VMODE"), "NTSC");
|
|
assert_eq!(cnf_value(cnf, "MISSING"), "");
|
|
}
|
|
|
|
#[test]
|
|
fn ntsc_verdict_mentions_sixty_hertz() {
|
|
let d = DiscInfo {
|
|
path: "x.iso".into(),
|
|
image_size: 0,
|
|
volume_id: String::new(),
|
|
serial: TARGET_SERIAL.into(),
|
|
version: TARGET_VERSION.into(),
|
|
vmode: "NTSC".into(),
|
|
boot_elf_size: 0,
|
|
boot_elf_sha1: TARGET_BOOT_ELF_SHA1.into(),
|
|
};
|
|
assert!(d.verdict().contains("60 Hz"), "{}", d.verdict());
|
|
}
|
|
|
|
#[test]
|
|
fn target_verdict_for_the_supported_build() {
|
|
let d = DiscInfo {
|
|
path: "x.iso".into(),
|
|
image_size: 0,
|
|
volume_id: "RATCHETANDCLANK2".into(),
|
|
serial: TARGET_SERIAL.into(),
|
|
version: TARGET_VERSION.into(),
|
|
vmode: "NTSC".into(),
|
|
boot_elf_size: 0,
|
|
boot_elf_sha1: TARGET_BOOT_ELF_SHA1.into(),
|
|
};
|
|
assert!(d.is_target());
|
|
assert!(!d.is_pal());
|
|
assert!(d.verdict().contains("the build we target"));
|
|
}
|
|
|
|
#[test]
|
|
fn greatest_hits_is_called_out_as_a_different_build() {
|
|
let d = DiscInfo {
|
|
path: "x.iso".into(),
|
|
image_size: 0,
|
|
volume_id: String::new(),
|
|
serial: "SCUS_972.68GH".into(),
|
|
version: "2.00".into(),
|
|
vmode: "NTSC".into(),
|
|
boot_elf_size: 0,
|
|
boot_elf_sha1: String::new(),
|
|
};
|
|
assert!(!d.is_target());
|
|
assert!(d.verdict().contains("different build"));
|
|
}
|
|
|
|
#[test]
|
|
fn pal_is_flagged_as_fifty_hertz() {
|
|
let d = DiscInfo {
|
|
path: "x.iso".into(),
|
|
image_size: 0,
|
|
volume_id: String::new(),
|
|
serial: "SCES-516.07".into(),
|
|
version: "1.00".into(),
|
|
vmode: "PAL".into(),
|
|
boot_elf_size: 0,
|
|
boot_elf_sha1: String::new(),
|
|
};
|
|
assert!(d.is_pal());
|
|
assert!(d.verdict().contains("50 Hz"), "{}", d.verdict());
|
|
}
|
|
}
|