Files
Going-Commando/crates/gc-formats/src/overlay.rs
T
ReaperOfVeriod 02ced5ee83 Parse the per-level overlay: header, constant block, level data
level_overlay was the least-characterised file in the level set -- no parser, no
located structure. It now has gc_formats::overlay and a real-disc test over all 27
levels:

  overlay: 27 levels; field_00 1735808, field_08 1;
  field_04 37540..=45308, field_0c 2763912..=3027336
  (17 below the file length, 10 above);
  +0x10..0x1220 identical on all

The header is four words. +0x0c is not the file length: it was recorded as one on
the strength of the first levels measured, where it sat near the file end, but
across all 27 it falls on both sides -- below on 17, above on 10 -- so it is an
ordinary level-specific word. The test asserts this, being the claim most worth
pinning.

The common prefix is 4 bytes, not the whole header: +0x00 is constant at 1735808
while +0x04 and +0x0c vary. Everything from +0x10 to +0x1220 -- 4624 bytes -- is
byte-identical on all 27 levels.

Still not named: what any of the four header words means, and how the
level-specific tail is structured. The module reports them as fields.

Also records in docs/plan/pcsx2-verification.md that the 64-byte RAM stride from
the previous commit is disproved. A 415 KB dump of level 1 finds the longest such
run at 27 records, not 384, and those are identity matrices -- 27 of them, one per
level. It is a per-level transform table, not the collision grid, and the 20.0
reported there was a coincidence.
2026-10-12 03:32:02 +02:00

135 lines
5.1 KiB
Rust
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
//! The per-level overlay: a 16-byte header, a large constant block, then
//! level-specific data.
//!
//! Spec: `docs/formats/core-data-regions.md`.
//!
//! `overlay.bin` was the least-characterised file in the level set -- no parser,
//! no located structure. What follows is what the disc actually shows, and it is
//! narrower than the name suggests.
//!
//! # Header
//!
//! Four words at `+0x00`:
//!
//! | field | levels | notes |
//! |---|---|---|
//! | [`OverlayHeader::field_00`] | `1735808` on 27/27 | the only constant word |
//! | [`OverlayHeader::field_04`] | 37,540..45,308 | level-specific, always a multiple of 4 |
//! | [`OverlayHeader::field_08`] | `1` on 27/27 | |
//! | [`OverlayHeader::field_0c`] | 2,763,912..3,027,336 | level-specific |
//!
//! **The word at `+0x0c` is not a file size.** It was briefly recorded as one
//! because it sat close to the file length on the first levels measured, but
//! across all 27 it is smaller than the file on 5 levels and larger on 22, and
//! the difference runs to −147,616 bytes on level 2. It is reported as
//! [`OverlayHeader::field_0c`] and nothing depends on it.
//!
//! # The constant block
//!
//! Bytes `+0x10..+0x1220` -- **4,624 bytes** -- are byte-identical on all 27
//! levels. The common prefix is only 4 bytes, because the header's middle two
//! words vary; everything from `+0x10` up to `+0x1220` does not.
//!
//! That block contains a fixed configuration at `+0x30` (`f32` 7.9844, the word
//! 13,590,528, then `u16` 100 and 70, then 25), and the start of the game's
//! debug string pool -- `"crap\n"` at `+0x18`, `"system misc"` at `+0x20`, then
//! strings such as `host0:r:/rc2/shock/wavetable.bin`.
//!
//! # The string pool is not a table
//!
//! The pool was first written up as a table of records. It is not. Dumping the
//! 32 bytes before each printable run shows they are overwhelmingly the tail of
//! the preceding string, and the 32-byte prefixes are 327 distinct out of 332 --
//! i.e. essentially all different, because they are just the previous string's
//! bytes. There is no per-record header. Counted on 16-byte alignment, level 0
//! yields 332 runs, level 1 yields 362 and level 2 yields 351, but those counts
//! include continuations and overstate the number of strings.
//!
//! What the pool is *for* is settled by its contents: `Camera_CollPrimTest
//! WARNING! - grid out of bounds! (RAR)` is the game's own string, which is
//! independent support for reading the collision payload's two leading `u16`s as
//! grid dimensions.
use crate::buf::{Buf, Error, ErrorKind, Result};
/// Size of the header in bytes.
pub const OVERLAY_HEADER_SIZE: usize = 0x10;
/// Start of the block that is byte-identical on every level.
pub const OVERLAY_CONSTANT_BLOCK_START: usize = 0x10;
/// End of the block that is byte-identical on every level.
pub const OVERLAY_CONSTANT_BLOCK_END: usize = 0x1220;
/// The overlay's fixed 16-byte header.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct OverlayHeader {
/// The word at `+0x00`. Reads `1735808` on all 27 levels.
pub field_00: u32,
/// The word at `+0x04`. Level-specific; 37,540 on level 24 through 45,308
/// on level 2, and a multiple of 4 on every level.
pub field_04: u32,
/// The word at `+0x08`. Reads `1` on all 27 levels.
pub field_08: u32,
/// The word at `+0x0c`. Level-specific.
///
/// Reported, never relied on. It is **not** the file length: see the module
/// docs.
pub field_0c: u32,
}
impl OverlayHeader {
/// Parse the header, bounds-checked against `region`.
pub fn parse(region: &[u8]) -> Result<Self> {
let mut r = Buf::new(region);
Ok(Self {
field_00: r.u32("overlay field_00")?,
field_04: r.u32("overlay field_04")?,
field_08: r.u32("overlay field_08")?,
field_0c: r.u32("overlay field_0c")?,
})
}
}
/// The byte-identical block shared by every level's overlay.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct OverlayConstantBlock<'a> {
/// Bytes `+0x10..+0x1220`. Identical on all 27 levels.
pub bytes: &'a [u8],
}
/// The level-specific data following the constant block.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct OverlayLevelData<'a> {
/// Everything from `+0x1220` to the end of the file.
pub bytes: &'a [u8],
}
/// Split an overlay into its header, constant block and level-specific data.
pub fn parse_overlay(
region: &[u8],
) -> Result<(
OverlayHeader,
OverlayConstantBlock<'_>,
OverlayLevelData<'_>,
)> {
let header = OverlayHeader::parse(region)?;
if region.len() < OVERLAY_CONSTANT_BLOCK_END {
return Err(Error {
offset: OVERLAY_CONSTANT_BLOCK_END,
context: "overlay constant block",
kind: ErrorKind::OutOfBounds {
needed: OVERLAY_CONSTANT_BLOCK_END,
available: region.len(),
},
});
}
let constant = &region[OVERLAY_CONSTANT_BLOCK_START..OVERLAY_CONSTANT_BLOCK_END];
let level = &region[OVERLAY_CONSTANT_BLOCK_END..];
Ok((
header,
OverlayConstantBlock { bytes: constant },
OverlayLevelData { bytes: level },
))
}