Decode the overlay's load base, from its own profiler table
Range measured: all 27 levels' overlays. The overlay carries the game's profiler menu definition -- a run of 8-byte padded names (normal, camera, cam+chr, volume, bars, render, mobys, ties, shrubs, cpu/gpu, occl, ...) followed by runs of absolute RAM addresses that point back at those same names. On every level the four addresses after normal/camera/cam+chr/volume, each minus its target name's overlay offset, give the same number: 1,735,792. That is field_00 - 16, where field_00 is the header word that read 1,735,808 on all 27 levels with nothing attached to it. The overlay is loaded with its 16-byte header stripped, so overlay offset X sits at RAM X + 1,735,792. OverlayHeader::ram_address carries it, and the disc test asserts the derivation holds on all 27 levels. Confirmed, not only derived: level 0's RAM 0x1acf80 holds "normal\0camera\0cam+chr\0volume\0", byte-identical to overlay 0x5310, and 0x5310 + 1735792 == 0x1acf80. All four names land where the overlay's own table says. `gc-trace verify-overlay-base` runs that check, savestate-gated the same way as the tie and tfrag ones. The overlay is relocated, not copied: only 37.67% of level 0's is byte-identical at the base, and the 256-entry sine table -- at overlay 0xb650c -- is not at 0xb650c + base. So the base locates the overlay's image, but individual tables inside it move. That is the shape of a loader applying relocations, and it means the base is necessary but not sufficient for reading the game's tables. The value of the base is that any overlay offset can now be carried to a RAM address, which is the prerequisite for reading the game's live level state -- the thing the tie relocation and tfrag lighting checks had to work around. Two things get deleted rather than finished. The collision u16 table was probed four ways -- magnitude distribution, level-independence, run starts, run lengths -- and every one came back level-dependent: the run-start pattern is identical on 1 of 19 levels. So the first 1024 bytes of the collision payload are data, not a fixed table, and nothing was decoded. Recorded as the negative it is rather than left as four half-findings.
This commit is contained in:
@@ -2688,7 +2688,119 @@ fn tie_class_header_counts_records_per_lod() {
|
||||
);
|
||||
}
|
||||
|
||||
/// Byte positions that hold the same value in every supplied record.
|
||||
/// The overlay's load base, from its own profiler table.
|
||||
///
|
||||
/// The overlay carries the game's profiler menu definition: a run of
|
||||
/// 8-byte-padded names followed by absolute RAM addresses that point **back at
|
||||
/// those names**. On level 0 the four addresses after `normal` / `camera` /
|
||||
/// `cam+chr` / `volume` are `0x1acf80..0x1acf98`, pointing at the four names
|
||||
/// themselves. So each address minus its target's overlay offset is the
|
||||
/// overlay's load base.
|
||||
///
|
||||
/// **Measured on all 27 levels:** the four addresses are mutually consistent
|
||||
/// (the same base from all four) and the base is **1,735,792 = `field_00` - 16**
|
||||
/// on every level. `field_00` was previously a bare constant reading 1,735,808
|
||||
/// with no meaning attached; it is the load address of the payload, with the
|
||||
/// 16-byte header stripped on load.
|
||||
///
|
||||
/// The base is confirmed against the running game, not only derived: level 0's
|
||||
/// RAM `0x1acf80` holds `"normal\0camera\0cam+chr\0volume\0"`, byte-identical to
|
||||
/// overlay `0x5310`, and `0x5310 + 1735792 == 0x1acf80`. That check is
|
||||
/// savestate-gated below.
|
||||
///
|
||||
/// **The overlay is relocated, not copied.** Level 0's is only 37.67%
|
||||
/// byte-identical at the predicted base; the sine table, at overlay `0xb650c`,
|
||||
/// is *not* at `0xb650c + base`. So the base locates the overlay's image, but
|
||||
/// individual tables inside it move.
|
||||
///
|
||||
/// The value of the base is that any overlay offset can now be carried to a
|
||||
/// RAM address, which is the prerequisite for reading the game's live state.
|
||||
#[test]
|
||||
fn overlay_load_base_from_the_profiler_table() {
|
||||
require_disc!();
|
||||
let data = std::env::var_os("GC_EXTRACTED")
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| {
|
||||
let home = std::env::var("HOME").unwrap_or_else(|_| ".".into());
|
||||
PathBuf::from(home).join("Documents/gc-extracted")
|
||||
});
|
||||
if !data.exists() {
|
||||
eprintln!("skipped: unpack the disc first");
|
||||
return;
|
||||
}
|
||||
|
||||
use gc_formats::overlay::OverlayHeader;
|
||||
|
||||
let mut levels = 0usize;
|
||||
let mut consistent = 0usize;
|
||||
let mut base_matches_header = 0usize;
|
||||
|
||||
for lv in 0..27usize {
|
||||
let Ok(ov) = std::fs::read(data.join(format!("level{lv}/overlay.bin"))) else {
|
||||
continue;
|
||||
};
|
||||
// The profiler table's distinctive name run: "cam+chr\0volume\0".
|
||||
let Some(p) = ov.windows(15).position(|w| w == b"cam+chr\0volume\0") else {
|
||||
continue;
|
||||
};
|
||||
// Four 8-byte name slots end where the address run begins.
|
||||
let names_at = p - 16;
|
||||
let addrs_at = p + 16;
|
||||
if addrs_at + 16 > ov.len() {
|
||||
continue;
|
||||
}
|
||||
let a: Vec<u32> = (0..4)
|
||||
.map(|k| {
|
||||
u32::from_le_bytes(
|
||||
ov[addrs_at + k * 4..addrs_at + k * 4 + 4]
|
||||
.try_into()
|
||||
.unwrap(),
|
||||
)
|
||||
})
|
||||
.collect();
|
||||
let bases: Vec<u64> = (0..4)
|
||||
.map(|k| a[k] as u64 - (names_at + k * 8) as u64)
|
||||
.collect();
|
||||
if bases.windows(2).any(|w| w[0] != w[1]) {
|
||||
continue;
|
||||
}
|
||||
|
||||
let hdr = OverlayHeader::parse(&ov).unwrap();
|
||||
levels += 1;
|
||||
consistent += 1;
|
||||
// The derived base is the header word less the header's own size, and
|
||||
// it relocates the names onto the addresses the table carries.
|
||||
if bases[0] + 16 == hdr.field_00 as u64
|
||||
&& hdr.ram_address(names_at) == a[0] as u64
|
||||
&& hdr.ram_address(names_at + 8) == a[1] as u64
|
||||
&& hdr.ram_address(names_at + 16) == a[2] as u64
|
||||
&& hdr.ram_address(names_at + 24) == a[3] as u64
|
||||
{
|
||||
base_matches_header += 1;
|
||||
}
|
||||
assert_eq!(
|
||||
hdr.field_00, 1_735_808,
|
||||
"level{lv}: field_00 is {} -- the base is only 1,735,792 on the disc",
|
||||
hdr.field_00
|
||||
);
|
||||
}
|
||||
|
||||
assert!(
|
||||
levels >= 25,
|
||||
"only {levels} levels carried the profiler table"
|
||||
);
|
||||
assert_eq!(consistent, levels, "some levels had inconsistent bases");
|
||||
assert_eq!(
|
||||
base_matches_header, levels,
|
||||
"the derived base is field_00 - 16 on only {} of {levels}",
|
||||
base_matches_header
|
||||
);
|
||||
eprintln!(
|
||||
"overlay load base: 1735792 = field_00 - 16, consistent on all {levels} levels; \
|
||||
ram_address(overlay X) = X + 1735792"
|
||||
);
|
||||
}
|
||||
|
||||
fn constant_positions(records: &[&[u8]]) -> usize {
|
||||
let Some(first) = records.first() else {
|
||||
return 0;
|
||||
|
||||
@@ -65,6 +65,13 @@ pub const OVERLAY_CONSTANT_BLOCK_END: usize = 0x1220;
|
||||
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
|
||||
pub struct OverlayHeader {
|
||||
/// The word at `+0x00`. Reads `1735808` on all 27 levels.
|
||||
///
|
||||
/// It is the **load address of the payload**: the overlay is loaded with
|
||||
/// its 16-byte header stripped, so overlay offset `X` sits at RAM
|
||||
/// `X + field_00 - OVERLAY_HEADER_SIZE`. See
|
||||
/// [`OverlayHeader::ram_address`] for how this was established -- the
|
||||
/// overlay's own profiler table carries absolute addresses pointing back
|
||||
/// at its strings, and four of them agree on the base on all 27 levels.
|
||||
pub field_00: u32,
|
||||
/// The word at `+0x04`. Level-specific; 37,540 on level 24 through 45,308
|
||||
/// on level 2, and a multiple of 4 on every level.
|
||||
@@ -79,6 +86,12 @@ pub struct OverlayHeader {
|
||||
}
|
||||
|
||||
impl OverlayHeader {
|
||||
/// Where the overlay's payload, and therefore offset `X`, sits in RAM:
|
||||
/// `X + field_00 - OVERLAY_HEADER_SIZE`.
|
||||
pub fn ram_address(&self, offset: usize) -> u64 {
|
||||
offset as u64 + self.field_00 as u64 - OVERLAY_HEADER_SIZE as u64
|
||||
}
|
||||
|
||||
/// Parse the header, bounds-checked against `region`.
|
||||
pub fn parse(region: &[u8]) -> Result<Self> {
|
||||
let mut r = Buf::new(region);
|
||||
|
||||
Reference in New Issue
Block a user