Correct the chunk loader finding and decode a record field

The loader note said every record gains a 02 00 prefix in RAM. It does not:
the word is the record's own leading field, already on the disc at every
record. What the loader changes is the pointer -- it addresses the word, two
bytes earlier than the disc's table -- and that pointer shift is what made the
bytes look inserted.

Measured, in order:

- the two bytes before a payload are identical in core_data and in RAM, record
  for record: 02 00 on level 0's 90-record chunk at 0x100b302, 08 00 on the
  level's first chunk. Nothing is inserted;
- the RAM table's low 24 bits are the payload's minus 2, consistently, so the
  record is [leading u16][two index bytes][payload] and the loaded table
  addresses the leading word. That is also why the index bytes read as +2 from
  the pointer -- the same +2 the earlier RAM result quotes;
- the word is a field, not a pad: uniform across a chunk's records on 1,439 of
  1,509 chunks (None on the 70 that vary, reported rather than averaged),
  fixed within all 71 chunk types, and only ever 1, 2, 4, 6, 8 or 10 -- 554
  chunks carry 2, 520 carry 4.

Two readings of the value were measured and disproved, so they are recorded as
disproved rather than left as candidates: it is not an entry count (the payload
divided by it leaves 57 distinct fractional "sizes", and 162 chunks do not
divide at all), and it is not the number of distinct leading bytes (agrees on 0
of 1,506 chunks).

Chunk gains `prefix: Option<u16>`, and the disc test asserts the uniformity,
the per-type fixity and the value set. The scratch example that produced the
per-type class-ID table is dropped now that the same scan is a committed test,
which also lands here: moby class IDs sit in one body slot per chunk type, with
the slot varying by type and the distinct >= 4 filter load-bearing -- without it
the top slots are constants that read as 305x because any value below the moby
count counts as a hit.
This commit is contained in:
2026-10-12 06:00:29 +02:00
parent 8147110377
commit c2af0528e3
4 changed files with 324 additions and 5 deletions
+54
View File
@@ -43,6 +43,16 @@ pub const CHUNK_GAP_BYTES: usize = 10;
/// The value every tagged offset's top byte carries, on all 27 levels.
pub const CHUNK_TAG: u8 = 0xf0;
/// Bytes between a chunk record's leading word and the payload the table
/// points at.
///
/// A record is `[leading u16][two index bytes][payload]`, and the disc's
/// offset table points at the *payload*. The loaded table in RAM points two
/// bytes earlier, at the leading word -- the payload's index bytes are then at
/// `+2` from the pointer -- so the game's record starts here and the disc's
/// table is two bytes late. See [`Chunk::prefix`].
pub const RECORD_PREFIX_BYTES: usize = 2;
/// The two bytes immediately preceding every chunk region on the disc.
///
/// The last class body in a level -- shrub where the level has shrub, tie
@@ -112,6 +122,26 @@ pub struct Chunk {
pub offsets: Vec<u32>,
/// Record stride, from the first two offsets. Zero if fewer than two.
pub stride: usize,
/// The `u16` immediately before the first record's payload -- the record's
/// own leading word, which the loaded table points at.
///
/// **Measured on the disc:** the same at every record of a chunk on
/// **1,439 of 1,509 chunks** -- `None` on the 70 that vary -- and fixed
/// within every chunk type, over the 71 types. Its values are 1, 2, 4, 6,
/// 8 and 10; 554 chunks carry 2 and 520 carry 4.
///
/// **Measured against the running game:** in PCSX2's RAM the rewritten
/// offset table points at this word rather than at the payload, two bytes
/// earlier than the disc's table, so the game's record starts here.
///
/// Not named. What the value counts is not established: it is *not* the
/// payload's entry count (the payload length divided by it is a whole
/// number on 1,344 chunks and an odd fraction on the other 162, leaving 57
/// distinct "sizes"), and it is not the number of distinct leading bytes
/// (agrees on 0 of 1,506). An earlier note in this file read the
/// `02 00` seen in RAM as a prefix the loader had inserted; those bytes
/// are on the disc already, at every record, which is what this field is.
pub prefix: Option<u16>,
}
/// One chunk, with its records already bounds-checked.
@@ -324,12 +354,35 @@ pub fn parse_chunk(region: &[u8], offset: usize) -> Result<Option<Chunk>> {
0
};
// The leading word sits two bytes before the payload the table points at.
// Reading it and requiring it to be the same at every record is a
// signature check like the header's last byte: it holds on every chunk on
// The leading word sits two bytes before the payload the table points at.
// It is the same at every record of a chunk on 1,439 of 1,509 chunks; the
// other 70 vary, so it stays an Option rather than a value a caller could
// mistake for a field that is always there.
let read_prefix = |low: u32| -> Option<u16> {
let at = offset
.checked_add(low as usize)?
.checked_sub(RECORD_PREFIX_BYTES)?;
region
.get(at..at + RECORD_PREFIX_BYTES)
.map(|b| u16::from_le_bytes([b[0], b[1]]))
};
let prefix = offsets.first().copied().and_then(read_prefix).filter(|&p| {
offsets
.iter()
.skip(1)
.all(|&low| read_prefix(low) == Some(p))
});
Ok(Some(Chunk {
count: header[14],
header,
offset,
offsets,
stride,
prefix,
}))
}
@@ -360,6 +413,7 @@ pub fn resolve<'r>(chunk: &Chunk, region: &'r [u8]) -> Result<ResolvedChunk<'r>>
chunk: Chunk {
count: chunk.count,
header: chunk.header,
prefix: chunk.prefix,
offset: chunk.offset,
offsets: chunk.offsets.clone(),
stride: chunk.stride,