Decode the shrub class blob: a 24-normal palette and a billboard record
The shrub body's payload encoding was the open question after the span-bound work, and it answered itself once the R&C1 reference was in view: R&C1's shrub class header keeps the field offsets R&C2 shows (o_class at 0x24, a count at 0x28, the data offset at 0x2c), which says which bytes to look at, and the disc says what they hold. Range measured: all 358 shrub class spans over the 24 levels that have them, each bounded by its successor class or the located chunk container. Nothing outside the spans is read. Result. The body is a palette of 24 unit-length s16 normals -- s16 x, y, z and a zero pad word per 8-byte record -- followed by all-zero padding records. Three independent measurements, each of which fails for any other reading: - all 8,592 live entries have squared length within 65,535 of 32767^2, and the bound is tight (the largest observed deviation is exactly 65,535). That is 32768^2 - 32767^2, which is what a unit vector scaled by 32768 produces when an axis-aligned normal quantises to -32768 and the positive axis clamps to 32767; components run the full s16 range and 1,765 entries are axis-aligned. A random s16 triple is off by about a billion, so one entry in fifteen thousand would pass by chance: all 8,592 pass. - the padding records are always at index 24 or later and always all zero, never a live entry. That is what separates the palette from geometry, and it is the observation the three earlier stride-fitting attempts were missing. - 17 classes carry a palette of 24 identical (0, 0, 1) entries. A position or index table cannot produce that; a normal palette for a flat-facing class must. The header decodes as: sphere at +0x00 (centre + radius, radius positive on all 358 spans, 1.72..41.57), size at +0x10, mode_bits and scratch at +0x14, a zero word at +0x18, the billboard offset at +0x1c, and scale at +0x20. The parser asserts the two invariants that hold without exception (scratch and the reserved word are zero on all 358) plus a positive radius and scale, and reports mode_bits rather than asserting it: it is 4 on 16 of the 358 spans. The billboard offset is the strongest self-identification after o_class because it is confirmed from a second file: it is non-zero on exactly the 25 classes whose core-index entry carries billboard texture fields, pairing correctly both ways. At the offset sits R&C1's ShrubBillboard shape, register block and all -- fade_distance, then TEX1_1 (K -233..-168, MMIN 4, address 0x14), TEX0_1 (index 1..4, address 0x06) and MIPTBP1_1 (address 0x34), all three register addresses checked in the disc test. Two corrections this produces, both recorded in docs/plan/level-assets.md because the old readings were confidently wrong rather than merely unverified: - "reserved is 0 or a denormal" had the two u16 halves the wrong way round: the 16 non-zero spans are mode_bits = 4 with scratch = 0. As an f32 the word is still a denormal, so the PS2 reads zero either way -- but the field is a flag word, not a magnitude, which is exactly the FPU point worth making. - "bounds" was tested as a min.xyz/max.xyz box, which holds on only 146 of 358 spans, and the field was left unnamed as a result. It is a bounding sphere: the fourth float is positive on all 358 spans and the centres cluster near the class origin. The 146/358 figure was a property of the wrong question. Still open: the three floats in the billboard record (they repeat byte-identically across levels -- 16 distinct triples among 25 records, one on four classes across levels 1, 4, 12 and 21 -- so they are shared art data rather than dimensions), size, mode_bits, count, and where the shrub mesh geometry lives. R&C1 puts the mesh packets in this blob; R&C2's blob holds only the palette, so the 333 classes without a billboard must draw meshes from somewhere else.
This commit is contained in:
@@ -1620,11 +1620,17 @@ fn moby_parts_tile_on_every_level() {
|
||||
);
|
||||
}
|
||||
|
||||
/// Shrub class headers.
|
||||
/// Shrub class headers and blobs.
|
||||
///
|
||||
/// Two fields self-identify exactly on every span, and the parser checks both
|
||||
/// rather than merely reporting them: `+0x24` must equal the owning class
|
||||
/// entry's own `o_class`, and `+0x2c` must land inside the span.
|
||||
///
|
||||
/// The body is then checked against the R&C1 shape this format follows: a
|
||||
/// palette of 24 unit-length `s16` normals with a zero pad word, then zero
|
||||
/// padding records. And the billboard offset at `+0x1c` is checked against the
|
||||
/// class entry's billboard texture fields, which is the one header field that
|
||||
/// can be confirmed from a second file.
|
||||
#[test]
|
||||
fn shrub_headers_name_their_own_class_on_every_level() {
|
||||
require_disc!();
|
||||
@@ -1640,14 +1646,23 @@ fn shrub_headers_name_their_own_class_on_every_level() {
|
||||
}
|
||||
|
||||
use gc_formats::level_core::ShrubClassEntry;
|
||||
use gc_formats::shrub::{ShrubHeader, SHRUB_CLASS_ENTRY_BYTES};
|
||||
use gc_formats::shrub::{ShrubHeader, SHRUB_CLASS_ENTRY_BYTES, SHRUB_NORMAL_COUNT};
|
||||
let mut levels = 0usize;
|
||||
let mut spans = 0usize;
|
||||
let mut absent = 0usize;
|
||||
let (mut min_size, mut max_size) = (f32::MAX, 0.0f32);
|
||||
let mut denormals = 0usize;
|
||||
let mut scratch_nonzero = 0usize;
|
||||
let mut mode_hist: std::collections::BTreeMap<u16, usize> = Default::default();
|
||||
let mut scratch_hist: std::collections::BTreeMap<u16, usize> = Default::default();
|
||||
let mut radius_range = (f32::MAX, 0f32);
|
||||
let mut scale_range = (f32::MAX, 0f32);
|
||||
let mut bounded_records = 0usize;
|
||||
let mut bounded_bodies = 0usize;
|
||||
let mut palette_records = 0usize;
|
||||
let mut padding_records = 0usize;
|
||||
let mut norm2_deviation = 0i64;
|
||||
let mut billboards = 0usize;
|
||||
let mut billboard_entries = 0usize;
|
||||
|
||||
for n in 0..27 {
|
||||
let dir = data.join(format!("level{n}"));
|
||||
@@ -1720,26 +1735,81 @@ fn shrub_headers_name_their_own_class_on_every_level() {
|
||||
);
|
||||
assert!(!s.data.is_empty(), "level{n} class {i}: empty shrub data");
|
||||
assert!(
|
||||
s.header.bounds.iter().all(|v| v.is_finite()),
|
||||
"level{n} class {i}: non-finite shrub bounds"
|
||||
s.header.sphere.iter().all(|v| v.is_finite()),
|
||||
"level{n} class {i}: non-finite shrub sphere"
|
||||
);
|
||||
assert!(
|
||||
s.header.size.is_finite() && s.header.size > 0.0,
|
||||
"level{n} class {i}: shrub size {}",
|
||||
s.header.size
|
||||
);
|
||||
// The PS2 has no denormals, so a denormal here and a zero
|
||||
// are the same value to the hardware. Both occur on disc.
|
||||
if s.header.reserved != 0.0 {
|
||||
assert!(
|
||||
s.header.reserved.is_subnormal(),
|
||||
"level{n} class {i}: reserved {} is neither zero nor denormal",
|
||||
s.header.reserved
|
||||
);
|
||||
denormals += 1;
|
||||
// The scratch word is zero on every span; the parser
|
||||
// refuses anything else. `mode_bits` is 4 on 16 spans and
|
||||
// is reported rather than asserted. On those 16 the word
|
||||
// at +0x14 reads as an f32 denormal, which the Emotion
|
||||
// Engine flushes to zero like everything else.
|
||||
if s.header.scratch != 0 {
|
||||
scratch_nonzero += 1;
|
||||
}
|
||||
*mode_hist.entry(s.header.mode_bits).or_default() += 1;
|
||||
*scratch_hist.entry(s.header.scratch).or_default() += 1;
|
||||
radius_range.0 = radius_range.0.min(s.header.sphere[3]);
|
||||
radius_range.1 = radius_range.1.max(s.header.sphere[3]);
|
||||
scale_range.0 = scale_range.0.min(s.header.scale);
|
||||
scale_range.1 = scale_range.1.max(s.header.scale);
|
||||
min_size = min_size.min(s.header.size);
|
||||
max_size = max_size.max(s.header.size);
|
||||
|
||||
// The palette: 24 unit normals, then zero padding.
|
||||
let normals: Vec<_> = s.normals().collect();
|
||||
assert_eq!(
|
||||
normals.len(),
|
||||
SHRUB_NORMAL_COUNT,
|
||||
"level{n} class {i}: palette is {} entries, not {SHRUB_NORMAL_COUNT}",
|
||||
normals.len()
|
||||
);
|
||||
for (k, normal) in normals.iter().enumerate() {
|
||||
assert!(
|
||||
normal.is_unit(),
|
||||
"level{n} class {i}: palette entry {k} {normal:?} is not a unit normal"
|
||||
);
|
||||
let n2 = (normal.x as i64).pow(2)
|
||||
+ (normal.y as i64).pow(2)
|
||||
+ (normal.z as i64).pow(2);
|
||||
norm2_deviation = norm2_deviation.max((n2 - 32767i64.pow(2)).abs());
|
||||
}
|
||||
palette_records += normals.len();
|
||||
for (k, record) in s.body_records().skip(SHRUB_NORMAL_COUNT).enumerate() {
|
||||
assert!(
|
||||
record.iter().all(|&b| b == 0),
|
||||
"level{n} class {i}: padding record {k} is not all zero"
|
||||
);
|
||||
}
|
||||
padding_records += s.padding_record_count();
|
||||
|
||||
// The billboard: present exactly when the class entry
|
||||
// carries billboard texture fields.
|
||||
let entry_has_billboard = e.billboard_width != 0 || e.billboard_height != 0;
|
||||
match s.billboard().expect("billboard parses") {
|
||||
Some(b) => {
|
||||
billboards += 1;
|
||||
assert!(
|
||||
entry_has_billboard,
|
||||
"level{n} class {i}: billboard record without billboard texture fields"
|
||||
);
|
||||
assert_eq!(b.tex1_addr, 0x14, "level{n} class {i}: TEX1 address");
|
||||
assert_eq!(b.tex0_addr, 0x06, "level{n} class {i}: TEX0 address");
|
||||
assert_eq!(b.miptbp_addr, 0x34, "level{n} class {i}: MIPTBP address");
|
||||
assert_eq!(b.mmin, 4, "level{n} class {i}: MMIN");
|
||||
}
|
||||
None => assert!(
|
||||
!entry_has_billboard,
|
||||
"level{n} class {i}: billboard texture fields with no billboard record"
|
||||
),
|
||||
}
|
||||
if entry_has_billboard {
|
||||
billboard_entries += 1;
|
||||
}
|
||||
spans += 1;
|
||||
}
|
||||
Ok(None) => absent += 1,
|
||||
@@ -1759,11 +1829,19 @@ fn shrub_headers_name_their_own_class_on_every_level() {
|
||||
assert!(spans > 340, "only {spans} shrub spans with geometry");
|
||||
// The container bounds every class now, so every body is checkable.
|
||||
assert_eq!(bounded_bodies, spans, "every shrub class should be bounded");
|
||||
assert_eq!(scratch_nonzero, 0, "the scratch word is zero on every span");
|
||||
assert_eq!(
|
||||
billboards, billboard_entries,
|
||||
"billboard records and billboard class entries disagree: \
|
||||
{billboards} records vs {billboard_entries} entries"
|
||||
);
|
||||
eprintln!(
|
||||
"shrub: {spans} headers naming their own class, {absent} without geometry; \
|
||||
all {bounded_bodies} class bodies bounded by the chunk container, \
|
||||
{bounded_records} 8-byte records in total, \
|
||||
{denormals} denormal reserved words"
|
||||
all {bounded_bodies} bodies bounded, {bounded_records} records, \
|
||||
{palette_records} of them palette normals and {padding_records} zero padding; \
|
||||
{billboards} billboards, all with texture fields; \
|
||||
mode_bits {mode_hist:?}, radius {radius_range:?}, scale {scale_range:?}, \
|
||||
largest palette deviation from unit length {norm2_deviation} of a possible 65535"
|
||||
);
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user