Parse the level core class tables: moby, tie and shrub

These were ruled out for want of an entry size. The size is knowable
exactly, because each table ends precisely where the next begins:

  moby  238 entries at   192, 32 bytes:  192 + 238*32 =  7808 = tie's
  tie    74 entries at  7808, 32 bytes: 7808 +  74*32 = 10176 = shrub's
  shrub   7 entries at 10176, 48 bytes: 10176 +   7*48 = 10512 = textures

That closes on 27 of 27 levels, which is what makes it evidence rather
than a coincidence: any wrong entry size breaks the chain immediately.
It is the same "does this hold on every level" discriminator the project
already uses for the container pair table.

Across the disc: 5,796 moby, 1,864 tie and 358 shrub classes, plus the
1,595 tfrag textures. Class entries are 32 bytes -- asset offset, class
number, two unknown words and 16 bytes of texture slots -- and a shrub
entry adds 16 bytes of billboard slots for 48.

The header now decodes through to the texture tables rather than
skipping 0x18..0x30 wholesale, which also recovers the moby, tie and
shrub texture ranges. Two more pairs sit between shrub textures and the
textures base offset.

Two things worth recording. ClassEntry::parse read its 16 texture bytes
before its four i32s, because the struct literal that uses them comes
after the loop that advances the cursor -- so every entry was parsed
backwards. Struct literal field order does not reorder the reads that
already happened. And every shrub class on level 0 carries all-zero
billboard slots, so the dimensions are optional rather than something
worth asserting on.

This is the third instance of the same failure shape. The gameplay
extent was "an unidentified compression" because it was parsed
compressed; the tfrag record framing was "unrecoverable" because the
entry size was never known. Both looked like properties of the format
and were properties of the question.

Also records that point lights are a runtime feature rather than missing
work: every one of the 18,102 patches has point_lights == 0xffff and the
eight-slot bank is filled by gameplay, so there is nothing for them to
light in a static level render.

And that PCSX2 is not installed, so the comparison loop cannot be built
yet -- which is why lighting magnitudes stay unverified.

crates/gc-extract/tests/disc.rs

@@ -970,3 +970,125 @@ fn tfrag_lod_parents_resolve_for_every_entry() {
          {lod01} LOD-01 and {lod0} LOD-0, all resolved"
     );
 }
+
+/// The level core index's class tables.
+///
+/// These are what put the environment on top of the terrain: tie instances,
+/// moby actors and shrub foliage. They were ruled out earlier for want of an
+/// entry size, which turned out to be knowable exactly.
+///
+/// The check is the chain: each table must end precisely where the next one
+/// begins, moby (32 bytes/entry) into tie (32) into shrub (48) into the tfrag
+/// texture table. That closes on all 27 levels and would not close if any
+/// entry size were wrong, so it is the evidence rather than a coincidence.
+#[test]
+fn level_core_class_tables_tile_the_index() {
+    require_disc!();
+    gc_formats::iso9660::IsoImage::open(std::path::Path::new(
+        &std::env::var("GC_DISC").expect("GC_DISC is set by require_disc!"),
+    ))
+    .expect("ISO should open");
+    let data = std::env::var_os("GC_EXTRACTED")
+        .map(PathBuf::from)
+        .unwrap_or_else(|| {
+            let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
+            PathBuf::from(home).join("Documents/gc-extracted")
+        });
+    if !data.exists() {
+        eprintln!("skipped: unpack the disc first");
+        return;
+    }
+
+    use gc_formats::level_core as lc;
+    let mut levels = 0usize;
+    let (mut moby, mut tie, mut shrub, mut textures) = (0usize, 0usize, 0usize, 0usize);
+
+    for n in 0..27 {
+        let dir = data.join(format!("level{n}"));
+        let Ok(index) = std::fs::read(dir.join("core_index.bin")) else {
+            continue;
+        };
+        let h = lc::LevelCoreHeader::parse(&index).expect("core header");
+
+        let end = |r: lc::ArrayRange, size: usize| r.offset as usize + r.count as usize * size;
+        assert_eq!(
+            end(h.moby_classes, lc::MOBY_CLASS_SIZE),
+            h.tie_classes.offset as usize,
+            "level{n}: moby -> tie"
+        );
+        assert_eq!(
+            end(h.tie_classes, lc::TIE_CLASS_SIZE),
+            h.shrub_classes.offset as usize,
+            "level{n}: tie -> shrub"
+        );
+        assert_eq!(
+            end(h.shrub_classes, lc::SHRUB_CLASS_SIZE),
+            h.tfrag_textures.offset as usize,
+            "level{n}: shrub -> textures"
+        );
+
+        // Every entry parses, and its asset offset lands inside the level.
+        if n == 0 {
+            eprintln!(
+                "hdr moby={:?} tie={:?} shrub={:?} tex={:?} base={}",
+                h.moby_classes, h.tie_classes, h.shrub_classes, h.tfrag_textures, h.textures_base
+            );
+            eprintln!("index[192..200] = {:?}", &index[192..200]);
+        }
+        let moby_v = lc::class_table(
+            &index,
+            h.moby_classes,
+            lc::MOBY_CLASS_SIZE,
+            lc::ClassEntry::parse,
+        )
+        .expect("moby classes");
+        let tie_v = lc::class_table(
+            &index,
+            h.tie_classes,
+            lc::TIE_CLASS_SIZE,
+            lc::ClassEntry::parse,
+        )
+        .expect("tie classes");
+        let shrub_v = lc::class_table(
+            &index,
+            h.shrub_classes,
+            lc::SHRUB_CLASS_SIZE,
+            lc::ShrubClassEntry::parse,
+        )
+        .expect("shrub classes");
+        let tex_v = lc::texture_table(&index, h.tfrag_textures).expect("tfrag textures");
+
+        assert_eq!(moby_v.len(), h.moby_classes.count as usize);
+        for (i, c) in moby_v.iter().chain(tie_v.iter()).enumerate() {
+            assert!(
+                c.offset_in_asset_wad >= 0,
+                "level{n} entry {i}: asset offset {}",
+                c.offset_in_asset_wad
+            );
+        }
+        // Billboard slots are optional: every shrub class on level 0 carries
+        // them all zero, so only their sign is meaningful here.
+        for s in &shrub_v {
+            assert!(s.base.offset_in_asset_wad >= 0);
+            assert!(
+                s.palette_offset >= -1 && s.texture_offset >= -1,
+                "level{n}: shrub billboard offsets below the -1 sentinel"
+            );
+        }
+        moby += moby_v.len();
+        tie += tie_v.len();
+        shrub += shrub_v.len();
+        textures += tex_v.len();
+        levels += 1;
+    }
+
+    assert_eq!(levels, 27, "expected all 27 levels, got {levels}");
+    assert!(
+        moby + tie + shrub > 1000,
+        "only {moby}+{tie}+{shrub} class entries"
+    );
+    eprintln!(
+        "level core index: {moby} moby, {tie} tie, {shrub} shrub classes, \
+         {textures} tfrag textures across {levels} levels"
+    );
+}

crates/gc-formats/src/level_core.rs

@@ -21,6 +21,76 @@ pub struct ArrayRange {
     pub offset: i32,
 }
 
+/// One entry in the moby, tie or shrub class table (32 bytes).
+///
+/// These name the environment geometry sitting on top of the terrain -- the
+/// props, foliage and actors. `offset_in_asset_wad` points into the level's
+/// asset WAD, not into this index.
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct ClassEntry {
+    pub offset_in_asset_wad: i32,
+    pub o_class: i32,
+    pub unknown_8: i32,
+    pub unknown_c: i32,
+    pub textures: [u8; 16],
+}
+
+impl ClassEntry {
+    pub fn parse(bytes: &[u8]) -> Result<ClassEntry> {
+        let mut b = Buf::new(bytes);
+        // Order matters: these reads advance the cursor, so the four words have
+        // to come before the trailing texture bytes, not after.
+        let offset_in_asset_wad = b.i32("class asset offset")?;
+        let o_class = b.i32("class number")?;
+        let unknown_8 = b.i32("class unknown_8")?;
+        let unknown_c = b.i32("class unknown_c")?;
+        let mut textures = [0u8; 16];
+        for slot in textures.iter_mut() {
+            *slot = b.u8("class textures")?;
+        }
+        Ok(ClassEntry {
+            offset_in_asset_wad,
+            o_class,
+            unknown_8,
+            unknown_c,
+            textures,
+        })
+    }
+}
+
+/// One shrub class entry: a class entry plus the billboard's texture slots (48
+/// bytes).
+#[derive(Debug, Clone, Copy, PartialEq, Eq)]
+pub struct ShrubClassEntry {
+    pub base: ClassEntry,
+    pub billboard_width: i16,
+    pub billboard_height: i16,
+    pub max_mip: i16,
+    pub palette_offset: i16,
+    pub texture_offset: i16,
+    pub mip1: i16,
+    pub mip2: i16,
+    pub mip3: i16,
+}
+
+impl ShrubClassEntry {
+    pub fn parse(bytes: &[u8]) -> Result<ShrubClassEntry> {
+        let base = ClassEntry::parse(bytes)?;
+        let mut b = Buf::new(&bytes[32..]);
+        Ok(ShrubClassEntry {
+            base,
+            billboard_width: b.i16("shrub billboard width")?,
+            billboard_height: b.i16("shrub billboard height")?,
+            max_mip: b.i16("shrub max mip")?,
+            palette_offset: b.i16("shrub palette offset")?,
+            texture_offset: b.i16("shrub texture offset")?,
+            mip1: b.i16("shrub mip1")?,
+            mip2: b.i16("shrub mip2")?,
+            mip3: b.i16("shrub mip3")?,
+        })
+    }
+}
+
 /// One texture in the level's texture table (16 bytes).
 #[derive(Debug, Clone, Copy, PartialEq, Eq)]
 pub struct TextureEntry {
@@ -66,6 +136,48 @@ impl TextureEntry {
     }
 }
 
+/// Bytes per class-table entry, by table.
+///
+/// These sizes are not stated anywhere in the reference; they are fixed by the
+/// fact that each table ends exactly where the next one begins. That chain
+/// closes on all 27 levels, which is the only thing that makes it trustworthy.
+pub const MOBY_CLASS_SIZE: usize = 32;
+pub const TIE_CLASS_SIZE: usize = 32;
+pub const SHRUB_CLASS_SIZE: usize = 48;
+
+/// Parse a class table.
+pub fn class_table<T, F>(index: &[u8], range: ArrayRange, size: usize, parse: F) -> Result<Vec<T>>
+where
+    F: Fn(&[u8]) -> Result<T>,
+{
+    let count = range.count.max(0) as usize;
+    let offset = range.offset.max(0) as usize;
+    let need = count
+        .checked_mul(size)
+        .and_then(|n| offset.checked_add(n))
+        .ok_or(Error {
+            offset,
+            context: "class table range overflows",
+            kind: ErrorKind::InvalidLength("class table"),
+        })?;
+    if need > index.len() {
+        return Err(Error {
+            offset,
+            context: "class table runs past the index",
+            kind: ErrorKind::OutOfBounds {
+                needed: need,
+                available: index.len(),
+            },
+        });
+    }
+    let mut out = Vec::with_capacity(count);
+    for i in 0..count {
+        let at = offset + i * size;
+        out.push(parse(&index[at..at + size])?);
+    }
+    Ok(out)
+}
+
 /// Parse a texture table.
 pub fn texture_table(index: &[u8], range: ArrayRange) -> Result<Vec<TextureEntry>> {
     let count = range.count.max(0) as usize;
@@ -114,8 +226,15 @@ pub struct LevelCoreHeader {
     pub sky: i32,
     /// Byte offset of the collision block.
     pub collision: i32,
+    /// Class tables: the environment geometry on top of the terrain.
+    pub moby_classes: ArrayRange,
+    pub tie_classes: ArrayRange,
+    pub shrub_classes: ArrayRange,
     /// The table tfrag ad-gifs index into.
     pub tfrag_textures: ArrayRange,
+    pub moby_textures: ArrayRange,
+    pub tie_textures: ArrayRange,
+    pub shrub_textures: ArrayRange,
     /// Byte offset of texture pixel data within `core_data`.
     pub textures_base: i32,
 }
@@ -133,10 +252,15 @@ impl LevelCoreHeader {
         let occlusion = b.i32("occlusion")?;
         let sky = b.i32("sky")?;
         let collision = b.i32("collision")?;
-        // 0x18..0x30: moby/tie/shrub/gadget class tables, not decoded yet.
-        b.skip(0x30 - 0x18, "class tables")?;
+        let moby_classes = pair(&mut b, "moby classes")?;
+        let tie_classes = pair(&mut b, "tie classes")?;
+        let shrub_classes = pair(&mut b, "shrub classes")?;
         let tfrag_textures = pair(&mut b, "tfrag textures")?;
-        b.skip(0x60 - 0x38, "texture tables")?;
+        let moby_textures = pair(&mut b, "moby textures")?;
+        let tie_textures = pair(&mut b, "tie textures")?;
+        let shrub_textures = pair(&mut b, "shrub textures")?;
+        // 0x50..0x60: particle and effect texture tables, then the banks.
+        b.skip(0x60 - 0x50, "particle texture tables")?;
         let textures_base = b.i32("textures base")?;
         Ok(LevelCoreHeader {
             gs_ram,
@@ -144,7 +268,13 @@ impl LevelCoreHeader {
             occlusion,
             sky,
             collision,
+            moby_classes,
+            tie_classes,
+            shrub_classes,
             tfrag_textures,
+            moby_textures,
+            tie_textures,
+            shrub_textures,
             textures_base,
         })
     }