Identify the container the class tables point into: core_data itself

assets_compressed_size and assets_decompressed_size at index offsets
0x88 and 0x8c are 9,738,678 and 16,969,296 on level 0 -- exactly the
sizes of core_data before and after decompression. There is no separate
asset container. The "asset WAD" is core_data, and offset_in_asset_wad
indexes the decompressed form directly.

Checked rather than assumed, on all 27 levels: the decompressed size
equals core_data's exact length, every class offset is either 0 or inside
it, and the moby, tie and shrub spans do not overlap each other.

An offset of 0 means the slot has no geometry, not the start of the
lump; level 0's ninth moby class is 4909 at offset 0. Two assumptions
worth discarding while here -- the tables are not sorted by class number
(level 0's run 0, 1, 2, 14 ... 4909) and the offsets within a table are
not monotonic. Neither is needed, but both look true from the first few
entries.

The chunk descriptors after the named extents are not it: they exist on
only 3 of 27 levels, because level 0's four named extents tile the
container exactly. An asset blob there would be missing almost
everywhere.

What sits at the offsets is raw GS/VIF packet data rather than a further
container -- moby[0] begins 00 00 00 00 3f 16 02 55 6f 6f and tie[0]
00 01 00 00 70 50 00 00, which read as packet headers rather than a
magic number.

None of the three is decoded yet and this does not start that. Moby is
skeletal and is the largest single piece left before the rest of a level
can draw.

Also: the pipeline now writes each level's chunk descriptors where they
exist, and LevelCoreHeader decodes the asset lump's two sizes.

crates/gc-extract/data/gc-files.tsv

@@ -2496,6 +2496,9 @@ e1baca5f8178da95e7dc496305a36f984be542aa	15	hud/0780.bin
 acef5e387c8051229eecf70a64fd6daa077b34b3	518656	level0/gs_ram.bin
 d7045d6c1256a9ffa241acca76e3761c34da3341	2751268	level0/overlay.bin
 7ec9566aeae513fe272bc97899a4dbe15a2a74ff	1503232	level0/sound_bank.bin
+6d98a8d7b65d6c743c23c27863c9c7dbf3fc4493	3110912	level1/chunk0.bin
+e01bcf80d072da3298d854fb095f13cd6834fca6	237568	level1/chunk1.bin
+62b48a3295f5d0cb8094a63ec77c2f514b4c5077	321536	level1/chunk3.bin
 6b1a358c6eb6039d0df51fce061b58c274e8c99c	19023392	level1/core_data.bin
 96b33cef1ac3c9bab5283ccf11bd86b0aa20ebcb	12165629	level1/core_data.wad
 7cae5c7c2258f391ef72a8503183e36747214fbd	34016	level1/core_index.bin
@@ -2512,6 +2515,9 @@ fe9e5169820bc9272aa5fcfbbabfc74e7b12d1ce	12498944	level10/data.bin
 4036bc105fa429b97a725ad40bd4051168c886ad	367872	level10/gs_ram.bin
 76fa348a8764cbe97c8caf5a892105007c2bf4a2	2870708	level10/overlay.bin
 d3e48237b03649578f6962e36f1bc197c3a8af9f	1306624	level10/sound_bank.bin
+74ef643e332327bc142155c5570be38ecfe18d10	1988608	level11/chunk0.bin
+62eab219f89f2c1374e29dd302e8f86f69882e6e	1521664	level11/chunk1.bin
+ab08381b884f6585818780791a27fd8f44366e76	704512	level11/chunk3.bin
 da253d8b4dc4f8fc482880c54cd832c53bfd4aa0	20717584	level11/core_data.bin
 a0173c7577180d32e9bfed67464eecf3bc12c033	11414905	level11/core_data.wad
 0e0d5289451a3bffa376992d0b6a288421a7a55d	36000	level11/core_index.bin
@@ -2577,6 +2583,11 @@ d0f931677cb6589a564937c3d15c13344b923969	15501312	level18/data.bin
 7f6a823c4b47fdfb58e29237b680d03e946329db	571136	level18/gs_ram.bin
 e5370c97b9a2ccdb00678e57f9e18042eab6735d	2937576	level18/overlay.bin
 b26483cc29ecb8b6d2c71919ff7cd56b09522bc1	1492992	level18/sound_bank.bin
+11dbeeb1847ff09621d132adb9ab0a149fc65a5a	3252224	level19/chunk0.bin
+0e05525617f49895b26da1521b8d2ceb9f0c52a2	2742272	level19/chunk1.bin
+f9110d5cb18a0fbd45a760dc4d926a87db099b7c	1017856	level19/chunk2.bin
+c739bdb3ea348b169b63114f2b13149e647070e0	92160	level19/chunk3.bin
+77cbf1d1847c0846c13a23983e664f9546feddf8	92160	level19/chunk4.bin
 0918d824b72b892a24f46108bbc60f6e875236f6	20233504	level19/core_data.bin
 b79df8e074db7d3c75bc2b3b9c2a4de61e5dd9cd	11797811	level19/core_data.wad
 b0ea377a39a616dbd2b15224a9996925c7b28f4d	31536	level19/core_index.bin
@@ -2586,6 +2597,9 @@ d5061a551eb354ddf7952c4bfd62767c6e074c3f	497920	level19/gs_ram.bin
 877ca7fb8e80d810d6c37fb993795aa10b5b1033	47104	level19/occlusion.bin
 428ea51a157b35cdf831f2a6908647d8ffe6a91b	2821856	level19/overlay.bin
 8298f799c394542e912e41ff3321ee55c70e0c8f	1570816	level19/sound_bank.bin
+e6c6de0fafb808093d6e3f2282e86b3fc623d7cb	2486272	level2/chunk0.bin
+84bb6b19112861026a24d84cbf83946e2bf31892	321536	level2/chunk1.bin
+6cb42e4727ce5a55cf6e5a9147a32149a2d15891	438272	level2/chunk3.bin
 6d56189ad74f04d366116b736ab12888154de833	20209408	level2/core_data.bin
 f0eb5c033e84092748cf3457818b0d4c85da7a0a	11372886	level2/core_data.wad
 73411362c1cfa8faec733705d93e953f1381ef30	33808	level2/core_index.bin
@@ -2595,6 +2609,11 @@ f0eb5c033e84092748cf3457818b0d4c85da7a0a	11372886	level2/core_data.wad
 01c7d4c7b0ecb44a91694ffde796ff941057191a	28672	level2/occlusion.bin
 d7182afdf4a986c9c781a9658373281bfc37df22	2928424	level2/overlay.bin
 bce56b3b899d72c5d89f612bd70447f066c10757	1118208	level2/sound_bank.bin
+015dfaf782ade5bdd3815a66b4c381a93ec7cc6d	1462272	level20/chunk0.bin
+93481eb3afd7edfe679c878a74d5ec627196cf86	1878016	level20/chunk1.bin
+1867fb80ce6d3b592c687e712e9782cd5a43dd6f	565248	level20/chunk2.bin
+b81d361a7c7136d92990d720f6f22fbbee9b0d89	342016	level20/chunk3.bin
+1baaf377e209e353b2fbb80d408bbc52a67f6b5f	342016	level20/chunk4.bin
 daf162b58756e2661f3ec891c8736a1611b4dc52	19637472	level20/core_data.bin
 7e7203ac3a2dbf578546573a34fa6dce4c336d35	10449039	level20/core_data.wad
 b12e8599d211addc85cbe138e58157ade380a575	37360	level20/core_index.bin
@@ -2658,6 +2677,9 @@ eb8366046b5643cc54d42d800c06bf66ccd4ef36	15157248	level3/data.bin
 ed1f77cd35e2572adc51f4f43de030f31b8c6ed9	675840	level3/gs_ram.bin
 6fb6a7e7a64afd2f1e897f23640866f28154aa1d	2820776	level3/overlay.bin
 d986825dfc2bed713eb9699504536346658aaa3f	1705984	level3/sound_bank.bin
+2778532b95fe6a545004f5a922977cac656519e7	3436544	level4/chunk0.bin
+59aead3118f5be41ab3ab545e2476e7fed6842f1	2426880	level4/chunk1.bin
+482ac30a9958e484708f9b82e2784a62a0cf753e	303104	level4/chunk3.bin
 f8c0aa3778b5bcb1995e516045b28ca6ac35502f	21107088	level4/core_data.bin
 ec75f324670188af581c4fceaa4cb285353f95b8	12171908	level4/core_data.wad
 cc4997acbdc0a6ac85d427227c04e6af6e34cd46	38080	level4/core_index.bin
@@ -2683,6 +2705,7 @@ d5874f71309fb750d37a08977653a4fe1b48672a	2617984	level6/gameplay.bin
 c404dfa08cab9603e5deeeed9778e4d29b755d88	604160	level6/gs_ram.bin
 4d981fd750196894f96f0a24437bc76210a1663a	3086372	level6/overlay.bin
 82fcce964396eb508768933d092a557bc1b05c89	1669120	level6/sound_bank.bin
+ef8e0a3a46dcb8f29595332ff9545b654130b194	3280896	level7/chunk0.bin
 6692290d78706a2b92cba541264ddea9a764e2f2	18209392	level7/core_data.bin
 f1cae801f1050c57258f93a7035148b3094ece03	10780010	level7/core_data.wad
 79e3957d149ce3b00f713eb1757fec89d7db86c4	32736	level7/core_index.bin
@@ -2692,6 +2715,7 @@ cc78412e868725caa8e9736653749e4b7eeacba4	2147328	level7/gameplay.bin
 e02d1b0d2339d72ed5d07abfe105f41e2288f59f	16384	level7/occlusion.bin
 82f26617c99f502324fb556006f8f859fda1b5f1	2733840	level7/overlay.bin
 8e07c803b709100023c3b44c8842eec9b8f49898	1556480	level7/sound_bank.bin
+78e0c64ccb11e29c9bdbfb9c9283a754867d6b5c	868352	level8/chunk0.bin
 264a306c3a83b9137d9753b282e9c773cd386713	21086224	level8/core_data.bin
 0a96094edd63dc4dd940894b007f85b21874cfeb	9700660	level8/core_data.wad
 8d1b6c3a05e9b2140c056df3e538ed537b9d3a33	30624	level8/core_index.bin

crates/gc-extract/src/unpack.rs

@@ -137,6 +137,26 @@ pub fn unpack(iso_path: &Path, out_dir: &Path) -> Result<Vec<Extracted>, String>
                         }
                         parts += 1;
                     }
+                    // Chunk descriptors follow the named extents. One of them
+                    // is the level's asset WAD, which the moby/tie/shrub class
+                    // tables point into by byte offset.
+                    for (i, range) in header.present_chunks() {
+                        let Some(offset) = range.byte_offset() else {
+                            continue;
+                        };
+                        let len = range.byte_len() as usize;
+                        if offset + len as u64 > raw.len() as u64 {
+                            continue;
+                        }
+                        write_file(
+                            out_dir,
+                            &format!("{stem}/chunk{i}.bin"),
+                            &raw[offset as usize..offset as usize + len],
+                            &mut written,
+                        );
+                        parts += 1;
+                    }
+
                     // The `data` extent is itself a container of byte ranges.
                     // Split out the geometry directory and the geometry, so the
                     // latter is usable without re-deriving the layout later.

crates/gc-extract/tests/disc.rs

@@ -1092,3 +1092,110 @@ fn level_core_class_tables_tile_the_index() {
          {textures} tfrag textures across {levels} levels"
     );
 }
+
+/// The moby, tie and shrub class tables point into `core_data`, and they do so
+/// without overlapping.
+///
+/// `assets_decompressed_size` is the size of `core_data` itself, so there is
+/// no separate asset container: the class entries index the decompressed asset
+/// lump directly. If that is right, every offset must land inside the lump, the
+/// three tables must occupy disjoint spans, and each must be internally sorted
+/// so the geometry is laid out in class order.
+#[test]
+fn class_geometry_offsets_tile_core_data_without_overlap() {
+    require_disc!();
+    gc_formats::iso9660::IsoImage::open(std::path::Path::new(
+        &std::env::var("GC_DISC").expect("GC_DISC is set by require_disc!"),
+    ))
+    .expect("ISO should open");
+    let data = std::env::var_os("GC_EXTRACTED")
+        .map(PathBuf::from)
+        .unwrap_or_else(|| {
+            let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
+            PathBuf::from(home).join("Documents/gc-extracted")
+        });
+    if !data.exists() {
+        eprintln!("skipped: unpack the disc first");
+        return;
+    }
+
+    use gc_formats::level_core as lc;
+    let mut levels = 0usize;
+
+    for n in 0..27 {
+        let dir = data.join(format!("level{n}"));
+        let (Ok(index), Ok(core)) = (
+            std::fs::read(dir.join("core_index.bin")),
+            std::fs::read(dir.join("core_data.bin")),
+        ) else {
+            continue;
+        };
+        let h = lc::LevelCoreHeader::parse(&index).expect("core header");
+
+        // The claim under test: the "asset lump" is core_data itself.
+        assert_eq!(
+            h.assets_decompressed_size as usize,
+            core.len(),
+            "level{n}: assets_decompressed_size is not core_data's length"
+        );
+
+        let span = |r: lc::ArrayRange, size: usize| -> Vec<usize> {
+            lc::class_table(&index, r, size, lc::ClassEntry::parse)
+                .unwrap_or_default()
+                .iter()
+                .map(|c| c.offset_in_asset_wad as usize)
+                .collect()
+        };
+        // Shrub entries are read as ShrubClassEntry so the offsets line up.
+        let shrub: Vec<usize> = lc::class_table(
+            &index,
+            h.shrub_classes,
+            lc::SHRUB_CLASS_SIZE,
+            lc::ShrubClassEntry::parse,
+        )
+        .unwrap_or_default()
+        .iter()
+        .map(|s| s.base.offset_in_asset_wad as usize)
+        .collect();
+        let moby = span(h.moby_classes, lc::MOBY_CLASS_SIZE);
+        let tie = span(h.tie_classes, lc::TIE_CLASS_SIZE);
+
+        // An offset of 0 marks a slot with no geometry rather than the start of
+        // the lump, so those are excluded from the span checks. Entries are not
+        // sorted by class number and the offsets are not monotonic, so nothing
+        // here assumes an order.
+        let moby_l: Vec<usize> = moby.iter().copied().filter(|&o| o != 0).collect();
+        let tie_l: Vec<usize> = tie.iter().copied().filter(|&o| o != 0).collect();
+        let shrub_l: Vec<usize> = shrub.iter().copied().filter(|&o| o != 0).collect();
+
+        for (name, v) in [("moby", &moby), ("tie", &tie), ("shrub", &shrub)] {
+            for &o in v {
+                assert!(
+                    o < core.len(),
+                    "level{n}: {name} offset {o} past core data ({})",
+                    core.len()
+                );
+            }
+        }
+        // The three spans must not overlap each other.
+        for (an, a) in [("moby", &moby_l), ("tie", &tie_l), ("shrub", &shrub_l)] {
+            for (bn, b) in [("moby", &moby_l), ("tie", &tie_l), ("shrub", &shrub_l)] {
+                // A level may have no geometry of one kind at all.
+                if an >= bn || a.is_empty() || b.is_empty() {
+                    continue;
+                }
+                let (lo, hi) = (a[0], *a.last().unwrap());
+                for &o in b {
+                    assert!(
+                        o >= hi || o < lo,
+                        "level{n}: {an} [{lo},{hi}] overlaps {bn} at {o}"
+                    );
+                }
+            }
+        }
+        levels += 1;
+    }
+
+    assert_eq!(levels, 27, "expected all 27 levels, got {levels}");
+    eprintln!("class geometry offsets verified inside core_data on {levels} levels");
+}

crates/gc-formats/src/level_core.rs

@@ -237,6 +237,14 @@ pub struct LevelCoreHeader {
     pub shrub_textures: ArrayRange,
     /// Byte offset of texture pixel data within `core_data`.
     pub textures_base: i32,
+    /// Compressed size of `core_data`'s WAD stream, and its decompressed size.
+    ///
+    /// These describe the whole asset lump, not a separate container: on level
+    /// 0 they are 9,738,678 and 16,969,296, exactly the sizes of `core_data`
+    /// before and after decompression. The moby, tie and shrub class tables
+    /// name geometry by byte offset into the *decompressed* form.
+    pub assets_compressed_size: i32,
+    pub assets_decompressed_size: i32,
 }
 
 impl LevelCoreHeader {
@@ -262,6 +270,13 @@ impl LevelCoreHeader {
         // 0x50..0x60: particle and effect texture tables, then the banks.
         b.skip(0x60 - 0x50, "particle texture tables")?;
         let textures_base = b.i32("textures base")?;
+        // 0x60..0x80: particle/effect banks and defs, sound remap, then the
+        // ratchet sequence table and the scene view size.
+        b.skip(0x80 - 0x64, "particle banks")?;
+        // 0x80: gadget table, then the asset lump's two sizes.
+        b.skip(0x88 - 0x80, "gadget table")?;
+        let assets_compressed_size = b.i32("assets compressed size")?;
+        let assets_decompressed_size = b.i32("assets decompressed size")?;
         Ok(LevelCoreHeader {
             gs_ram,
             tfrags,
@@ -276,6 +291,8 @@ impl LevelCoreHeader {
             tie_textures,
             shrub_textures,
             textures_base,
+            assets_compressed_size,
+            assets_decompressed_size,
         })
     }
 }
@@ -286,7 +303,7 @@ mod tests {
 
     /// A header as measured on level 0.
     fn header() -> Vec<u8> {
-        let mut v = vec![0u8; 0x64];
+        let mut v = vec![0u8; 0x90];
         v[0x00..0x04].copy_from_slice(&404i32.to_le_bytes());
         v[0x04..0x08].copy_from_slice(&18_704i32.to_le_bytes());
         v[0x08..0x0c].copy_from_slice(&0i32.to_le_bytes());
@@ -296,6 +313,8 @@ mod tests {
         v[0x30..0x34].copy_from_slice(&93i32.to_le_bytes());
         v[0x34..0x38].copy_from_slice(&10_512i32.to_le_bytes());
         v[0x60..0x64].copy_from_slice(&3_518_848i32.to_le_bytes());
+        v[0x88..0x8c].copy_from_slice(&9_738_678i32.to_le_bytes());
+        v[0x8c..0x90].copy_from_slice(&16_969_296i32.to_le_bytes());
         v
     }
 
@@ -321,11 +340,14 @@ mod tests {
             }
         );
         assert_eq!(h.textures_base, 3_518_848);
+        assert_eq!(h.assets_compressed_size, 9_738_678);
+        assert_eq!(h.assets_decompressed_size, 16_969_296);
     }
 
     #[test]
     fn a_short_header_is_an_error() {
         assert!(LevelCoreHeader::parse(&[0u8; 0x20]).is_err());
+        assert!(LevelCoreHeader::parse(&[0u8; 0x8c]).is_err());
     }
 
     #[test]

docs/plan/level-assets.md

@@ -562,6 +562,37 @@ tfrag record framing was "unrecoverable" because the entry size was never
 known. Both looked like properties of the format and were properties of the
 question.
 
+### Where the class tables point
+
+`assets_compressed_size` and `assets_decompressed_size` at index offsets 0x88
+and 0x8c are 9,738,678 and 16,969,296 on level 0 -- exactly the sizes of
+`core_data` before and after decompression. There is no separate asset
+container: the "asset WAD" *is* `core_data`, and `offset_in_asset_wad` indexes
+the decompressed form directly.
+
+That is checkable rather than merely plausible, and a test checks it on all 27
+levels:
+
+* `assets_decompressed_size` equals `core_data`'s exact length;
+* every class offset is either 0 or inside `core_data`;
+* the moby, tie and shrub spans do not overlap each other.
+
+An offset of **0 means the slot has no geometry**, not the start of the lump.
+On level 0 the moby table's ninth entry is class 4909 at offset 0.
+
+Two assumptions worth discarding: the tables are **not** sorted by class number
+(level 0's moby classes run 0, 1, 2, 14, ... 4909), and the offsets within a
+table are **not** monotonic. Neither is needed, but both look true from the
+first few entries.
+
+The chunk descriptors after the named extents are not it. They exist on only
+3 of 27 levels -- level 0's four named extents tile the container exactly --
+so an asset blob there would be missing almost everywhere.
+
+At the offsets sit raw GS/VIF packet data, not a further container: moby[0]
+begins `00 00 00 00 3f 16 02 55 6f 6f`, tie[0] `00 01 00 00 70 50 00 00`,
+which read as packet headers rather than a magic number.
+
 ### Why the frame is still dark: the light bank does not parse on RAC2
 
 The reference frames the bank as "a u32 offset at +0x04, then an `i32` count
@@ -659,9 +690,11 @@ the fidelity rule.
 - **Lighting magnitudes.** The pass runs and the output is plausible; whether
   it matches the game needs comparison against reference frames, which needs
   the PCSX2 loop that is still to come. Nothing in this file establishes it.
-- **tie, moby and shrub geometry.** The class tables now parse (below), but
-  nothing reads the geometry they name yet. `offset_in_asset_wad` points into
-  the level's asset WAD, which is a separate container.
+- **tie, moby and shrub geometry.** The class tables parse and the container
+  they point into is identified (below), but none of the three is decoded yet.
+  Moby is the largest of the three and is skeletal, with an animation evaluator
+  in the microcode; it is the single biggest piece left before the rest of a
+  level can draw.
 - **Point lights** are spawned by gameplay at runtime and are absent from the
   disc -- every one of the 18,102 patches has `point_lights == 0xffff` -- so
   there is nothing for them to light until the game runs.

docs/roadmap.md

@@ -14,8 +14,9 @@ early, while they are still cheap to retire.
 - [x] Repository hygiene in place (pre-commit guard, `.gitignore`, licence
       record)
 - [x] **Phase 1 complete.** The disc pipeline runs end to end and is verified
-      against a committed hash table: 5,213 files, 5,795,847,447 bytes, every
-      one of the 92 asset containers parsing.
+      against a committed hash table, every one of the 92 asset containers
+      parsing. The pipeline now also writes each level's chunk descriptors
+      where they exist (3 of 27 levels).
 - [x] **Phase 2 in progress.** Level containers are parsed through their typed
       header, and the `data` extent is parsed as a container of byte ranges. The
       geometry decompresses: level 0 yields 16,969,296 bytes of level content