Going-Commando/docs/plan/level-assets.md

# Level assets

Phase 2 work record. What has been established about the four extents inside
each `LEVEL<n>.WAD`, and what remains.

## A bug found and fixed

Reading the `data` extent's contents exposed a defect in the Phase 1 extractor.

Level extents are declared in **sector counts**; the generic `G/` pair table
uses **byte counts**. The extractor routed level containers through the generic
path, so `LEVEL0.WAD`'s `data` extent — 6,484 sectors — was taken as 6,484
*bytes*. Level 0 yielded a 6,484-byte "geometry" file instead of 13,279,232
bytes. Across 27 levels that silently discarded roughly 300 MB of level content
while every test still passed.

The tell was arithmetic, not a crash: the extent's own header declared an
uncompressed total of 13,277,920 bytes, and the ratio against the 6,484-byte
file was 2047.7× — one sector, to within rounding. A ratio that lands on 2048
is a unit error.

Fixed by extracting level containers through the typed `LevelHeader` instead of
the generic table. Level 0 now yields:

```text
data.bin         13,279,232   the level container's data extent
core_data.wad     9,738,678   the geometry, compressed
core_data.bin    16,969,296   the geometry, decompressed
core_index.bin       31,600   the geometry directory
gameplay.bin       659,456
sound_bank.bin   1,503,232
```

**Lesson:** a unit error is invisible to a type system and to a passing test. The
check that catches it is arithmetic against an independent number the file
itself declares. Any extent whose contents describe a larger structure should
have that structure's own total compared against the extent.

## Done: the container header

`crates/gc-formats/src/level_wad.rs` parses the 0x60-byte header on all 27
levels and names the four extents. See
[`../formats/level-wad.md`](../formats/level-wad.md) for the layout and the
measured invariants.

Two facts worth carrying forward:

- **Sizes in this header are sector counts**, unlike the generic `G/*.WAD` pair
  table where they are byte counts. Both are on the same disc.
- **`LEVEL21.WAD` reports level id 30.** Container order is not level identity.

## `gameplay`: solved, and the old answer was wrong

**It is a WAD LZ77 stream.** It decompresses with the decoder in `wad.rs` like
`core_data` does, and the decompressed extent holds the level's light bank.

This section used to be the longest open question here, and it was wrong. The
reasoning was sound and the conclusion was not: byte 3 of the magic is `0x79`
on level 0 rather than the `0x0d` the WAD docs suggest, which reads as a
per-level variation; but byte 3 is part of the 32-bit little-endian stream
length, and `0x0a0d79` is simply level 0's compressed length. An earlier
attempt decompressed "starting after an 8-byte header" and read the resulting
garbage as evidence of a different format.

The failure mode is the one worth keeping: a uniform failure on all 27 inputs
was taken to mean the decoder was wrong, when it meant the *data* was being
handed over uncompressed. A positive control -- decompress it and look at the
result -- was never run, and it would have answered the question in one line.

See "The light bank was there the whole time" below.

## `sound_bank`: signatures found, structure unknown

```text
+0x18  "SBlk"   present in all 27 levels
+0x24  "DAW"    present in all 27 levels
```

Sizes 458..833 bytes. Too small to be the level's audio; likely an index into
the global `AUDIO.WAD`.

## `occlusion`: header located, contents a u16 array

The header's `occlusion` offset (0x0c) is trustworthy — every offset in it lands
inside `core_data`. Level 0's occlusion block starts at 2,117,888 and reads as a
`u16` array:

```text
5152 0 12 23 14 19 24 29 32 35 38 41 44 47 50 53 56 59 65 82 96 100 113 126 143 164 183 ...
```

An ascending run of 65 values with mostly small differences, then it restarts.
That is consistent with per-cell visibility records rather than geometry. The
meaning of the pairs — and the 0 that ends each group — is not yet established.

## `sky`: plausible, unparsed

Header offset 0x10 = 2,192,576. Opens `0 0 0 0 0 0 6 0 0 0 0 0 5 0 0 0 40 0 0 0 c0 0 0 0 40 ...`
— small values including 64 and 192, which look like sky-shell counts and
dimensions rather than offsets.

## `tfrag` region: `[0, occlusion)`

The header's fields at 0x0c, 0x10, 0x14 and 0x60 are **offsets**, and they are
strictly increasing on all 27 levels. The differences between them give region
sizes that are plausible on every level:

| Region | Range | Level 0 size |
|---|---|---|
| tfrag | `[0, occlusion)` | 2.12 MB |
| occlusion | `[occlusion, sky)` | 0.07 MB |
| sky | `[sky, collision)` | 0.42 MB |
| collision | `[collision, textures_base)` | 0.90 MB |

Ruled out: reading 0x0c/0x10/0x14 as *sizes* laid end to end. The cumulative
total does not reach `textures_base` on any level, and the byte at the start of
each supposed region is not a plausible structure. The offsets model is the one
that holds everywhere.

### What is in the terrain region

**Textures are stored inline with the geometry**, not in a separate bank. A scan
for the PS2 texel pattern `18 18 18 80` (grey texels with the 0x80 alpha that
means "opaque" in the GS's convention) finds 6,739 occurrences spread across 138
clusters, the first at `0xd9f8` — inside the tfrag region. So terrain carries its
own texels, and `textures_base` names something else.

Earlier claims in this file that have since been **disproven**:

- ~~"The terrain is 382 records of 64 bytes."~~ The 0x48 byte at offset +3 is
  simply the most common byte in the region (945 of 2000 sampled slots); it is
  not a record marker. Over 33,091 slots at that position only 4,127 hold 0x48.
- ~~"Byte +3 is constant at 0x48 across 300 records."~~ True for the first 382
  records and false beyond, which is how the false pattern was found.
- ~~"Coordinates are fixed-point s16 with a scale of 8."~~ The scale-8 reading
  is supported by one coincidence (17,496 / 8 = 2,187, the planet radius) but no
  grouping of the record into triples yields points on a sphere of that radius.
  Tested: every stride from 2 to 16 bytes, every starting offset, three and four
  components. Best result 5.0% of points on-sphere, which is chance.

So the record framing is **not recovered** — but that framing was a false lead,
and the terrain has since been parsed in full. This section records the wrong
turn and is kept only so it is not repeated.

The region layout below is right and terrain really is packed alongside inline
texels, but the "64-byte record" idea was the wrong way in. Tfrags are not a
flat vertex table: they are VIF1 command lists whose unpacks are classified by
format *and* destination VU address, and the vertices come out of a V3_16
unpack spread across three lists per patch. See "tfrag terrain" below for the
framing that works and the four wrong turns along the way.

The lesson is the recurring one in this file: a hypothesis that produces
plausible-looking output is not thereby recovered, and a "ruled out" entry is
only as good as the control it was tested against.

## Where the class tables are not

The header's `moby_classes` and friends do not decode. Read as
`(offset, count)` they give plausible numbers, but the entries at those offsets
are not valid records. Ruled out:

- **Field order.** Both `(offset, count)` and `(count, offset)` produce garbage.
- **Stride.** Tried 8, 16 and 32 bytes per entry.
- **Which file the offsets index.** Tried `core_data` and `core_index`.
- **Header base.** Not the issue: the compressed size (0x88) and decompressed
  size (0x8c) of `core_data` appear at the *same offsets* in all 27 levels'
  `core_index`, which pins the header base at 0 with certainty.

One observation worth keeping: every second value in the class-table fields is
exactly divisible by 16, and they increase monotonically across the nine tables.
That is consistent with `count * 16` sizes, but the corresponding offsets
cluster at the very start of the file and do not yield valid records either.

The discriminator used to reject candidates is worth stating, because weaker
tests produce false positives: a real class table's `offset_in_asset_wad`
values increase. A search for "32-byte records with increasing offsets" returns
several hundred candidates across the file, and the top one turned out to be a
texture table whose `o_class` field merely happened to equal `offset + 16`.
Scoring on plausibility alone is not enough; the entries have to actually chain.

Next: rather than guess the header further, identify `tfrag` by its own
structure. Terrain is the largest subsystem and the one that decides whether
anything draws, so it is the right next target even though it is the harder one.

*(Done — see "tfrag terrain" below. Identifying tfrag by its own structure
worked where the class-table guess did not.)*

## `data`: the real target, untouched

6,484 sectors on level 0, down to 2,102 on level 24 — 4.2 to 21.9 MB. This is
the level's geometry and the largest thing in the container. Next target.

What is known from the community reference and from Insomniac's engine family:
it splits into four geometry subsystems — `tfrag` (terrain fragments),
`tie` (instanced environment structures), `shrub` (foliage) and `moby` (dynamic
actors) — plus sky shells, collision data and textures. ReRAC's documentation of
the equivalent structures in the first game is the closest reference available,
though R&C2's spherical worlds change how they are used.

## Order of work

1. `gameplay` — smallest, fully bounded, and the format is identified enough to
   make progress. A win here validates the approach before the large effort.
2. `occlusion` — tiny.
3. `sound_bank` — small.
4. `data` — the large one, and the one that decides whether anything renders.

Doing the small extents first is not just convenience: each one is a contained
reverse-engineering problem that either resolves or tells us something, whereas
`data` will keep producing ambiguous readings until we have something to check
them against.

## Method note

Every format question this phase has been settled the same way: write the
hypothesis, test it against **all 27 levels at once**, and require an invariant
that holds everywhere. The sector/byte ambiguity in Phase 1 survived because it
happened to be correct on the one container checked by hand. The level header
was fixed in a single pass because the tiling check failed immediately on
levels 1, 2, 4, 7, 8, 11, 19 and 20.
## tfrag terrain: block structure and vertex positions

Settled. `gc-formats::tfrag` parses the block, and a gated test decodes every
patch on all 27 levels.

### Block and patch framing

The tfrags block opens with a 12-byte header: `table_offset` (always `0x40`),
`tfrag_count`, and an `f32` LOD distance (`32.0`). The patch table follows at
`table_offset`, `tfrag_count` records of `0x40` bytes each. On level 0 that is
833 patches.

A patch header's `data` offset is **relative to `table_offset`**, not to the
start of the block. Getting this wrong happens to work for the first patch and
silently produces garbage afterwards.

### Positions live in three lists, not one

`vert_count` is the *total* across all three position tiers. Within a patch's
data block the VIF lists do not run in ascending offset order — the LOD-2 list
sits *before* the common list — and the LOD-0-only list has no offset of its
own, starting instead at `shared_ofs + lod_1_size * 16`. The ranges, in transfer
order, are:

| List | Range (from the patch data block) | Holds |
|---|---|---|
| common | `shared_ofs .. lod_1_ofs` | VU header, ad-gifs, vertex info, origin, positions |
| LOD 2 | `lod_2_ofs .. shared_ofs` | strips, indices |
| LOD 1 | `lod_1_ofs .. lod_0_ofs` | strips, indices |
| LOD 0+1 | `lod_0_ofs .. shared_ofs + lod_1_size*16` | parents, vertex info, positions |
| LOD 0 | `shared_ofs + lod_1_size*16 .. rgba_ofs` | positions, strips, indices, parents |

Scanning only the common list and expecting `num == vert_count` finds positions
in roughly a quarter of patches. The other three quarters keep most of their
positions in the LOD lists. That was the first wrong turn and it looked
plausible: every patch it did match matched *exactly*.

### The VIF1 code word

Two rules had to be right before any of this worked, and both were wrong on the
first attempt.

**The vector format lives in the command byte, not the immediate.** Bits 2-3 of
the command are `vn` (component count minus one) and bits 0-1 are `vl` (0 =
32-bit, 1 = 16-bit, 2 = 8-bit, 3 = 5-bit), so

    element_size = ((32 >> vl) * (vn + 1)) / 8

Terrain positions are `vn=2, vl=1` — command `0x69` — at 6 bytes each. Reading
the format out of the immediate's low nibble, which is the more commonly quoted
layout, yields 12 or 16 bytes and quietly misaligns everything after it.

**Unpack payloads pad to 4 bytes, and several non-unpack commands carry inline
data.** An unpack advances `(count * element_size + 3) & !3`. `STROW`/`STCOL`
carry 16 bytes, `STMASK` 4, `MPG` `num * 8`, `DIRECT`/`DIRECTHL` `imm * 16`;
every other non-unpack is a bare 4-byte code word. Ignoring the first four
desynchronises the walk, and a desynchronised walk still finds code words that
look like positions — it just finds the wrong number of them.

The 4-byte padding is easy to misread. With 75 elements the payload is 450
bytes and the following word lands at +452, where `0x01000404` (STCYCL) reads
perfectly plausibly. The word at +450 is `0x04040000` (ITOP), which is the real
one. Taking the plausible-looking reading is how this was briefly wrong.

### Verification

Requiring every patch to yield exactly its own `vert_count` is the invariant
that caught both errors, and it is the check the parser is built around: a
mismatch means the walk desynchronised, so `tfrag_positions` returns `None`
rather than partial geometry. Across all 27 levels: 18,102 patches,
821,439 vertices, zero mismatches.

### Topology: strips, indices and triangles

Also settled. `parse_tfrag` walks each list and classifies every unpack by
format *and* destination VU address, using the VU header at the head of the
common list (5 V4_16 vectors, unsigned, to address 0). That table is what
separates the strips from the indices from the parent arrays — they are all
V4_8, and only the address tells them apart.

| `(vn, vl)` | Meaning |
|---|---|
| (3, 1), unsigned, addr 0 | the VU header itself |
| (3, 0) | ad-gifs, 5 quadwords each |
| (2, 1) | positions, V3_16 |
| (3, 1) | vertex info: UV, LOD parent, own position |
| (3, 2) | strips (signed) and indices (unsigned) |

**Strip indices address the vertex-info array, not the position array.** Several
vertex-info entries can share a position, so a strip index resolves through
`vertex_info[i].vertex / 2` to reach a position.

A strip record is four bytes: a vertex count, a packet flag and an ad-gif
offset. A negative count is biased by −128, and *the first record of a list is
always biased* even when its count is positive — a real patch opens with −116 or
−124, never a small positive number. A zero count ends the list. A run of `n`
vertices is a GS triangle strip of `n − 2` triangles with alternating winding.

**Bank the LODs by index, not by arrival.** The lists are walked in transfer
order but the strip and index arrays have to be stored by *which LOD they
belong to*. Appending them as they are found fills the arrays LOD-2-first,
which looks fine until the triangle counts come out low.

### Verification

`tri_count` is stored in the patch header and cannot be derived from the lists
without walking them, so reproducing it is a strong check. All 27 levels:
**18,102 patches, 809,879 triangles, every patch matching its header**, with
every triangle index inside that LOD's vertex-info range and every ad-gif index
inside the ad-gif array.

Getting there took four wrong turns, each of which failed quietly: reading the
element size from the immediate, 2-byte payload padding, ignoring non-unpack
payloads, and banking strips by arrival order. Every one produced parseable,
plausible geometry; only the header's own triangle count caught them.

### Texture bindings, colours and lights

Also settled. Adds `level_core.rs` for the index header and the texture table,
which is the join between tfrags and texel data.

A tfrag's **ad-gifs** are five A+D quadwords (0x50 bytes) holding packed
fields rather than GS registers; the level load rewrites them in place before
drawing. On disc `TEX0_1.data_lo` indexes the level's texture table,
`TEX1_1.data_lo` is the LOD bias K and `data_hi` the minification filter, and
`CLAMP_1` holds the two wrap modes. The texture table is a count/offset pair at
index offset 0x30, entries of 16 bytes.

The table checks out on its own terms: on level 0 it is 93 entries at 0x2900,
and 0x2900 + 93 x 16 lands exactly on the next range at 0x38.

Three details worth writing down:

* **K is a signed 12-bit value in 1/16 units**, sign-extended by shifting the
  field up 4 and back down. Retail values are -137..-89, which is -8.56..-5.56
  mip levels. Reading the field as unsigned gives 1911 instead of -137.
* **`data_hi` of TEX1_1 is MMIN, and 4 is `LinearMipmapNearest`**, not
  `LinearMipmapLinear` as the bit layout alone suggests.
* **RGBA and the light/normal records are not in the VIF lists.** They sit at
  fixed offsets in the patch's data block: `rgba_size * 4` four-byte colours at
  `rgba_ofs`, and `vert_count` eight-byte records at `light_ofs + 0x10`. The
  origin also lives there — if no STROW row supplied one, the quadword at
  `light_ofs` is the origin.

### Verification

Every ad-gif must index a texture that exists, or the geometry and the texture
data have been misread independently of each other. Across all 27 levels:
**47,950 ad-gifs over 18,102 patches, all indices in range**, referencing
1429 of the 1595 level texture entries (90%). Each patch's light array is
exactly `vert_count` records and its RGBA array exactly `rgba_size * 4`.

### Texels: 8-bit indexed images and CSM1 palettes

Settled. Adds `texture.rs` and makes the pipeline write `gs_ram.bin`.

A level texture is not pixels, it is **palette indices**: `width * height`
bytes at `core_data[textures_base + data_offset]`. The colour comes from a
256-entry CLUT (1024 bytes) in GS RAM at `palette * 0x100`. `gs_ram` is a byte
range in the level's data extent header at offset 0x10 — 518,656 bytes on level
0 — and the extractor was not writing it out, so nothing could be decoded
before this.

Two PS2 conventions have to be honoured, and neither is a design choice:

* **CSM1 CLUT order.** Within each 32-entry group the two middle 8-entry
  blocks are swapped: bits 3 and 4 of the index are exchanged. The remap is an
  involution.
* **Alpha runs 0..=0x80, not 0..=255.** `0x80` is fully opaque and anything
  above it clamps.

The mip chain splits across two buffers. Levels 0 and 1 are in the texture
block, with level 1 immediately after level 0 at `data_offset + w*h`; levels 2
and 3 are resident in GS RAM at `mipmap * 0x100` and `pad * 0x100`. Tfrag
textures are square, so every level halves evenly.

### Verification

Numbers alone do not settle this — every step is a table lookup, so a wrong
CLUT remap still yields a well-formed image of the wrong thing. Two layers:

* **Shape.** All 1,595 tfrag textures across all 27 levels decode; 96.8% are
  fully opaque and 96.3% use more than eight distinct colours. A flat,
  transparent or single-colour result would point at a CLUT or offset mistake.
* **Eyes.** `gc-extract dump-texture` writes any tfrag texture, or any of its
  mip levels, as a binary PPM. Decoded level 0 texture 0 is a clean bevelled
  metal panel with rivets and specular highlights; texture 74 is a vent grille
  whose punched windows come back fully transparent. Both confirm the CLUT
  remap, the alpha scale and the pixel base at once, which no assertion in the
  suite does.

That tool is worth keeping: it is the cheapest way to tell a decode that is
merely well-formed from one that is right.

### LOD selection

Mostly settled. The block header's float at +0x08 gives `L`, and the switch
distances are `[D0, D1, D2] = [6L, 4L, 2L]` world units; on every level `L` is
32.0, so `[192, 128, 64]`. The distance test compares the patch's bounding
sphere, truncated to integers, against `trunc(D * 1024)`. Six modes result —
LOD 2, LOD 1 collapse, LOD 1 morph, LOD 0 collapse, LOD 0 morph, LOD 0 — and
the *sphere* is what is tested, not the centre, so a patch straddling a
threshold takes the coarser mode. That is what stops geometry popping as the
camera moves, and it is worth testing for directly: sweeping the depth must
never move back to a finer mode.

A morph weight runs 0 to 1 across its tier's band: LOD-01 between D1 and D0,
LOD-0 between D2 and D1. Past the far edge of the band an entry is not morphed
but *replaced* by its first parent — a collapse, not a blend.

### Parent indices: settled, but only after two corrections

The four V4_8 arrays (`parent_indices_lod01`, `unk_indices_2_lod01`,
`parent_indices_lod0`, `unk_indices_2_lod0`) are classified by VU address,
and **the addresses are not unique**. On level 0 patch 1,
`unk_indices_2_lod_01_addr == parent_indices_lod_0_addr == 291`, and
`unk_indices_2_lod_0_addr == indices_addr == 294`. So address matching alone
cannot separate them, and getting it wrong is silent: the arrays still parse,
they just hold the wrong numbers.

Two things were needed to fix it, and the first attempt at getting it wrong is
worth recording because it looked like a data problem rather than a code one.

**Each V4_8 element is 4 bytes and holds four indices**, so an array of N
entries is `count * 4` bytes, not `count`. An early analysis compared the
unpack's element count against the position counts and concluded that 15,000
patches could not possibly hold enough parents. That was the wrong unit; the
data was fine.

**`unk_06` and `unk_0a` suppress an empty tier.** When the extra vertex-info
count is zero there is no extra array, and its VU address is simply the next
tier's. Guarding those two branches on `unk_06 != 0` / `unk_0a != 0` lets the
address fall through to the array that actually exists. This one change took
resolved LOD-0 entries from 28,739 to 341,306.

### Verification

Every vertex-info entry across all 27 levels is now accounted for: of 886,841
entries, 516,492 are common and have no link by design, and **all 370,349 LOD
entries resolve** — 29,043 LOD-01 and 341,306 LOD-0 — with zero failures. Each
resolves to a first parent that is real vertex info, a LOD-01 parent that is
always a common entry, and three positions inside the patch. A common entry
never claims a parent.

Ruled out along the way: address-only classification (addresses collide), and
one parent array entry per unpack element (wrong unit).

### The origin, and why no parser test could catch it

Positions are `i16`, so a single patch spans at most +-32 world units around
its own origin. The origin comes from an STROW row, and the obvious rule --
the last non-index STROW in the list -- is **wrong on RAC2**. RAC2 also uses
STROW for the per-vertex light records, and those rows come *after* the origin,
so taking the last one yields `[186, 186, 186, 186]` and every patch collapses
onto the world origin.

Nothing in the test suite noticed, and that is the point worth recording:

* `vert_count` still matched, because it counts positions, not places.
* `tri_count` still matched, because the topology never changed.
* Every parent, morph target and triangle index still resolved, because those
  are all relative to the same broken origin.

The entire level fit in a 63 x 62 x 62 unit box centred on the origin, and all
of it was self-consistent. Only rendering exposed it, and only because the
screenshot put the camera where the bounding spheres said the terrain was.

The rule that works: **the origin is the STROW immediately preceding the first
V3_16 position unpack** in the common list. With that, level 0's vertices span
x 86..444, y 46..211, z -477..40, which matches the patch bounding spheres.

Lesson for the rest of the project: a parser can be entirely self-consistent
and still describe the wrong thing. Where a value is absolute rather than
relative, check it against an independent source -- here, the bounding spheres.

### Why the first frame is dark

The renderer draws unlit: texture times the tfrag's own per-vertex colour.
That product is genuinely dark -- level 0 patch 0's colours are `(25, 26, 26)`
and `(92, 121, 119)` in 0..255 -- and the game's lighting is what brightens it.

Trying a host directional light or cranking ambient brightness was the wrong
instinct and it showed: with no normals the PBR path is degenerate, so
brightness changes did nothing at all. Faking it would also have hidden the
gap. The colour attribute is picked up from the mesh automatically, so it is
the *colour* data that is missing lighting, not a flag that was left off.

The real model is `LightTfrags`. The PS2 float model and the pass itself are
now written (`gc-engine::ps2`, `gc-engine::tfrag_light`); what is missing is the
light bank. The three inputs were:

* the **light records** -- already parsed: azimuth and elevation as *indices*
  into the overlay's sine table, plus the base colour;
* the **normal table** -- now decoded (`gc_formats::tfrag_light`), and normals
  are built and fed to the renderer;
* the **light bank** from the level's `gameplay` extent -- **not working**,
  see below.

### Point lights are a runtime feature, not a level-load one

Every patch on every level has `point_lights == 0xffff` (none) and
`dir_lights_one == -1`, checked across all 18,102 patches. The eight-slot point
light bank is filled by gameplay at run time, not read from the disc.

So `point_light_vertex` is not missing work for a static level render -- it
would light nothing. The directional pass *is* the complete lighting for a
level-load frame, and the point-light path belongs with the hero, explosions
and the other things that spawn light.

`dir_lights_one == -1` on every patch also means the per-vertex `select` word
is what selects a light set, never the header field -- which is what the
implementation uses.

### The class tables were not unreadable, only unsized

`core_index` holds moby, tie and shrub class tables at 0x18, 0x20 and 0x28,
followed by the texture tables. They had been ruled out for want of an entry
size, and every field order and stride tried.

The sizes are knowable exactly, because **each table ends precisely where the
next begins**:

- moby 238 entries at 192, 32 bytes each: 192 + 238x32 = 7808 = tie's offset
- tie 74 at 7808, 32 bytes: 7808 + 74x32 = 10176 = shrub's offset
- shrub 7 at 10176, 48 bytes: 10176 + 7x48 = 10512 = the tfrag texture table
- tfrag textures 93 at 10512, 16 bytes: 10512 + 93x16 = 12000 = moby textures

That closes on **27 of 27 levels**, which is what makes it evidence rather
than a coincidence: any wrong entry size would break the chain immediately.

The class entries are 32 bytes -- `{asset offset, class number, two unknown
words, 16 bytes of texture slots}` -- and a shrub entry adds 16 bytes of
billboard slots for 48. Across all 27 levels: **5,796 moby, 1,864 tie and 358
shrub classes**, and 1,595 tfrag textures.

Every shrub class on level 0 carries all-zero billboard slots, so the billboard
dimensions are optional rather than something to assert on.

The failure here is the third instance of the same shape. The gameplay extent
was "an unidentified compression" because it was being parsed compressed; the
tfrag record framing was "unrecoverable" because the entry size was never
known. Both looked like properties of the format and were properties of the
question.

### Where the class tables point

`assets_compressed_size` and `assets_decompressed_size` at index offsets 0x88
and 0x8c are 9,738,678 and 16,969,296 on level 0 -- exactly the sizes of
`core_data` before and after decompression. There is no separate asset
container: the "asset WAD" *is* `core_data`, and `offset_in_asset_wad` indexes
the decompressed form directly.

That is checkable rather than merely plausible, and a test checks it on all 27
levels:

* `assets_decompressed_size` equals `core_data`'s exact length;
* every class offset is either 0 or inside `core_data`;
* the moby, tie and shrub spans do not overlap each other.

An offset of **0 means the slot has no geometry**, not the start of the lump.
On level 0 the moby table's ninth entry is class 4909 at offset 0.

Two assumptions worth discarding: the tables are **not** sorted by class number
(level 0's moby classes run 0, 1, 2, 14, ... 4909), and the offsets within a
table are **not** monotonic. Neither is needed, but both look true from the
first few entries.

The chunk descriptors after the named extents are not it. They exist on only
3 of 27 levels -- level 0's four named extents tile the container exactly --
so an asset blob there would be missing almost everywhere.

At the offsets sit raw GS/VIF packet data, not a further container: moby[0]
begins `00 00 00 00 3f 16 02 55 6f 6f`, tie[0] `00 01 00 00 70 50 00 00`,
which read as packet headers rather than a magic number.

### Tie: the class header only partly matches

Tie is the easy one of the three -- instanced static geometry, no skeleton -- so
it is the right place to start.

RAC1's class header is 0x80 bytes. RAC2 agrees with it over the first `0x48`:
the per-LOD packet offsets, the three LOD distances and `o_class` all check out.
From `0x48` on it does not. `lod_info` at `0x50` reads back as noise,
`packet_count` and `texture_count` are zero on every class examined, and the
`normals` offset at `0x0c` points 117,788,434 bytes into a 16.9 MB lump.

So only the agreed-on prefix is decoded. The evidence for trusting it:

- **`o_class` at `0x44` equals the core index's class number on all 1,864 tie
  classes across all 27 levels.** A field landing on the right number that many
  times is not a coincidence, and it is what pins the header base.
- The LOD distances are ordered `0 < near < mid < far` for **1,784 of 1,864**.
  The 80 exceptions are real, unexplained, and are reported rather than
  asserted away.

Level 0's first tie class has distances 12.0 / 25.0 / 40.0, which is a
plausible prop LOD band, and packet offsets 256, 20,592 and 26,048.

The remainder is not guessed at. A field order that produces plausible output
and is wrong is the specific failure this file keeps recording -- the gameplay
extent, the tfrag record framing and the class tables have all been it.

### Why the frame is still dark: the light bank does not parse on RAC2

The reference frames the bank as "a u32 offset at +0x04, then an `i32` count
there, then the sets". On RAC2 level 0 that offset is `0xa0d`, and the `i32`
there is `698810368` -- not a count, and not a plausible one. `gameplay.bin`
begins with `0x794e...`, not with a header of the shape the reference expects,
so RAC2's gameplay extent is not laid out this way, or is compressed before
this point.

The pass therefore never runs. Rather than let the renderer quietly draw the
stored colours and look plausible, it counts what it lit and says so:

```text
lighting: 0 of 80274 vertices shaded by LightTfrags; the rest fall back to the
tfrag's stored colours (no usable light bank)
```

That line is the honest state, and it is why the frame has not changed. What
`gameplay.bin` actually is, and where RAC2 keeps its light bank, is the open
question — `0x794e...` looks like it may be a compressed or encrypted
container, in which case the bank is somewhere else entirely.

### The PS2 float model

`gc-engine::ps2` is the EE's FPU on raw `u32` bit patterns, and it is not
`f32`:

* **no denormals** -- exponent 0 *is* zero, whatever the mantissa
* **no NaN, no infinity** -- exponent 255 is an ordinary value, which means
  `MAX` reads back in Rust as NaN. A test asserts exactly that, because it is
  the clearest single statement of why this module exists.
* **every result truncates toward zero**, and `1/3` comes out below `1/3`
* **the adder aligns operands**: the smaller operand loses the bits more than
  one below the larger's LSB, and a 25-bit exponent gap returns the larger
  operand unchanged
* **`rsqrt` is sqrt-then-divide**, with truncation between, not one rounded step

`mul`, `add`, `div`, `sqrt`, `itof12` and `max`/`min` all differ from `f32` in
at least one case a lighting pass will hit. The pass accumulates them across
every vertex of every patch, so this is a visible difference rather than a
rounding curiosity.

### The lighting pass itself

Per vertex: `base + A.rgb * max(dA, dA*wA) + B.rgb * max(dB, dB*wB)`, where
`base` is the record's 5:5:5:1 colour expanded by PEXT5, `dA`/`dB` are
three-component dot products against the light directions, and `wA`/`wB` come
out of the colour w lanes. `select` picks one light set or blends two by
`bits 8..15 / 256`, and a blended direction is re-normalised where an unblended
one is not.

One field had to be added to `TfragLight` to carry this: `select`, the `u16` at
+6 of the record. It was briefly reconstructed with an `unsafe` pointer cast
over the struct -- which is exactly the kind of thing that breaks silently the
moment a field is reordered, and is now a plain parsed field.

### The normal table, and why it is found by searching

The reference reads the table from an ELF at `BOOT_NORMAL_TABLE_VADDR`
(`0x165500`) and notes every level overlay carries a copy. **The overlay has no
ELF header** -- it is a raw loadable image starting with code -- so there is no
program header to map a virtual address through, and the offset is not fixed
either: level 0 has the table at `0xb650c`, level 5 at `0xcca30`, level 26 at
`0xb6284`.

So it is located by content. Every table starts with exactly `(1.0, 0.0)` --
angle zero -- and the scan then requires all 256 entries to match
`cos(i·pi/128), sin(i·pi/128)`. Validating the whole curve is what makes the
search safe: a lone `(1.0, 0.0)` pair of floats occurs by accident in a 2.7 MB
executable, and a test plants one as a decoy. The values read from the disc are
used as-is rather than recomputed, since those are the numbers the game indexes.

A normal is `(cos(az)cos(el), sin(az)cos(el), sin(el))` **negated** -- the
stored normals point into the surface -- then given the same Z-up to Y-up swap
as positions, or the lighting comes out sideways. The azimuth and elevation
fields are stored signed but index 0..=255, so the byte is what matters, not its
sign: entry 200 is read as 200, not as -56.

Given the table, a normal is `(cos(az)cos(el), sin(az)cos(el), sin(el))`,
negated, exactly as `decode_normal` builds it. The lighting pass itself is
`base + A.rgb * max(dA, dA*wA) + B.rgb * max(dB, dB*wB)` per vertex, and it
must be reproduced in the PS2's float model rather than in Rust `f32`, per
the fidelity rule.

### Still open

- **Point lights.** The directional pass runs; `point_light_vertex` and its
  `r^2 - |p - L|^2` cull test are not written, and they are the one part of
  the lighting that *is* distance-dependent.
- **The GS register set** the level load builds from the texture entry
  (TBP0/TBP1 and so on), for a renderer that emulates the GS rather than
  resolving texture addresses on the host.
- **The guard-band clip test** that sends patches crossing the guard band to
  the clipping program.
- **Lighting magnitudes.** The pass runs and the output is plausible; whether
  it matches the game needs comparison against reference frames, which needs
  the PCSX2 loop that is still to come. Nothing in this file establishes it.
- **tie, moby and shrub geometry.** The class tables parse and the container
  they point into is identified (below), but none of the three is decoded yet.
  Moby is the largest of the three and is skeletal, with an animation evaluator
  in the microcode; it is the single biggest piece left before the rest of a
  level can draw.
- **Point lights** are spawned by gameplay at runtime and are absent from the
  disc -- every one of the 18,102 patches has `point_lights == 0xffff` -- so
  there is nothing for them to light until the game runs.

A tfrag can now be turned into positioned, UV-mapped, textured, lit triangles,
told which LOD to draw, and morphed toward its parents at the right rate.
Everything needed for a first frame is parsed, and the frame is verified by
screenshot rather than by assertion.

## Core index field order (2026-10-11)

Repeatedly re-deriving these by hand wasted real time this session, so the
order is written out. It matches `LevelCoreHeader::parse` in
`crates/gc-formats/src/level_core.rs`:

| offset | field |
|--------|-------|
| 0x00   | gs_ram count |
| 0x04   | gs_ram offset |
| 0x08   | tfrags |
| 0x0c   | occlusion |
| 0x10   | sky |
| 0x14   | collision |
| 0x18   | moby_classes count |
| 0x1c   | moby_classes offset |
| 0x20   | tie_classes count |
| 0x24   | tie_classes offset |
| 0x28   | shrub_classes count |
| 0x2c   | shrub_classes offset |

Note `gs_ram` and the three class tables are **count/offset pairs**, so every
one of them is two words. The region offsets (`tfrags` through `collision`) are
single words. Counting the class tables as one word each shifts every field
after them, which is how `collision` got read as `sky` and produced a page of
implausible collision headers on all 27 levels.

## Sky (2026-10-11)

Region is `[sky, collision)` and is well defined, but the header is **not**
yet decoded and the layout is not uniform.

Reading `u16` at the head, most levels give:

```
+0x00 0
+0x02 0
+0x04 4 or 6      varies per level
+0x06 0
+0x08 0           level 2 is the exception
+0x0a 3..6        looks like a layer count
+0x0c 0
+0x0e 64 (0x40)
+0x10 128 or 192  varies
+0x12 64 (0x40)
```

Level 2 breaks the pattern: `+0x08` is 0x0109 and the three words after the
apparent count are shifted and different (`80 00 00 00 00 01 00 00 40 00`).
So a single fixed header does not fit every level, and the count at `+0x0a`
cannot be trusted to be one.

A table of offsets follows at `+0x20` for the layers. The offsets ascend and
sit in the last 8-15% of the region, but the count of *non-zero* offsets is
sometimes less than the declared count (level 26 declares 6, has 4 real
offsets and 2 zero slots). Unresolved.

## Moby geometry (2026-10-11)

Class entries are 0x20 bytes in `core_index.bin`, fields
`geom_offset, o_class, unknown_8, unknown_c, textures[16]`; all 238 level-0
moby classes carry `textures == ffff..ff`, i.e. no texture slots.

`geom_offset` points into `core_data`, and the spans tile without overlap.

The span does not start with the geometry. Its first `u32` is the offset of a
part table, and the table is 16-byte records:

```
+0x00 u32 vertex_offset   from the start of the class
+0x04 u16 vertex_count
+0x06 u16 second_count    meaning not established
+0x08 u32 next_part
+0x0c u32 flags
```

Parts tile: `vertex_offset + 16 * vertex_count == next_part`. The stride is not
assumed -- it falls out of that identity, which is what pins the record size
and the field boundaries. It holds for 43,467 of 43,467 part records across
every moby class on all 27 levels.

Two terminations are needed, not one. A zero `vertex_offset` ends the run, but
172 records across the disc have a zero `vertex_offset` *and* a zero
`next_part` while still carrying a vertex count; stopping only on the second
is what leaves 99.6% instead of 100%. A zero `next_part` means the record
starts a different table rather than a part.

Totals: 1,270,802 vertices over 4,560 class spans, 116 of which carry no
geometry.

Vertices are 16 bytes of VIF-packed half-float quads, cycling through four
distinct 32-bit words. Not unpacked -- handed back raw.

## Shrub geometry (2026-10-11)

Class entry strides differ between the three tables. Measured by how many
entries yield a geometry offset inside `core_data` and a plausible `o_class`,
across all 27 levels:

| table | stride | sane entries |
|-------|--------|--------------|
| moby  | 0x20   | 5769 / 5796  |
| tie   | 0x20   | 1864 / 1864  |
| shrub | 0x30   | 356 / 358    |

Reading shrub at 0x20 still yields 231 sane-looking entries out of 358, which
is the problem: it is wrong often enough to look right. The stride is already
handled -- `ShrubClassEntry` is 48 bytes, carrying billboard texture fields the
other two do not have.

Shrub geometry is **not** the moby part-table format. All 358 spans have a
leading word of 0 or one too large to index a table into, so none of them
contain a moby-style part table.

The header does self-identify in two places, both exact across all 358 spans:

- `span + 0x24` is a `u32` equal to the class entry's own `o_class`. 358/358.
- `span + 0x2c` is a `u32` data offset that lands inside the span. 358/358.

`span + 0x10` reads 5.0f on 281 of 358 spans, so it is a value rather than a
constant marker and carries no structural weight. The header is otherwise
undecoded: the first 0x20 bytes are packed data rather than floats, despite
`+0x10` landing exactly on 5.0.

## Tie geometry (2026-10-11)

Tie does **not** reuse moby's part-table format. The rule
`vertex_offset + 16 * vertex_count == next_part` holds for **0 of 25,024**
records tried across the 1,864 tie class spans, against 43,467/43,467 for
moby. Tie's second field also reads as an absolute offset rather than a
span-relative one, which is the first sign the record shape differs.

At `+0x48`, where the verified header currently ends, the bytes are f32-shaped
(10.0, 40.0, 50.0, 4.0, -100.0, -25.0, 5.0, 3.125), not a VIF marker as
expected. They are not a fixed-size array either: the run of consecutive
plausible floats varies from 1 to 6 across spans, with 822 spans having 6,
455 having 1 and 373 having 2. So there is more header past `0x48`, and it
is not a single array of distances.

### Probe tool

`gc-extract probe <unpacked-dir> <level> [max-rows]` dumps the core header and
all three class tables through `LevelCoreHeader`, which is the point. The
table above and the shrub offset below were both established by hand first and
one of them nearly sent me to fix a bug that was not there. The tool also
reports how many entries carry geometry inside `core_data`, which is how the
shrub stride was pinned: 7/7 at 0x30 against 4/7 read at 0x20.

Note the LOD distances already decoded sit *before* this point (12.0, 25.0,
40.0 at `+0x10`), and the floats after `0x48` are a different population --
negative values, and a repeated leading 40.0.

Unresolved. The packet geometry remains the open question, unchanged.

## Shrub body (2026-10-11, later)

Stride-fitting the body failed three times, so this came from a different
angle: group spans by `(count, data_offset, tail_length)` and byte-diff every
member of each group. Byte positions that differ across a group are geometry;
positions that never differ are structure.

The result is blunt. **144 bytes vary in every group**, whether the group has
count 1, 2, 3, 5 or 6 and a tail of 192, 208 or 240 bytes. So the varying data
is not sized by `count` at all, and the three strides that failed were fitting
the wrong thing.

Those 144 bytes sit in runs of 6, every 8 bytes: `(0..5), (8..13), (16..21),
...`. The body is an array of **8-byte records, 6 varying and 2 constant**.

The payload is not half-floats. Decoding the 6 bytes as a `half3` gives 245.25
and -168.375, which no shrub has.

**Correction, later the same day.** This section originally concluded the
payload was VIF-packed, on the strength of recurring patterns like `7ea5` and
`1ab6`. That conclusion does not survive a control. Decoding every 32-bit word
of every shrub body as a `VifCode`:

| region | words decoding as UNPACK | distinct cmd bytes |
|---|---|---|
| shrub bodies | 24.9% | 128 (all of them) |
| uniform random | 25.1% | 128 (all of them) |
| sound_bank (definitely not VIF) | 19.0% | 122 |
| gs_ram (mixed) | 13.6% | 128 |
| moby class 0 span (known VIF) | 40.1% | 105 |

Shrub sits on the random baseline, and all 128 possible command bytes occur,
which is what unstructured data looks like. The VIF reading was pattern
matching on a handful of recurring byte values, and it was wrong.

Note the test is weak in general: even the known-VIF region only reads 40.1%.
So this does not prove shrub is *not* VIF -- it proves the claim was never
established. Treating three recurring bytes as a format is the same mistake as
the three stride guesses, just at a different level.

What survives from this section is the differential finding, which rests on
grouping and diffing rather than on recognising byte patterns: 8-byte records,
6 varying and 2 constant, 144 varying bytes per group regardless of `count`.
The payload's encoding is still unknown.