Decode the shrub class blob: a 24-normal palette and a billboard record

The shrub body's payload encoding was the open question after the span-bound
work, and it answered itself once the R&C1 reference was in view: R&C1's shrub
class header keeps the field offsets R&C2 shows (o_class at 0x24, a count at
0x28, the data offset at 0x2c), which says which bytes to look at, and the disc
says what they hold.

Range measured: all 358 shrub class spans over the 24 levels that have them,
each bounded by its successor class or the located chunk container. Nothing
outside the spans is read.

Result. The body is a palette of 24 unit-length s16 normals -- s16 x, y, z and
a zero pad word per 8-byte record -- followed by all-zero padding records.
Three independent measurements, each of which fails for any other reading:

- all 8,592 live entries have squared length within 65,535 of 32767^2, and
  the bound is tight (the largest observed deviation is exactly 65,535). That
  is 32768^2 - 32767^2, which is what a unit vector scaled by 32768 produces
  when an axis-aligned normal quantises to -32768 and the positive axis clamps
  to 32767; components run the full s16 range and 1,765 entries are
  axis-aligned. A random s16 triple is off by about a billion, so one entry in
  fifteen thousand would pass by chance: all 8,592 pass.
- the padding records are always at index 24 or later and always all zero,
  never a live entry. That is what separates the palette from geometry, and it
  is the observation the three earlier stride-fitting attempts were missing.
- 17 classes carry a palette of 24 identical (0, 0, 1) entries. A position or
  index table cannot produce that; a normal palette for a flat-facing class
  must.

The header decodes as: sphere at +0x00 (centre + radius, radius positive on
all 358 spans, 1.72..41.57), size at +0x10, mode_bits and scratch at +0x14,
a zero word at +0x18, the billboard offset at +0x1c, and scale at +0x20. The
parser asserts the two invariants that hold without exception (scratch and the
reserved word are zero on all 358) plus a positive radius and scale, and
reports mode_bits rather than asserting it: it is 4 on 16 of the 358 spans.

The billboard offset is the strongest self-identification after o_class
because it is confirmed from a second file: it is non-zero on exactly the 25
classes whose core-index entry carries billboard texture fields, pairing
correctly both ways. At the offset sits R&C1's ShrubBillboard shape, register
block and all -- fade_distance, then TEX1_1 (K -233..-168, MMIN 4, address
0x14), TEX0_1 (index 1..4, address 0x06) and MIPTBP1_1 (address 0x34), all
three register addresses checked in the disc test.

Two corrections this produces, both recorded in docs/plan/level-assets.md
because the old readings were confidently wrong rather than merely
unverified:

- "reserved is 0 or a denormal" had the two u16 halves the wrong way round:
  the 16 non-zero spans are mode_bits = 4 with scratch = 0. As an f32 the word
  is still a denormal, so the PS2 reads zero either way -- but the field is a
  flag word, not a magnitude, which is exactly the FPU point worth making.
- "bounds" was tested as a min.xyz/max.xyz box, which holds on only 146 of
  358 spans, and the field was left unnamed as a result. It is a bounding
  sphere: the fourth float is positive on all 358 spans and the centres
  cluster near the class origin. The 146/358 figure was a property of the
  wrong question.

Still open: the three floats in the billboard record (they repeat
byte-identically across levels -- 16 distinct triples among 25 records, one on
four classes across levels 1, 4, 12 and 21 -- so they are shared art data
rather than dimensions), size, mode_bits, count, and where the shrub mesh
geometry lives. R&C1 puts the mesh packets in this blob; R&C2's blob holds
only the palette, so the 333 classes without a billboard must draw meshes from
somewhere else.
This commit is contained in:
2026-10-11 13:33:37 +02:00
parent ccea53807a
commit 2c7555fe16
4 changed files with 658 additions and 134 deletions
+123 -1
View File
@@ -1003,4 +1003,126 @@ Consequences, all measured over the new bounds:
invented edge. The chunk test now reads the region from `core_layout`.
What is still not known: the marker's meaning (constant across 27 levels, at
exactly this position, unnamed), and the shrub body payload's encoding.
exactly this position, unnamed). The shrub body payload's encoding is the next
section.
## The shrub class blob: palette and billboard (2026-10-11, after the bound)
With every span bounded by the chunk container, the shrub class blob reads
cleanly against the R&C1 format (`vendor/rerac/docs/formats/shrub_sky_rac1.md`,
derived from the Wrench sources). R&C1's shrub class header keeps the same
field offsets R&C2 shows -- `o_class` at 0x24, a count at 0x28, the data offset
at 0x2c -- so the reference says which bytes to look at and the disc says what
they hold.
**Range measured:** all 358 shrub class spans over the 24 levels that have
them, each bounded by its successor class or the located chunk container.
Nothing outside the spans is read.
### The body is a 24-entry normal palette
Two things made this identifiable where three stride-fitting attempts had
failed. First the shape: every body is 8-byte records whose last two bytes are
zero, and R&C1's palette is documented as 24 records of exactly that shape
(`s16` x, y, z, pad). Second the count: bodies hold 24, 26, 28 or 30 records
with the extra ones **always all zero and always at index 24 or later** --
never a live entry. So the body is 24 normals plus zero padding, and the
padding is what distinguishes it from geometry.
The palette is real normals, and the test says how firmly:
- all 8,592 live entries have squared length within **65,535** of `32767^2`,
and the bound is tight -- the largest observed deviation is exactly 65,535.
That number is not fitted: it is `32768^2 - 32767^2`, which is what a unit
vector scaled by 32768 produces when an axis-aligned normal quantises to
`-32768` while the positive axis clamps to `32767`. Components run the full
`s16` range (-32768..32767) and 1,765 of the 8,592 entries are axis-aligned,
which is what a coarse direction codebook looks like.
- a random `s16` triple has squared length spread about a *billion* either
side of `32767^2`, so one entry in fifteen thousand would pass by chance.
All 8,592 pass.
- the clincher: 17 classes carry a palette of 24 identical `(0, 0, 1)`
entries. A position or index table cannot produce that; a normal palette for
a flat-facing class must.
This is also where shrub lighting comes from -- R&C1 lights each instance
against its class's 24-direction palette. Unlike the tfrag's per-vertex light
records, which store azimuth/elevation *indices* into the overlay's sine table,
the shrub palette stores the directions outright, so the overlay table is not
needed for shrubs.
### The header, byte by byte
**Range measured:** the 0x30 bytes of every span.
| offset | field | measured |
|---|---|---|
| 0x00 | f32[4] `sphere` | centre (xyz) + radius (w). Finite on all 358, **radius positive on all 358** (1.72..41.57), centres near the class's own origin |
| 0x10 | f32 `size` | 3.0..20.0, positive on all 358 |
| 0x14 | u16 `mode_bits` | **0 on 342 spans, 4 on 16** |
| 0x16 | u16 `scratch` | **0 on all 358** (asserted by the parser) |
| 0x18 | u32 `reserved` | **0 on all 358** (asserted by the parser) |
| 0x1c | u32 `billboard_offset` | 0 on 333 spans, in-span on 25 |
| 0x20 | f32 `scale` | 0.0044..1.8, positive on all 358 |
| 0x24 | u32 `o_class` | equals the class entry's own, 358/358 |
| 0x28 | u32 `count` | 1..55, meaning not established |
| 0x2c | u32 `data_offset` | inside the span, 358/358 |
Two corrections this produces, both recorded because the old readings were
confidently wrong rather than merely unverified:
- **"reserved is 0 or a denormal" was the wrong shape.** The 16 spans with a
non-zero value there have `mode_bits = 4` and `scratch = 0`, not the other
way round. As an `f32` the word at `+0x14` is still a denormal, so the PS2
reads zero either way -- but the field is a flag word, not a magnitude, and
the demo is the point: the EE's FPU has no denormals, so `04000000` and zero
are the same value to the hardware.
- **"bounds" is a bounding sphere, not a min/max box.** The old reading tested
`min.xyz/max.xyz` ordering, found it on only 146 of 358 spans, and stopped
naming the field. It is centre + radius, as in R&C1: the fourth float is
positive on all 358 spans and the centres cluster near the class origin
within +-24, which is where a class-local sphere belongs. The 146/358 figure
was a property of the wrong question.
### The billboard record
The billboard offset at `+0x1c` is the strongest self-identification in the
header after `o_class`, because it is **confirmed from a second file**: it is
non-zero on exactly the 25 classes whose core-index entry carries billboard
texture fields, in both directions -- 25 records and 25 entries, pairing
correctly. R&C1 states the same correspondence ("a class has a billboard record
exactly when its core-index billboard texture has width != 0").
At the offset sits R&C1's `ShrubBillboard` shape, register block and all:
| offset | field | measured |
|---|---|---|
| 0x00 | f32 `fade_distance` | 5..1000; 512 reads as 0 under R&C1's `trunc(fade) & 0xff`, i.e. billboard-only |
| 0x04 | f32[3] | **not established** -- see below |
| 0x10 | TEX1_1: K, MMIN, address | K -233..-168, MMIN 4 on all 25, address 0x14 |
| 0x20 | TEX0_1: data_lo, address | index 1..4, address 0x06 |
| 0x30 | MIPTBP1_1: address | 0x34, data words zero (filled at level load) |
The three floats at `+0x04..+0x10` are the part that does *not* carry over.
R&C1's are `width`, `height` and `z_ofs` in class units; R&C2's are an order of
magnitude too large and too varied to be dimensions of a 32..128 px billboard,
and they **repeat byte-identically across levels**: 16 distinct triples among
the 25 records, with one appearing on four classes across levels 1, 4, 12 and
21. Dimensions would not do that; shared art assets would. They are handed back
unnamed.
Two further facts about them, measured: the whole record repeats with them --
three classes on different levels carry identical `fade`, floats, K and scale
-- and the class entries they belong to carry identical `sphere` too. So the
blobs themselves are shared between levels, which is the same behaviour the
chunk container showed with its byte-identical tables.
### Still open
- The three billboard floats, `size`, `mode_bits`, and `count`.
- **Where shrub mesh geometry lives.** R&C1 keeps the class's mesh packets in
the blob between the header and the palette; R&C2's blob is header, palette
and optional billboard only -- 24 normals and nothing else. The 333 classes
without a billboard still have to draw a mesh, and it is not here. The
candidates are the chunk container that follows shrub and the gameplay
extent, and neither is decoded.
+12 -7
View File
@@ -217,13 +217,18 @@ Phase 1 hands over the extents; this phase works out what each one contains.
- [ ] `tie` — instanced environment geometry, fat vertices
- [~] `shrub` — class table done (48-byte entries, not 32; reading shrub at
32 still gives 231 sane-looking entries of 358, which is the hazard).
Geometry header parsed and both fields **checked**, not merely reported:
`+0x24` equals the entry's own `o_class`, and `+0x2c` is an in-span data
offset, on all 358 spans. Shrub does **not** use moby's part-table format.
The body is **8-byte records, 6 varying bytes and 2 constant**, and now
that every span is bounded by the located chunk container, all 358
bodies are whole records — 9,600 in total. The payload's encoding is
still unknown, and so is the 2-byte marker that closes the region.
Geometry header parsed and checked field by field on all 358 spans:
`+0x24` equals the entry's own `o_class`, `+0x2c` is an in-span data
offset, `+0x16`/`+0x18` are zero, and `+0x1c` names a billboard on
exactly the 25 classes whose core-index entry carries billboard texture
fields. Shrub does **not** use moby's part-table format. The body is a
**24-entry palette of unit-length s16 normals** (squared length within
65,535 of 32767² on all 8,592 entries — a random triple would miss by a
billion) plus zero padding records; the billboard record carries K, MMIN
and the TEX0/MIPTBP addresses as R&C1's does. What remains open: the
three floats in the billboard record, `size`, `mode_bits`, `count`, and
**where the mesh geometry lives** — R&C1 puts the mesh packets in this
blob and R&C2's blob holds only the palette.
- [x] `moby` — per-class model geometry. A class span opens with a `u32`
pointing at a part table of 16-byte records; parts tile exactly, with
`vertex_offset + 16 * vertex_count == next_part` holding for **43,467 of