Decode the shrub class blob: a 24-normal palette and a billboard record
The shrub body's payload encoding was the open question after the span-bound work, and it answered itself once the R&C1 reference was in view: R&C1's shrub class header keeps the field offsets R&C2 shows (o_class at 0x24, a count at 0x28, the data offset at 0x2c), which says which bytes to look at, and the disc says what they hold. Range measured: all 358 shrub class spans over the 24 levels that have them, each bounded by its successor class or the located chunk container. Nothing outside the spans is read. Result. The body is a palette of 24 unit-length s16 normals -- s16 x, y, z and a zero pad word per 8-byte record -- followed by all-zero padding records. Three independent measurements, each of which fails for any other reading: - all 8,592 live entries have squared length within 65,535 of 32767^2, and the bound is tight (the largest observed deviation is exactly 65,535). That is 32768^2 - 32767^2, which is what a unit vector scaled by 32768 produces when an axis-aligned normal quantises to -32768 and the positive axis clamps to 32767; components run the full s16 range and 1,765 entries are axis-aligned. A random s16 triple is off by about a billion, so one entry in fifteen thousand would pass by chance: all 8,592 pass. - the padding records are always at index 24 or later and always all zero, never a live entry. That is what separates the palette from geometry, and it is the observation the three earlier stride-fitting attempts were missing. - 17 classes carry a palette of 24 identical (0, 0, 1) entries. A position or index table cannot produce that; a normal palette for a flat-facing class must. The header decodes as: sphere at +0x00 (centre + radius, radius positive on all 358 spans, 1.72..41.57), size at +0x10, mode_bits and scratch at +0x14, a zero word at +0x18, the billboard offset at +0x1c, and scale at +0x20. The parser asserts the two invariants that hold without exception (scratch and the reserved word are zero on all 358) plus a positive radius and scale, and reports mode_bits rather than asserting it: it is 4 on 16 of the 358 spans. The billboard offset is the strongest self-identification after o_class because it is confirmed from a second file: it is non-zero on exactly the 25 classes whose core-index entry carries billboard texture fields, pairing correctly both ways. At the offset sits R&C1's ShrubBillboard shape, register block and all -- fade_distance, then TEX1_1 (K -233..-168, MMIN 4, address 0x14), TEX0_1 (index 1..4, address 0x06) and MIPTBP1_1 (address 0x34), all three register addresses checked in the disc test. Two corrections this produces, both recorded in docs/plan/level-assets.md because the old readings were confidently wrong rather than merely unverified: - "reserved is 0 or a denormal" had the two u16 halves the wrong way round: the 16 non-zero spans are mode_bits = 4 with scratch = 0. As an f32 the word is still a denormal, so the PS2 reads zero either way -- but the field is a flag word, not a magnitude, which is exactly the FPU point worth making. - "bounds" was tested as a min.xyz/max.xyz box, which holds on only 146 of 358 spans, and the field was left unnamed as a result. It is a bounding sphere: the fourth float is positive on all 358 spans and the centres cluster near the class origin. The 146/358 figure was a property of the wrong question. Still open: the three floats in the billboard record (they repeat byte-identically across levels -- 16 distinct triples among 25 records, one on four classes across levels 1, 4, 12 and 21 -- so they are shared art data rather than dimensions), size, mode_bits, count, and where the shrub mesh geometry lives. R&C1 puts the mesh packets in this blob; R&C2's blob holds only the palette, so the 333 classes without a billboard must draw meshes from somewhere else.
This commit is contained in:
+123
-1
@@ -1003,4 +1003,126 @@ Consequences, all measured over the new bounds:
|
||||
invented edge. The chunk test now reads the region from `core_layout`.
|
||||
|
||||
What is still not known: the marker's meaning (constant across 27 levels, at
|
||||
exactly this position, unnamed), and the shrub body payload's encoding.
|
||||
exactly this position, unnamed). The shrub body payload's encoding is the next
|
||||
section.
|
||||
|
||||
## The shrub class blob: palette and billboard (2026-10-11, after the bound)
|
||||
|
||||
With every span bounded by the chunk container, the shrub class blob reads
|
||||
cleanly against the R&C1 format (`vendor/rerac/docs/formats/shrub_sky_rac1.md`,
|
||||
derived from the Wrench sources). R&C1's shrub class header keeps the same
|
||||
field offsets R&C2 shows -- `o_class` at 0x24, a count at 0x28, the data offset
|
||||
at 0x2c -- so the reference says which bytes to look at and the disc says what
|
||||
they hold.
|
||||
|
||||
**Range measured:** all 358 shrub class spans over the 24 levels that have
|
||||
them, each bounded by its successor class or the located chunk container.
|
||||
Nothing outside the spans is read.
|
||||
|
||||
### The body is a 24-entry normal palette
|
||||
|
||||
Two things made this identifiable where three stride-fitting attempts had
|
||||
failed. First the shape: every body is 8-byte records whose last two bytes are
|
||||
zero, and R&C1's palette is documented as 24 records of exactly that shape
|
||||
(`s16` x, y, z, pad). Second the count: bodies hold 24, 26, 28 or 30 records
|
||||
with the extra ones **always all zero and always at index 24 or later** --
|
||||
never a live entry. So the body is 24 normals plus zero padding, and the
|
||||
padding is what distinguishes it from geometry.
|
||||
|
||||
The palette is real normals, and the test says how firmly:
|
||||
|
||||
- all 8,592 live entries have squared length within **65,535** of `32767^2`,
|
||||
and the bound is tight -- the largest observed deviation is exactly 65,535.
|
||||
That number is not fitted: it is `32768^2 - 32767^2`, which is what a unit
|
||||
vector scaled by 32768 produces when an axis-aligned normal quantises to
|
||||
`-32768` while the positive axis clamps to `32767`. Components run the full
|
||||
`s16` range (-32768..32767) and 1,765 of the 8,592 entries are axis-aligned,
|
||||
which is what a coarse direction codebook looks like.
|
||||
- a random `s16` triple has squared length spread about a *billion* either
|
||||
side of `32767^2`, so one entry in fifteen thousand would pass by chance.
|
||||
All 8,592 pass.
|
||||
- the clincher: 17 classes carry a palette of 24 identical `(0, 0, 1)`
|
||||
entries. A position or index table cannot produce that; a normal palette for
|
||||
a flat-facing class must.
|
||||
|
||||
This is also where shrub lighting comes from -- R&C1 lights each instance
|
||||
against its class's 24-direction palette. Unlike the tfrag's per-vertex light
|
||||
records, which store azimuth/elevation *indices* into the overlay's sine table,
|
||||
the shrub palette stores the directions outright, so the overlay table is not
|
||||
needed for shrubs.
|
||||
|
||||
### The header, byte by byte
|
||||
|
||||
**Range measured:** the 0x30 bytes of every span.
|
||||
|
||||
| offset | field | measured |
|
||||
|---|---|---|
|
||||
| 0x00 | f32[4] `sphere` | centre (xyz) + radius (w). Finite on all 358, **radius positive on all 358** (1.72..41.57), centres near the class's own origin |
|
||||
| 0x10 | f32 `size` | 3.0..20.0, positive on all 358 |
|
||||
| 0x14 | u16 `mode_bits` | **0 on 342 spans, 4 on 16** |
|
||||
| 0x16 | u16 `scratch` | **0 on all 358** (asserted by the parser) |
|
||||
| 0x18 | u32 `reserved` | **0 on all 358** (asserted by the parser) |
|
||||
| 0x1c | u32 `billboard_offset` | 0 on 333 spans, in-span on 25 |
|
||||
| 0x20 | f32 `scale` | 0.0044..1.8, positive on all 358 |
|
||||
| 0x24 | u32 `o_class` | equals the class entry's own, 358/358 |
|
||||
| 0x28 | u32 `count` | 1..55, meaning not established |
|
||||
| 0x2c | u32 `data_offset` | inside the span, 358/358 |
|
||||
|
||||
Two corrections this produces, both recorded because the old readings were
|
||||
confidently wrong rather than merely unverified:
|
||||
|
||||
- **"reserved is 0 or a denormal" was the wrong shape.** The 16 spans with a
|
||||
non-zero value there have `mode_bits = 4` and `scratch = 0`, not the other
|
||||
way round. As an `f32` the word at `+0x14` is still a denormal, so the PS2
|
||||
reads zero either way -- but the field is a flag word, not a magnitude, and
|
||||
the demo is the point: the EE's FPU has no denormals, so `04000000` and zero
|
||||
are the same value to the hardware.
|
||||
- **"bounds" is a bounding sphere, not a min/max box.** The old reading tested
|
||||
`min.xyz/max.xyz` ordering, found it on only 146 of 358 spans, and stopped
|
||||
naming the field. It is centre + radius, as in R&C1: the fourth float is
|
||||
positive on all 358 spans and the centres cluster near the class origin
|
||||
within +-24, which is where a class-local sphere belongs. The 146/358 figure
|
||||
was a property of the wrong question.
|
||||
|
||||
### The billboard record
|
||||
|
||||
The billboard offset at `+0x1c` is the strongest self-identification in the
|
||||
header after `o_class`, because it is **confirmed from a second file**: it is
|
||||
non-zero on exactly the 25 classes whose core-index entry carries billboard
|
||||
texture fields, in both directions -- 25 records and 25 entries, pairing
|
||||
correctly. R&C1 states the same correspondence ("a class has a billboard record
|
||||
exactly when its core-index billboard texture has width != 0").
|
||||
|
||||
At the offset sits R&C1's `ShrubBillboard` shape, register block and all:
|
||||
|
||||
| offset | field | measured |
|
||||
|---|---|---|
|
||||
| 0x00 | f32 `fade_distance` | 5..1000; 512 reads as 0 under R&C1's `trunc(fade) & 0xff`, i.e. billboard-only |
|
||||
| 0x04 | f32[3] | **not established** -- see below |
|
||||
| 0x10 | TEX1_1: K, MMIN, address | K -233..-168, MMIN 4 on all 25, address 0x14 |
|
||||
| 0x20 | TEX0_1: data_lo, address | index 1..4, address 0x06 |
|
||||
| 0x30 | MIPTBP1_1: address | 0x34, data words zero (filled at level load) |
|
||||
|
||||
The three floats at `+0x04..+0x10` are the part that does *not* carry over.
|
||||
R&C1's are `width`, `height` and `z_ofs` in class units; R&C2's are an order of
|
||||
magnitude too large and too varied to be dimensions of a 32..128 px billboard,
|
||||
and they **repeat byte-identically across levels**: 16 distinct triples among
|
||||
the 25 records, with one appearing on four classes across levels 1, 4, 12 and
|
||||
21. Dimensions would not do that; shared art assets would. They are handed back
|
||||
unnamed.
|
||||
|
||||
Two further facts about them, measured: the whole record repeats with them --
|
||||
three classes on different levels carry identical `fade`, floats, K and scale
|
||||
-- and the class entries they belong to carry identical `sphere` too. So the
|
||||
blobs themselves are shared between levels, which is the same behaviour the
|
||||
chunk container showed with its byte-identical tables.
|
||||
|
||||
### Still open
|
||||
|
||||
- The three billboard floats, `size`, `mode_bits`, and `count`.
|
||||
- **Where shrub mesh geometry lives.** R&C1 keeps the class's mesh packets in
|
||||
the blob between the header and the palette; R&C2's blob is header, palette
|
||||
and optional billboard only -- 24 normals and nothing else. The 333 classes
|
||||
without a billboard still have to draw a mesh, and it is not here. The
|
||||
candidates are the chunk container that follows shrub and the gameplay
|
||||
extent, and neither is decoded.
|
||||
|
||||
+12
-7
@@ -217,13 +217,18 @@ Phase 1 hands over the extents; this phase works out what each one contains.
|
||||
- [ ] `tie` — instanced environment geometry, fat vertices
|
||||
- [~] `shrub` — class table done (48-byte entries, not 32; reading shrub at
|
||||
32 still gives 231 sane-looking entries of 358, which is the hazard).
|
||||
Geometry header parsed and both fields **checked**, not merely reported:
|
||||
`+0x24` equals the entry's own `o_class`, and `+0x2c` is an in-span data
|
||||
offset, on all 358 spans. Shrub does **not** use moby's part-table format.
|
||||
The body is **8-byte records, 6 varying bytes and 2 constant**, and now
|
||||
that every span is bounded by the located chunk container, all 358
|
||||
bodies are whole records — 9,600 in total. The payload's encoding is
|
||||
still unknown, and so is the 2-byte marker that closes the region.
|
||||
Geometry header parsed and checked field by field on all 358 spans:
|
||||
`+0x24` equals the entry's own `o_class`, `+0x2c` is an in-span data
|
||||
offset, `+0x16`/`+0x18` are zero, and `+0x1c` names a billboard on
|
||||
exactly the 25 classes whose core-index entry carries billboard texture
|
||||
fields. Shrub does **not** use moby's part-table format. The body is a
|
||||
**24-entry palette of unit-length s16 normals** (squared length within
|
||||
65,535 of 32767² on all 8,592 entries — a random triple would miss by a
|
||||
billion) plus zero padding records; the billboard record carries K, MMIN
|
||||
and the TEX0/MIPTBP addresses as R&C1's does. What remains open: the
|
||||
three floats in the billboard record, `size`, `mode_bits`, `count`, and
|
||||
**where the mesh geometry lives** — R&C1 puts the mesh packets in this
|
||||
blob and R&C2's blob holds only the palette.
|
||||
- [x] `moby` — per-class model geometry. A class span opens with a `u32`
|
||||
pointing at a part table of 16-byte records; parts tile exactly, with
|
||||
`vertex_offset + 16 * vertex_count == next_part` holding for **43,467 of
|
||||
|
||||
Reference in New Issue
Block a user