diff --git a/docs/plan/level-assets.md b/docs/plan/level-assets.md index 2207d0e..5350301 100644 --- a/docs/plan/level-assets.md +++ b/docs/plan/level-assets.md @@ -131,86 +131,50 @@ Header offset 0x10 = 2,192,576. Opens `0 0 0 0 0 0 6 0 0 0 0 0 5 0 0 0 40 0 0 0 — small values including 64 and 192, which look like sky-shell counts and dimensions rather than offsets. -## `tfrag` located: terrain is `[0, occlusion)` +## `tfrag` region: `[0, occlusion)` -The header's field at 0x08 is **zero on all 27 levels**, and 0x0c, 0x10, 0x14, -0x60, 0x64 and 0x68 are strictly increasing on every level. Reading 0x08 as the -terrain offset and 0x0c as its end gives: +The header's fields at 0x0c, 0x10, 0x14 and 0x60 are **offsets**, and they are +strictly increasing on all 27 levels. The differences between them give region +sizes that are plausible on every level: -```text -tfrag [0, occlusion) level 0: 2.12 MB of terrain -occlusion [occl, sky) -sky [sky, collision) -collision [collision, textures_base) -``` - -So the terrain is the first region of `core_data`, and `tfrag = 0` is a real -value, not a missing field. - -### What the terrain region contains - -The opening bytes are a small header, after which the data is a repeating -**64-byte record** starting at 0x40: - -```text -0x40 f5 ff 6d 48 01 00 77 48 cb a3 db 47 58 44 50 46 40 d0 00 ... -0x80 51 89 6b 48 9c 15 87 48 c2 57 b5 47 9a 24 fa 46 20 df 00 ... -0xc0 e3 a9 46 48 01 40 a4 48 00 00 f4 47 de b0 d0 46 50 ee 00 ... -``` - -What is established about these records: - -- Byte columns 3, 7, 11 and 15 are **`0x48` in every record**. Those are the - high bytes of four `s16` values, i.e. a fixed Y coordinate of 0x4848-ish on a - planet whose radius is about 2,187 units. -- Columns 16 and 17 increase steadily across records (0x40, 0x50, 0x60 … and - 0xd0, 0xdf, 0xee …), consistent with a grid walk. -- Columns 48–51 hold a repeating `ffff00ff` pattern, which is characteristic of - GS packet headers rather than vertex data. -- Bytes at 0x4c–0x4f repeat `ff 00 ff ff` on **every** record. - -### The record layout, as far as it is established - -```text -382 records of 64 bytes, from 0x40 to 0x5fc0 -``` - -That count is exact: 383 breaks the pattern at 0x5fc0. Per column, over 300 -records: - -| Column | Value | Notes | +| Region | Range | Level 0 size | |---|---|---| -| +3 | `0x48` | 300/300 records. A marker, not data: the high byte of the `s16` at +2 | -| +19, +20, +21, +23, +43, +45 | `0x00` | 300/300 | -| +52..55 | `ff 00 ff ff` | 300/300. A fixed signature | -| +2, +6, +10, +14, +18 … | varying | the data | +| tfrag | `[0, occlusion)` | 2.12 MB | +| occlusion | `[occlusion, sky)` | 0.07 MB | +| sky | `[sky, collision)` | 0.42 MB | +| collision | `[collision, textures_base)` | 0.90 MB | -**Coordinates are fixed-point `s16` with a scale of 8.** Established directly: -the value 17,496 appears in the first record and `17496 / 8 = 2187`, which is -the radius of a Ratchet & Clank planet to the unit. Values that are exactly -divisible by 8 occur in long runs elsewhere in the region too, which is what a -fixed-point coordinate array looks like. +Ruled out: reading 0x0c/0x10/0x14 as *sizes* laid end to end. The cumulative +total does not reach `textures_base` on any level, and the byte at the start of +each supposed region is not a plausible structure. The offsets model is the one +that holds everywhere. -**Not yet determined:** which column is which of x, y, z, and whether the 64 -bytes are one vertex, two, or a quad header plus data. A VIF packet -interpretation of the first tag (`0xfff5`, command `0x1f`, 53 words) runs off -its data after 216 bytes and does not chain, so the `ff 00 ff ff` signature is -more likely a record delimiter than a GS packet. +### What is in the terrain region -The next test is mechanical: take one 64-byte record, try every grouping into -3-component vertices, and keep the one whose resulting points all lie within a -sphere of radius ~2,187. That is a decidable question and does not need more -speculation. +**Textures are stored inline with the geometry**, not in a separate bank. A scan +for the PS2 texel pattern `18 18 18 80` (grey texels with the 0x80 alpha that +means "opaque" in the GS's convention) finds 6,739 occurrences spread across 138 +clusters, the first at `0xd9f8` — inside the tfrag region. So terrain carries its +own texels, and `textures_base` names something else. -Ruled out along the way: +Earlier claims in this file that have since been **disproven**: -- **Uncompressed `f32` vertex data.** The longest run of plausible coordinate - floats in the whole 16.9 MB is a run of zeros in the collision block. Terrain - vertices are packed. -- **`GS packet tags` at every position.** Byte frequencies in 0xE0..0xEF are - high but uniform, which is what compressed or packed data looks like, not a - discrete packet stream. -- **A uniform 48-byte record.** The pattern repeats at 64 bytes; 48 does not. +- ~~"The terrain is 382 records of 64 bytes."~~ The 0x48 byte at offset +3 is + simply the most common byte in the region (945 of 2000 sampled slots); it is + not a record marker. Over 33,091 slots at that position only 4,127 hold 0x48. +- ~~"Byte +3 is constant at 0x48 across 300 records."~~ True for the first 382 + records and false beyond, which is how the false pattern was found. +- ~~"Coordinates are fixed-point s16 with a scale of 8."~~ The scale-8 reading + is supported by one coincidence (17,496 / 8 = 2,187, the planet radius) but no + grouping of the record into triples yields points on a sphere of that radius. + Tested: every stride from 2 to 16 bytes, every starting offset, three and four + components. Best result 5.0% of points on-sphere, which is chance. + +So the record framing is **not recovered**. What is established is the region +layout and that terrain is vertex-soup packed alongside inline texels — which +means the vertex format must be identified from a longer run than a single +record, most likely by locating a run of vertices whose magnitudes are bounded +by the planet radius and whose Y values vary smoothly. ## Where the class tables are not diff --git a/docs/roadmap.md b/docs/roadmap.md index 53f3a03..8a0f37c 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -111,6 +111,12 @@ Phase 1 hands over the extents; this phase works out what each one contains. written up in `plan/level-assets.md`. Next: identify `tfrag` by its own structure instead of through the header. - [x] `occlusion` and `sky` offsets located and their first bytes read. +- [x] Region layout of `core_data` established: tfrag, occlusion, sky, collision + and `textures_base` are offsets, strictly increasing on all 27 levels. + Terrain is `[0, occlusion)` and carries inline texels. +- [ ] **tfrag vertex format: not recovered.** The 64-byte record framing and the + scale-8 fixed-point reading were both tested and disproven; see + `plan/level-assets.md` for what was ruled out. - [ ] `gameplay` — begins `WAD` + a per-level byte, and does *not* decode with the WAD LZ77 decoder. All 27 levels fail identically, so the input is wrong rather than the decoder. Ruled out: bit order, header length, the