ReaperOfVeriod/Going-Commando
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
234 Commits 1 Branch 0 Tags
36d8d2b00c65590255849697fd1c0a65ccf9b0e9
Commit Graph
93 Commits
Author SHA1 Message Date
ReaperOfVeriod 36d8d2b00c The frequency enrichment survives the redo; the disassembly does not 
Redone through overlay_bytes_at, the verified flat mapping:

    functions kept                     2,313
    moby_classes 0x00c0                  391   (displaced: 394)
    best neighbour (0x00c8)               15   (displaced:   4)

    bucket      n   0xc0   best neighbour   ratio    (displaced)
       0-15  1,033     73      2              36.5x     72.0x
      16-31    759    118      4              29.5x     29.8x
      32-47    283     89      2              44.5x     34.7x
      48-63    118     41      3              13.7x     19.5x
      64-79     55     32      2              16.0x     15.5x
      80-95     37     22      2              11.0x     13.0x

The reason it survives is worth stating, because it separates the two results. The
displacement was uniform -- every window shifted by the same amount -- so a
function's contents barely changed, and a frequency count over a window is robust
to that. A disassembly is not: it depends on the exact byte alignment, which is
what was wrong. Hence 394 becoming 391 with the same shape, while the 0x3a8060
walk and the 16-byte stride evaporated entirely.

So the investigation was not "all wrong". Precisely:

- the frequency enrichment is real, now on verified bytes, with a test that fails
  if it goes;
- the disassembly was never real, and the whole jal chain built on it is retracted;
- no loader has been located, and 0x2ee138 is VU microcode.

THE LESSON IS ABOUT WHICH MEASUREMENTS SURVIVE A MISTAKE

A count over a window tolerates a constant displacement. A decode does not. Sixteen
probes leaned on the kind that does not tolerate it, and reported each with the
same confidence as the kind that does. Nothing about the numbers distinguished them
at the time -- that is the whole difficulty, and why the verification in cd16485
was the missing step rather than a formality.

Next: the ~391 functions are a legitimate shortlist, but consuming them means
disassembly through tools/overlay-anchors, whose output is now checked against a
word the build pins.

286 passing with the disc present (23 + 19 + 66 + 156 + 22; disc binary 34.89s),
clippy clean.
 2026-10-14 06:23:49 +02:00
ReaperOfVeriod 4a826997f6 Redone on correct bytes: the disassembly chain is gone, 0x2ee138 is VU microcode 
Re-read through overlay_bytes_at, the verified flat mapping from cd16485.

0x2ee138 is VU1 microcode: five COP2 instructions and a jr $ra stub. Nothing you
jal into, so the jal read from 0x3a3d70 cannot be right either -- and that
instruction was itself read from displaced bytes and must be re-read.

0x3a8060 is not what cee17c9 said. On correct bytes there is no addiu r17, r18,
0xc0, no 16-byte-stride walk and no function prologue; the first word is opcode
0x1f, which is not a MIPS user opcode. cee17c9's central finding does not exist.

State: overlay_bytes_at verified; 0x2ee138 is VU microcode, verified; the 0x3a8060
walk retracted; the frequency and neighbour-enrichment numbers not re-measured and
not quotable as exact; the e1482ab jal chain unverified, built from displaced bytes
and pointing at now-known VU code.

What is left of moby_anim is nothing. No loader located, one frequency result to
redo from scratch on correct bytes, and no reason to expect it to survive.

285 passing with the disc present, clippy clean.
 2026-10-14 06:20:27 +02:00
ReaperOfVeriod e6b3e0b091 Root cause: the per-section indexing used throughout this investigation was wrong 
The verification f2d7e7e asked for. Reference case: the tool disassembles
0x2ee138 as 0x4bc00083; reading the flat overlay at 0x2ee138 - LIT_BASE gives
83 00 c0 4b, little-endian 0x4bc00083. Match. Correct mapping is
off = ram - LIT_BASE indexed into the whole overlay.bin.

What I used in every probe: off = addr - secs[6].dest indexed into secs[6].data,
and OverlaySection::data includes the 16-byte section header. Every read was
displaced.

Invalidated:
- cee17c9's addiu r17, r18, 0xc0 disassembly, the 16-byte-stride walk, and the
  'address not float' conclusion. Built on wrong bytes.
- the immediate-frequency and neighbour-enrichment numbers in 3ea5ab7 and
  a2f8c92. Scanned per-function windows, so uniformly displaced: ratios are
  probably robust, exact counts are not measured on the bytes they claim.
- e1482ab, d97e7df, f2d7e7e: the jal decoding was verified correct, so branch
  targets stand; what lives at them was read wrongly.

Survives intact: every chunk-offset measurement in the tie and overlay work,
which read core_data and the overlay flat and never went through the section
index.

Fix: never index secs[n].data by addr - secs[n].dest. Either use
overlay[addr - LIT_BASE], or add a helper that accounts for the 16-byte header so
call sites cannot forget it -- and verify it against a word the decomp pins, as
above.

This was available from the first probe and skipped sixteen times.

284 passing with the disc present, clippy clean.
 2026-10-14 06:14:14 +02:00
ReaperOfVeriod f2d7e7ecd4 d97e7df was wrong: my section offset was wrong, not the decoder 
Ran gc-overlay-anchors on 0x2ee138 as d97e7df directed. It reports 6 of 12 words
did not decode -- but the words disagree:

    tool:    0x2ee138  0x4bc00083
    my read: 0x2ee138  0xd8a10000

Different bytes at the same address. My offset into the section is wrong, so the
'opcode 0x36 is invalid' finding was about bytes never at that address. That
invalidates d97e7df's conclusion and its 'second decode error' framing: the
decoder was never the problem.

The tool's words are COP2 (0x12), VU1 vector-unit instructions, which capstone
will not render under plain MIPS32. So its 'did not decode' is expected for VU
code and says nothing either.

Four bugs in this investigation, all mine: un-masked branch target, transposed
address, invalid opcode read from wrong bytes, and the offset mapping itself.

Every one was a self-consistent probe reporting a clean number. The check that
catches all four is verifying the tool against a known-correct case before using
it to generate a finding -- which d97e7df recommended and I then skipped.

Nothing about 0x2ee138 is now known: not VU code, not data, not a function. The
address is the correctly decoded jal target; what is there is unestablished.

Next step is to fix and verify the address-to-overlay mapping before re-reading
anything. Until then no address computed by hand in this investigation is
trustworthy, including the 0x3a8060 walk that happened to decode.

284 passing with the disc present, clippy clean.
 2026-10-14 06:01:54 +02:00
ReaperOfVeriod d97e7df41f Second decode error in the same investigation; stopping rather than trusting this decoder 
Following e1482ab I disassembled the function supplying r2 and made two errors,
both mine. First I read the jal target as 0x2ee178 when it is 0x2ee138 --
0x0bb84e << 2 is 3,072,312. Second, at the correct address the bytes do not
decode: d8a10000 is opcode 0x36, which is not a valid MIPS major opcode for
user-space code. The coherent readings are that this is data, that the decomp's
extent is mis-bounded, or that the jal target is still wrong.

Stopping here deliberately. An un-masked branch target that reported a false
negative, and a transposed address that produced plausible-looking garbage, are
enough to make anything else from this decoder untrustworthy. The next step is not
another immediate scan or another hand-decoded disassembly -- it is
tools/overlay-anchors, which links capstone, verified against a known-good
function first.

The established chain, with nothing inferred beyond it:

    0x3a3d70  jal   0x2ee138     target decoded, callee unidentified
    0x3a3d84  jal   0x3a7c30     r2 = return value of the above
    0x3a7c30                     580 bytes, never writes r2
    0x3a7d88  jal   0x3a8060     passes r2 through
    0x3a8060  addiu r17, r18, 0xc0  with 0xd0 and 0xe0; 16-byte-stride walk

What 0x2ee138 returns is unknown, and it is the single remaining link between the
program and moby_classes.offset.

284 passing with the disc present, clippy clean.
 2026-10-14 05:56:00 +02:00
ReaperOfVeriod e1482abd75 Correct my own decode bug: there is a caller, and r2 is a return value 
349b498 reported that 0x3a7c30 has no jal callers. That was wrong and the bug was
mine: the scan built J-type targets as ((instr & 0x03ffffff) << 2) | 0x0c000000
without masking to the upper nibble, adding 0x0c000000 to every target so nothing
matched.

Corrected: exactly one jal caller, at 0x3a3d84. Still no j tail-calls, still not a
vtbl handler, still not a pointer in any level-0 data file.

Tracing back, r2 is never written between the call and the use:

    0x3a3ce4  lw    r3, 0x1500(r2)
    0x3a3d70  jal   0x2ee178
    0x3a3d84  jal   0x3a7c30

so r2 is the return value of 0x2ee178, passed through as the first argument. The
chain is now 0x2ee178 returns a pointer -> 0x3a7c30 -> 0x3a8060 -> addiu r17, r18,
0xc0.

Next question is one function: does 0x2ee178 return core_data + moby_classes.offset?
If so the reading is confirmed and the loader is located; if not, the 0xc0 in
0x3a8060 is an unrelated struct member.

Also recorded in docs: a negative from a hand-rolled decoder is worth nothing
until checked against a known case. jal 0x3a8060 decoded correctly under a
different constant in the same probe and I did not notice.

284 passing with the disc present, clippy clean.
 2026-10-14 05:47:29 +02:00
ReaperOfVeriod 349b498ddd r2 is never written in the enclosing function, and it has no callers 
The trace eca1a16 asked for. The call site 0x3a7d88 sits in the function at
0x3a7c30, 580 bytes.

r2 is never written there: scanning all 580 bytes for any instruction storing into
r2 finds none. So it arrives from outside the function.

And 0x3a7c30 has no jal callers anywhere in level-0 .text, and is not a lvl.vtbl
handler -- level 0's vtbl has 233 records and 185 distinct handlers, and neither
0x3a7c30 nor 0x3a8060 is among them.

That rules out the tidy story, which was the obvious guess given r2 arrives
unexplained: the vtbl dispatcher calling it with a class pointer in v0. What
remains is an indirect jalr call, a dispatch table in data, or code that is never
called. Not distinguished; the question for anyone continuing is whether
0x3a7c30 is a jalr target and from where.

The chain is now ? -> 0x3a7c30 (no static caller) -> 0x3a8060 -> addiu r17, r18,
0xc0. Every remaining question is about control flow and register provenance, not
counts, which is where this stretch's method stops working.

284 passing with the disc present, clippy clean.
 2026-10-14 05:44:46 +02:00
ReaperOfVeriod eca1a16b17 Call sites: one caller, and it does not compute base + 0xc0 
0x3a8060 has exactly one call site, at 0x3a7d88:

    0x3a7d84  addiu  r18, r2, 0x202d      r18 = r2 + 0x202d
    0x3a7d88  jal    0x3a8060

No 0xc0 in the eight instructions before the call, and r18 is built from a
different offset. So the 0xc0 inside the callee is a field offset applied to a
pointer the caller already built, not a chunk offset the caller resolved.

Two readings remain: r18 is core_data, making r18+0xc0 equal &moby_classes, which
fits since level 0's moby_classes.offset really is 0xc0; or 0xc0 is a member
offset in an unrelated structure and the match is coincidence.

Settling it needs the provenance of r2 at the call site -- real dataflow, not
counting. That is where this stops.

Recorded in docs/plan/level-assets.md. The previous commit's doc edit failed
silently on a line-break mismatch, so this appends rather than replaces.

284 passing with the disc present, clippy clean.
 2026-10-14 05:41:48 +02:00
ReaperOfVeriod cee17c95b1 Disassembled: 0xc0 is an address offset, and the walk is 16-byte strided 
Closes the ambiguity 05d9bf6 raised. At 0x3a8060:

    263200c0   addiu  r17, r18, 0x00c0     r17 = r18 + 0xc0
    263000d0   addiu  r17, r16, 0x00d0     r17 = r16 + 0xd0
    262400e0   addiu  r17, r4,  0x00e0     r17 = r4  + 0xe0
    3c013f80   lui    r0,  r1,  0x3f80     the actual float constant: 1.0f

0xc0 is an address offset, not a float constant. The float constant is in the same
function and is 0x3f80, where it belongs -- so the ambiguity does not arise here,
and the reading of 0xc0 as moby_classes.offset survives.

The three offsets are 0xc0, 0xd0, 0xe0: sixteen bytes apart. The function walks a
table of fixed-stride structures and indexes the field at offset 0xc0, which is
what a class-table lookup looks like.

WHAT THIS DOES NOT SETTLE

That 0x3a8060 is the moby class loader. It indexes a 16-byte-stride table at
offset 0xc0, which is consistent with a moby class walk, with a different
structure that also lives at 0xc0, and with ordinary field access on an object
handed in by the caller.

The entry code sets a 64-byte frame and saves r16 through r20, and the three
indexes come off three different registers -- r18, r16 and r4. So the walk is over
arguments, not over a global. That is informative in itself: a loader would more
plausibly be handed the chunk by its caller than re-derive core_data + 0xc0 on
every call. Which also means the chunk reference is in the *caller*, and that is
where the next step is.

NEXT STEP, narrow and concrete: trace where r18 and r4 come from, and find the
call sites of this function. If a caller does `addiu r, base, 0xc0` and passes it
in, that caller is doing the chunk lookup and this function is the per-class work
inside it.

Worth recording how long the ambiguity took to close. It was raised one commit ago
and resolved by reading twelve instructions. Five commits before that the same
constant was reported as a "real shortlist", and two before that as a size
artifact. Every one of those states was arithmetically correct; only this one was
the right question, and it took looking at the operands rather than counting them
to arrive at it.

284 passing with the disc present (23 + 19 + 64 + 156 + 22; disc binary 35.00s),
clippy clean.
 2026-10-14 05:40:08 +02:00
ReaperOfVeriod 05d9bf6cd7 The 72 concentrated candidates, and a float-constant ambiguity 
Looking at the 72 candidates with fewer than 16 distinct immediates, the sets are
full of values that look like lui high halves: 0x3f80 is the high half of 1.0f,
and 0xff40/0xff70/0xff80/0xffc0 are negative high halves. 0xc0 << 16 is 0x00c00000,
a small float.

So 0xc0 in these functions may be a float constant rather than a chunk offset.
The neighbour control cannot tell the difference, because 0xb8 and 0xc8 are
equally plausible float high halves and are rare for their own reasons.

Also recorded, and explicitly not as a finding: exactly 1 of 394 candidates
contains another chunk offset, and none of the 72 does. That sounds like evidence
of moby-specificity and is not -- the other chunk offsets are 0x1e80 and above and
are globally rare (0 to 2 functions each, measured in 3ea5ab7), so their absence
here is exactly what rarity predicts.

What survives is the enrichment fact from a2f8c92. What does not survive is
reading 0xc0 as a chunk offset. There are now two live explanations, a moby chunk
reference and a float constant, and only disassembly separates them.

Cheapest test: 0x3a8060, 148 bytes, six immediates. If the lui carrying 0xc0 is
paired with an ori it is a float; if it feeds an addiu computing base + 0xc0 it is
the chunk.

284 passing with the disc present, clippy clean.
 2026-10-14 05:36:13 +02:00
ReaperOfVeriod a2f8c924b4 moby_classes.offset enrichment is size-independent: 13x to 72x at matched size 
Runs the control 06531fb named. It settles the question that commit left open,
and the answer is that the enrichment is real.

Matching on immediate-set size, against the best of the sixteen adjacent values:

    bucket      n   0xc0   best neighbour   ratio
       0-15  1,082     72   0xb8:1             72.0x
      16-31    713    119   0xc8:4             29.8x
      32-47    291    104   0xb8:3             34.7x
      48-63    119     39   0xb8:2             19.5x
      64-79     59     31   0xc8:2             15.5x
      80-95     25     13   0xb8:1             13.0x

Enriched in every bucket, and the ratio does not decay as sets grow. That is the
opposite of a pigeonhole artifact -- a volume effect would shrink the ratio in the
big buckets, and it rises instead.

The load-bearing row is the smallest: 72 of 1,082 functions carrying fewer than 16
distinct immediates contain 0xc0, where the best neighbour appears once. A set that
small cannot produce that by count alone.

What this revives. The 300 candidates with 48 or fewer distinct immediates are not
noise: at matched size, neighbours hit 0.4% to 1.4% in those buckets where 0xc0
hits 35%. A function using fewer than 16 constants, one of which is 0xc0, uses
that offset deliberately. Those are the shortlist.

Why this took four commits to establish. 3ea5ab7 measured 394 against neighbours
1-14 and called it a shortlist. 06531fb showed the rate rises with set size and
said the 394 was a list of large functions. Both were right about their own
measurement and neither had the bucket-matched comparison, which is the only form
that separates "the constant is special" from "big functions contain more
constants". Twenty-two probes this stretch, and the thing that finally settled it
was not a better measurement but adding the right comparison to a measurement
taken three commits earlier.

Caveat unchanged: presence of an immediate is not proof the chunk is read there.
It is proof the constant is used, which is all either test asserts.

284 passing with the disc present (23 + 19 + 64 + 156 + 22; disc binary ~35s),
clippy clean.
 2026-10-14 05:34:42 +02:00
ReaperOfVeriod 06531fb66c The 394 does not survive: 0xc0's rate rises with immediate-set size 
Ran the size-independent statistic db1e69c asked for, and it confirms the
confound. Bucketing level-0 functions by distinct-immediate count, the rate
containing 0xc0 is:

    0-15   72/1082    6.7%      64-79   31/59   52.5%
    16-31 119/713   16.7%      80-95   13/25   52.0%
    32-47 104/291   35.7%     112-127   3/7    42.9%
    48-63  39/119   32.8%     624+      1/1   100%

Monotone from 6.7% to over 50%. Pigeonhole: a function with hundreds of distinct
immediates has more chances to contain any particular value.

So the 394 is largely a list of large functions and '394 of 2,323' was never a
shortlist. db1e69c was right; 3ea5ab7's framing was wrong and is corrected here.

This does not retract the neighbour result. 0xc0 being enriched over 0xb8-0xc8,
and 0xc0 being enriched in bigger functions, are separate claims and both hold.
What it retracts is the inference that the 394 are candidates for anything.

The control that would settle it: 0xc0's rate against a specific neighbour's rate
within the same set-size bucket. Not measured, and named so the next attempt
starts there rather than re-ranking by size.

No test -- the bucket rates are the finding, and an assertion on any single
threshold would be arbitrary.
 2026-10-14 05:32:34 +02:00
ReaperOfVeriod db1e69c0ab Ranking the 394 by size is backwards; record the confound 
Ranking the moby_classes candidates largest-first is the wrong first cut. The
top entries are 34,316 and 26,432 bytes with 631 and 421 distinct immediates,
median candidate 852.

A function with 631 distinct immediates is far more likely to contain any given
value, 0xc0 included, than one with 80, purely by pigeonhole. Sorting by size
therefore selects for the functions most likely to be false positives, and the
34,316-byte outlier is probably a mis-bounded extent -- the decomp corrects
three level-00 entries -- rather than a 34 KB loader.

The 394 needs a size-independent statistic: whether a function contains 0xc0 more
often than its immediate count predicts, not whether it contains it at all. Not
measured.

Recorded as a negative about the method rather than committed as progress. No test
-- there is no assertion here to make.

283 passing with the disc present, clippy clean.
 2026-10-14 05:30:07 +02:00
ReaperOfVeriod 3ea5ab799f moby_classes.offset is referenced by 394 functions; its neighbours by 1 to 14 
First positive result of the loader hunt, and it exists only because of the
control that two commits ago I said was missing.

Across 2,323 level-0 functions of at least 128 bytes -- extents from the decomp's
functions.tsv -- the immediate 0x00c0, which is moby_classes.offset, appears in
394 functions. Every value adjacent to it appears in 1 to 14.

    0xb8:8  0xb9:2  0xba:2  0xbb:2  0xbc:3  0xbd:5  0xbe:5  0xbf:3
    0xc1:3  0xc2:3  0xc3:2  0xc4:1  0xc5:2  0xc6:3  0xc7:1  0xc8:14

This is what 27891ce was asking for. 0xc0 is small and small immediates are
common -- 17,699 occurrences across the program, against 200,784 for 0x0001 -- so
the raw frequency was worthless. The neighbours have the same magnitude and none
of the frequency, which is the only reason 394 means anything rather than being an
artifact of the value being short.

A second control points the other way, so the two bracket it: the other five
level-0 chunk offsets are larger values and are referenced 0 to 2 times each.
Shrub_classes 1, tfrag_textures 2, moby_textures 2, tie_textures 0, shrub_textures
2. The chunk offsets a loader would use are referenced about once. moby_classes is
referenced 394 times. The program leans on this constant heavily.

The previous commit's 410 candidates were not a shortlist. These 394 are, because
the number is now separated from magnitude rather than merely stated.

WHAT THIS DOES NOT SAY

An immediate appearing in a function is not proof the chunk is READ there. It is
proof the constant is used, which is the claim the test asserts and the only claim
it makes. It does not say which of the 394 is the loader, or that the loader is
among them. Stated plainly because the 394 is the kind of number that gets
quoted without its caveat and then built on.

Also measured, negative: zero functions contain both the moby_classes and
tie_classes offsets, so nothing indexes both chunk tables by immediate. The moby
loader does not reach the tie table the same way, if it reaches it at all.

Boundaries: level 0 only, "level 0" meaning the function's first listed place is
level 00, and a 128-byte size floor -- 59957a0 recorded 20-byte "functions" that
are mis-bound extents.

283 passing with the disc present (23 + 19 + 63 + 156 + 22; disc binary 35.23s),
clippy clean.
 2026-10-14 05:28:34 +02:00
ReaperOfVeriod 59957a02a2 Attribute the 0xc0 immediate sites to functions: 410 candidates 
The decomp's functions.tsv makes this cheap -- 3,835 level-0 functions with start
and size, so each immediate-bearing instruction is attributable to its container.

410 of 3,835 functions (10.7%) contain one; sizes 20 to 34,316, median 808.

Ten per cent is what a common small immediate buys, so this confirms 56b00fb
rather than narrowing anything. 410 is not a shortlist. What it does establish is
that the attribution works and the decomp's extents are usable for it.

The loader hunt now has a concrete next step: of those 410, the ones that also
touch a class span. That is dataflow, not counting, and it is where this resumes.

Caveat recorded: a 20-byte 'function' holding the immediate is a mis-bound extent
whose tail happens to include another function's instruction. Sizes need a floor
before the set is used.

No test -- an attribution count with no assertion behind it would only make the
410 look like a finding.
 2026-10-14 05:23:14 +02:00
ReaperOfVeriod 56b00fb305 Fix the immediate scan by filtering on opcode; still no signal 
Saying the previous method 'cannot tell' was right but the reason deserved a step
rather than a hand-wave: it is fixable by counting immediates only where they are
immediates, which needs an opcode filter and no disassembler.

All 27 levels: 11,911,320 instructions, 2,985,897 carrying an immediate
(addiu/lui/ori/xori/andi/sltiu), 13,813 distinct immediate values.

    moby_classes 0x00c0   17,699
    tie_classes  0x1e80       17

And now there is a control the raw version never had -- the program's most
frequent immediates are 0x0001 (200,784), 0x001b (149,259), 0x0010 (137,523),
0xffff (98,312). 0xc0 at 17,699 sits below the top of that small-value cluster
rather than standing out, so it is not a distinctive reference.

Result unchanged from the raw scan and now defensible: no signal. The loader is
not findable by frequency, and the remaining route is structural dataflow --
functions touching both the chunk offset and a class span.

No test: asserting that an offset is unremarkable is not an invariant worth
locking, and the two numbers are already recorded with their denominator.
 2026-10-14 05:17:47 +02:00
ReaperOfVeriod 27891ce5b4 Cross-reference scan by raw immediate: the method cannot tell 
Sought the moby class loader by counting raw u16 occurrences of each chunk offset
in .text. moby_classes 0xc0 hits 49,230 times, tie_classes 0x1e80 hits 111 --
and that contrast means nothing.

Small u16 values are simply common: the most frequent value in .text occurs
1,834,535 times. A two-byte value that is frequent by construction beats a rarer
one regardless of what either references, so the comparison has no shared
denominator. Rule 3a again, one level below where it has bitten before -- not the
measurement being wrong but the comparison being uninterpretable.

Recorded as a methodological negative rather than a finding. Doing it properly
means counting immediates only where the instruction decoder says they are
immediates, which is what tools/overlay-anchors exists for.

No test added: a test asserting a meaningless comparison would make the next
reader trust it.
 2026-10-14 05:15:07 +02:00
ReaperOfVeriod 743107e864 moby_anim: the class table is not the loader 
Ruled out the cheap approach: moby_classes (offset 0xc0, count 238 in level 0)
holds float-shaped values, and only 120 of the first 4,096 u32s land inside the
overlay .text against ~259 expected by chance. Fewer than chance, so the loader
is not dispatched from a pointer here.

Not concluded what the 238 entries are -- the float reading is suggestive but was
taken at one stride on one level.
 2026-10-14 05:13:02 +02:00
ReaperOfVeriod 6d26da3630 The control fails my own claim: 0x1c/0x34 select nothing distinctive 
2ff2abb ended by implying that the class-header offsets point at identifiable
content. I said the next step would be a control. Here it is, and it fails the
claim.

Adjacent-equal-byte rate over 64 bytes, 1,864 classes:

    0x1c     0.12688
    0x34     0.12106
    control  0.14109    (the same offset + 0x2000)
    random   0.00391

All three are ~30x above the random baseline, so the region is structured data
and not noise. That part stands. But 0x1c and 0x34 are no more structured than an
arbitrary offset in the same blob, and 0x34 is slightly *less*.

So the hex fragments in 2ff2abb were three hand-picked examples from a structured
region. "They look like compiled shader programs with small-integer parameters"
said something about this part of core_data, not about what those fields address.
That is exactly the rule 4 failure -- a pattern that reads as meaningful with no
control to test it against -- and I wrote it anyway while explicitly declining to
name a format two paragraphs earlier.

RETRACTED: the suggestion that the offset targets are identifiable content.

Unaffected, because it was measured differently: the offsets are base-relative,
non-zero, in range on 1,864 of 1,864 classes, and always land past the class's
own tie packet blocks. What they address is still unknown. The earlier finding
that the packet's control region is not a tag stream also stands -- that was
measured by uniformity within the packet, not by comparison to elsewhere in
core_data.

A TEST THAT CAUGHT ITS OWN AUTHOR

The first version asserted |0x1c - control| < 0.02 symmetrically, and failed on
0x34 at a gap of 0.0200. The correct response was not to loosen the number until
it went green. The claim being guarded is "these offsets select distinctive
content", and a gap in the *other* direction does not support it -- both offsets
came out below control. So the assertion is one-sided, and the comment says why,
including that the symmetric form was the wrong shape.

That is the AGENTS 3b discipline working in the direction it is meant to: a
failing assertion is information about the assertion.

282 passing with the disc present (23 + 19 + 62 + 156 + 22; disc binary 35.16s),
clippy clean.
 2026-10-14 05:06:57 +02:00
ReaperOfVeriod 2ff2abbd32 Two class-header words are base-relative offsets pointing past the tie packets 
Chasing where the AD-GIF is, since f43a1c2 established it is not in the packet.
Two of the undecoded class-header words are offsets.

base + 0x1c and base + 0x34 land inside core_data on 1,864 of 1,864 classes, so
both are relative to the class base rather than absolute, and both are non-zero
on every class. Both also land **past** that class's own tie packet blocks on
every class, so they address per-class data held elsewhere in core_data rather
than the tie geometry.

That is consistent with f43a1c2: if the packet's own control region is not a tag
stream, the per-class draw state is referenced from somewhere else.

WHAT IS THERE, AND WHAT IS NOT CLAIMED

The bytes are structured, not random -- class 3 shows a fixed-stride table with
`80 01 00` recurring, class 2's field is a monotone byte table, class 5's has
`38 38 38` and `ff 00 ff ff`. They read like compiled shader programs with
small-integer parameters, or lookup tables.

**Not identified, and deliberately not named.** None of those three fragments has
been run against a control, and all three were picked because they looked like
something. That is precisely the finding rule 4 refuses, and having spent this
session on four retractions I would rather record an honest "structured,
unidentified" than a plausible format that has not earned it. If any of these
turns out to be a shader program the next step is a control, not a better-looking
hex dump.

ALSO SETTLED

0x24 is demonstrably not an offset: only 17 of 1,864 of its values land in range
for core_data. Recorded as packed data. 0x28, 0x2c, 0x38 remain high-entropy and
unexplained.

281 passing with the disc present (23 + 19 + 61 + 156 + 22), clippy clean.
 2026-10-14 05:03:34 +02:00
ReaperOfVeriod f43a1c2e6e bfc_distance is not the control region's length; the AD-GIF is not in the packet 
Follows 9bb04cc. The control region is now characterised, and the second half of
the answer is a correction to a field doc that has been wrong since the record
was first parsed.

WHAT THE CONTROL REGION IS

Per-record, not shared state. 12,294 distinct blobs across 13,042 records, mixed
within 2,410 of 3,549 classes, and different between LOD 0 and LOD 1 on all 500
classes that have both. It varies per primitive.

Partitioned by its own length, it is not uniform in character, and the pooled
count hid that:

    vert_ofs  48 bytes  (3 qw)      5 distinct blobs
    vert_ofs  64 bytes  (4 qw)  3,175
    vert_ofs  80 bytes  (5 qw)  4,309
    vert_ofs  96 bytes  (6 qw)  3,281
    vert_ofs 112 bytes  (7 qw)  1,523
    vert_ofs 128 bytes  (8 qw)      1

The shortest and longest are effectively constant -- five blobs, and one -- while
everything between is per-record unique. That is a shape, not a noise floor, and
it is invisible unless the measurement is partitioned by length.

THE CORRECTION

bfc_distance == 5 * shader_count holds on every record, and the field doc has
always read "quadwords of ad-gif data" -- which read as a packet-relative length
would make it the control region's size.

It cannot be that. bfc_distance is 5..20 quadwords; vert_ofs, the control region,
is 3..8.

Pooled, they overlap on 9,315 of 15,219 records -- 61%, a number describing
nothing. Partitioned by shader_count, as AGENTS 3a requires, it is exact:

    shader_count 1: 2,772 of 8,676 exceed  (32%)
    shader_count 2: 4,649 of 4,649 exceed  (100%)
    shader_count 3: 1,729 of 1,729 exceed  (100%)
    shader_count 4:   165 of   165 exceed  (100%)

Without exception for two or more shaders. For one shader it exceeds exactly when
vert_ofs is below 5, which is that 32%.

**The AD-GIF is not stored in the tie packet.** The packet is control, vertices
and scissor, in that order. bfc_distance is a per-record quantity tied to the
shader count whose meaning is still not established, and the "quadwords of ad-gif
data" reading of it is wrong. The doc now says so and says why, rather than
carrying a plausible description of a field whose meaning nobody has established.

This is the fourth retraction today and the second today found by partitioning
rather than by a test failing. The pooled number was not merely less precise than
the partitioned ones -- it pointed the other way, because 61% looks like noise and
100% looks like a boundary. Had I accepted the pooled figure I would have written
"sometimes" and moved on.

59 disc tests, 278 total across 23 + 19 + 59 + 156 + 22 with the disc present
(disc binary 35.31s), clippy clean.
 2026-10-14 04:57:41 +02:00
ReaperOfVeriod 9bb04cca7b The tie control region is not a VIF or GIF tag stream; and vert_ofs is mis-named 
Went after the GS's GIF packets in the tie AD-GIF region. Got a negative result,
a new universal invariant, and a retraction of my own first probe, which reached
the right answer for the wrong reason twice.

THE NEGATIVE RESULT

Tried the leading `5 * shader_count` quadwords of each packet (the
`bfc_distance` field's stated meaning) as VIF tags -- command in bits 0-6, size
in 7-14, mark in 15 -- and as GIF tags -- type in the top byte of word 1, length
in bits 0-2 of word 0. Both uniform. Over 13,042 records: 10,443 distinct values
of word 0, no GIF tag type above a count of 2. That is the control result rule 4
asks for, and it says stop.

THE RETRACTION

The first probe scanned `bfc_distance` quadwords, which for most records
overruns `vert_ofs` and runs into the vertex data. Pooling a control region with
vertex positions -- two populations with different structure -- gives a uniform
answer, so "not a tag stream" came out right from a measurement that was
measuring the wrong bytes. That is exactly the rule 3a failure, and I only caught
it because the hex dump I took next looked structured while the histogram said
random.

Re-measuring quadword 0 alone, on its own population, gives the same uniformity.
The conclusion survives; the reasoning did not, and both are recorded.

THE PARTITION, AND A NEW INVARIANT

The hex dump showed the packet layout directly:

    q[0 .. vert_ofs)            control        3..8 quadwords, peak at 5
    q[vert_ofs .. +vert_size)   vertices
    q[vert_ofs+vert_size ..]    scissor

and `scissor_ofs == vert_ofs + vert_size` holds on **15,219 of 15,219** records
across all 27 levels, including every one whose strip_count is non-zero. That is a
derived quantity stored explicitly, and it holds everywhere -- which is the check
that confirms the three regions really are laid out the way the existing packet
length formula assumed. It is now a hard check in invariants_hold(), which is
something the previous four could not be.

The fixture in tie.rs failed the moment I added it: it had vert_ofs 4 and
vert_size 0x20 with scissor_ofs 0x30, which is 4 + 0x24. The synthetic data was
never constrained by a relation that did not exist yet. Fixed to 0x24.

vert_ofs IS MIS-NAMED

It is not a vertex offset. It is the *length* of the leading control region, and
the vertex data begins at that many quadwords into the packet. The field name
stays because the on-disc order is fixed and everything already written about this
record says vert_ofs, but the doc now states what the value is and gives the
distribution.

STILL UNKNOWN

What the control region contains. Not standard DMA tags; possibly pre-merged GS
register writes in another framing, or per-class state rather than per-primitive
draw data. Recorded as an open question rather than guessed at.

23 + 19 + 58 + 156 + 22 = 278 passing with the disc present (disc binary 35.67s),
clippy clean.
 2026-10-14 04:51:43 +02:00
ReaperOfVeriod bbd8cdccab Phase 5 is Aranos, not Veldin -- Veldin is not on this disc 
Caught by the user, and it was wrong in four places: the Phase 5 heading, its
gate, its go/no-go line, and the PCSX2 verification plan's "Level 0 (Veldin)".

Veldin is Ratchet & Clank's home planet in the 2002 game. Going Commando opens on
Aranos, and there is no Veldin anywhere in this disc. The error looks like it
carried over from the first game when the roadmap was drafted, which is an easy
slip to make and an unhelpful one to leave standing: a vertical slice defined as
"completable and matching PCSX2 frame-for-frame" is not something anyone could
have attempted or judged while pointing at the wrong planet.

Confirmed against the vendored decomp rather than taken on faith. Its
docs/LEVELS.md refers to level 00 as Aranos directly -- "corrects three measured
Aranos entries before fingerprinting ... These corrections apply to level 00 only"
-- so level 0 is Aranos in the reference material this project has been using all
along, and the roadmap disagreed with the decomp.

There is a real R&C1 reference nearby and it is worth keeping, because the
reasoning under the gate is sound and only the noun was wrong. Phase 5 argues that
spherical gravity is the thing most likely to break, because it invalidates the
movement and camera assumptions the engine inherits from Ratchet & Clank (2002).
That is about inherited code from the previous game, not about the level being
built, and the text now says so explicitly so the next reader does not have to
work out which game is meant.

No code change. Documentation only.
 2026-10-14 04:29:57 +02:00
ReaperOfVeriod 1fa1913e38 Reconcile the docs after the last three corrections 
The last three commits changed things the documentation still asserted in the
old direction. Fixed:

roadmap
  * 227 tests -> 238.
  * The level_overlay bullet contradicted itself: it said "the header words are
    not named" in the same breath as "+0x00 load base is 0x1A7C80". Now says
    +0x00 IS the load base and is named, +0x0c is excluded as the .lit size by
    sign (below file length on 16/27, by up to 8.9%) and remains unnamed, and the
    unnamed remainder is stated as such.
  * Section names aligned with the Phase 3 list (lvl.vtbl, lvl.camvtbl,
    lvl.sndvtbl).

core-data-regions.md
  * The claim-assertion audit still said the two PCSX2 claims were "NOT
    assertable here -- needs a savestate". They are asserted now, gated on
    GC_SSTATE, in the same file. Updated, with the two corrections that fell out
    of writing them recorded right there.
  * "that string is inside the collision routine" was never established. It is
    in the string pool, yes; the reference to it is not found. Reworded to say
    so, including the zero-hit result.

research log
  * field_00 now carries its name and the .lit/$gp/program context.
  * The field_0c falsification and the byte-order retraction are both recorded
    here as well, so the log and the format file agree.

Nothing left asserting the withdrawn anchor, and no stale test counts.
 2026-10-13 07:13:45 +02:00
ReaperOfVeriod 26a0918178 Gate the PCSX2 claims as tests; two corrections fall out of writing them 
The audit two commits ago listed three claims as "not assertable here -- needs a
savestate". With PCSX2 running they are assertable, so they now are. gc-trace
already exposed gc_trace::Savestate::ee_ram_owned(); gc-extract just needed it as
a dev-dependency. Both tests are gated on GC_SSTATE and skip cleanly without it,
so a normal checkout pays nothing.

Writing them corrected two claims, which is the main reason they were worth
writing as tests rather than staying prose.

1. The collision payload is NOT "byte-for-byte identical". The original claim
   rested on a 4 KiB probe; the test compares the whole 903,248-byte payload:

     payload                903,248 bytes at RAM 0xac6800
     first difference       +0xdc82c, i.e. 903,212 bytes in
     bytes that differ      6  (0.00%)

   903,212 of 903,248 bytes are verbatim and 6 are not, all in the final 36.
   "Verbatim" was an overclaim on a probe that never looked at the tail. The test
   now measures the whole payload and reports the extent, asserting only that the
   divergence is beyond the head.

2. The tfrag relocation base is the TABLE address, not the block base. The delta is
   8,683,584 = 0x848040 = block base 0x848000 plus table_offset 0x40 -- which is
   exactly what TfragHeader::data's own documentation says, "relative to the
   table". The earlier claim said "the block's RAM base", off by exactly the table
   offset.

   Also: +0x10 and +0x38 do NOT share one constant. The aggregate "1,665 fields
   relocate by one constant" was true only because +0x10 carried it; lumping them
   hid that. The test asserts +0x10 and REPORTS +0x38's delta rather than assuming.

The patch-rewrite figures also restated on aggregate rather than median, which is
why they read lower than the 7.80%/8.12% medians recorded earlier:

  patch bytes rewritten -- cannot morph 1.9% of 334, can morph 2.1% of 499

Same conclusion, stated on a total instead of a per-patch median, and asserted to
within 3 percentage points of each other.

Net: the audit table in docs/formats/core-data-regions.md now has no "prose only"
rows for disc-or-savestate-readable claims.
 2026-10-13 06:59:22 +02:00
ReaperOfVeriod 3ee0b3d0dc Disprove that morphing causes the tfrag byte changes: LOD selection does 
Read the whole tfrags block from RAM -- 4,379,200 bytes at 0x848000 -- and compared
every one of the 833 patches against its disc copy, grouped by whether it CAN morph:

  patches with NO morphing entries   334   differing: max 10.59%, median 7.80%
  patches WITH morphing entries     499   differing: max 10.49%, median 8.12%
  patches 100% identical to disc   0/334 and 0/499

The two groups are indistinguishable. Morphing is NOT what changes the bytes, and
this disproofs the claim two commits ago that the 8.6% was "morphed positions with
topology and UVs left verbatim". If it were, the 334 patches with no morphing
entries would be untouched -- and none of them is.

Nor is it a pointer relocation: the differing bytes fall in short structured runs
(257-263, 272-277, 305 on patch 0) whose u32 deltas share no common value.

The explanation consistent with everything measured is that the loader REGENERATES
each patch's VU packet stream for the LOD tier it selected. That happens to every
patch, morphing or not -- and it is why morphing shows no signal.

This also settles the verification route, which the previous commit left as "needs
the camera distance at the savestate". It does not, and no camera distance would
help: this memory image shows the LOD-selected form rather than the morph state,
so there is nothing in it to solve a weight against. Verifying the derivation needs
a captured frame at a known view distance instead.

What is established now, all measured on the full population rather than inferred
from one patch:
  - data and cube_ofs are relocated pointers, delta 0x848000, on all 833 records
  - every patch's bytes are rewritten, ~8%, by LOD-driven packet regeneration
  - morphing is not observable in this memory image
  - 833 of 833 patches parse; 499 of 833 have morphing entries; 43,613 vertex-info
    entries across the level
 2026-10-13 05:14:40 +02:00
ReaperOfVeriod aff977e8c5 Correct the tfrag pointer-field claim, and show morphing regenerates VU packets 
Two corrections to the previous commit, both from continuing the RAM comparison.

1. The previous commit said the RAM comparison "names two fields the disc-side
   work had left as data and an unnamed word". It does not. +0x10 is
   TfragHeader::data and +0x38 is TfragHeader::cube_ofs, both already named in the
   struct. What RAM adds is that both are POINTERS -- relocated by adding the
   block's RAM base -- rather than plain offsets. That is still worth knowing, but
   it is not a naming.

2. The previous commit attributed the 8.6% of rewritten bytes to "morphed
   positions with topology and UVs left verbatim". That is wrong, and patch 1 shows
   why. Comparing 4 KiB, disc 0xdf60 against RAM 0x855f60:

     bytes differing      354 of 4,096 = 8.6%
     contiguous runs      120
     longest run          8 bytes
     extent               +0xc1 through the end of the window

Patch 1 has 83 positions and 83 vertex-info entries, of which 13 morph. If
morphing rewrote position fields alone, the differing bytes would be a 13/83 slice
of a 498-byte position array -- a couple of dozen bytes in one contiguous region.
Instead 354 bytes differ, spread over 120 short runs across the whole window.

So the loader is not patching positions in place: it REGENERATES the VU packet
stream for the current morph state. That is what morphing is on this hardware --
VU1 GIF packets, not a CPU-side vertex array.

Consequence for verification, and the reason this is worth committing rather than
just noting: the plan of "map RAM vertices to disc indices and solve for the weight"
does not work against this memory image. The positions are inside regenerated
packets, so there is no stable per-vertex correspondence to read a weight out of.
Verifying the morph weight against the original needs the CAMERA DISTANCE at the
savestate, so the weight can be computed from the derivation and compared with the
rendered result, rather than read back out of RAM.

Population figures gathered on the way: 833 of 833 patches parse, carrying 43,613
vertex-info entries; 499 of 833 patches have at least one morphing entry, and
patch 0 -- the first record -- is base-only with none.

Also fixed a measurement error mid-run: I compared patch 1 at disc 0xdf60 against
RAM 0x855f00, off by 0x60, and got "every byte differs". The corrected base is
0x855f60.
 2026-10-13 05:08:08 +02:00
ReaperOfVeriod 93591a79de Verify the tfrag record framing against RAM: two pointer fields, morphed vertices 
The tfrags block header -- table_offset 0x40, tfrag_count 833, lod_distance 32.0
-- hits once in the level-0 savestate at RAM 0x848000, so disc offset 0 maps to
RAM 0x848000 and the block can be compared position for position.

Result 1: exactly two fields per record are rewritten. Comparing all 833 64-byte
table records field by field:

  records compared        833
  u32 fields differing    1,665 = 833 x 2, and no others
  their offsets           +0x10 and +0x38, on every record
  relocation delta        8,683,584 = 0x848000, on 833 of them

So +0x10 and +0x38 are block-relative pointers and the loader adds the block's
RAM base to each. Every other byte of all 833 records is verbatim. Six fields
differ by 0xF80000 and 0x1F00000 instead -- three records pointing into a
different buffer, recorded rather than averaged.

This names two fields the disc-side work had left as `data` and an unnamed word.

Result 2: behind the pointers, about 8.6% of bytes are rewritten. Following
record 0's +0x10 to disc 0xd040 / RAM 0x855040 and comparing 16 KiB: 14,977 bytes
identical, 1,407 differ = 8.6%. The differences spread across many field positions
rather than concentrating, which is what morphed POSITIONS look like -- topology,
UVs, counts and command lists stay verbatim while a fraction of the position bytes
move. 8.6% is about half of a 16-byte vertex's six position bytes, consistent with
roughly half the vertices in this patch mid-morph at the saved camera distance.

Not established: that the morphed bytes match resolve_vertex's formula. That needs
mapping RAM vertices to disc vertex indices and solving for the weight, which is
the next step -- but the framing is now known well enough to attempt it, and the
two pointer fields are confirmed rather than inferred.

This also answers the "is morphing real?" question that made the earlier tfrags
probe fail to match RAM: the tfrags block is not verbatim in RAM, because both its
table pointers are relocated and its vertex positions are morphed.
 2026-10-13 05:04:25 +02:00
ReaperOfVeriod d5e637930b Show the collision payload is verbatim in RAM, and retract the "not in RAM" claim 
This overturns the premise collision work was blocked on.

Searching the level-0 savestate for a probe from collision payload +1024 hits at
0xac6c00; a probe from +64 hits at 0xac6840. The difference is 0x3C0 = 960, which
is exactly 1024 - 64, so the payload sits at a fixed relative address -- RAM
0xac6800 for the disc address 0x27e800 -- and reading it back is byte-for-byte the
disc's own bytes:

  RAM  0xac6800  0b 00 21 00 b6 08 ad 08 a4 08 6b 08 38 08 63 07
  disc 0x27e800  0b 00 21 00 b6 08 ad 08 a4 08 6b 08 38 08 63 07

Nothing is reshaped. That is the opposite of the tie block, where the loader
relocates a pointer inside each record.

The retraction. The previous note reads: "level 1's 24 x 16 grid dimensions
appear in memory, level 0's 11 x 33 appear nowhere, so disc-side decoding has to
work against the reshaped form." Level 0's dimensions ARE in RAM, at 0xac6800 --
0b 00 and 21 00 are the first two bytes of the block, 11 and 33. The search that
missed them looked for 0b 00 00 00 21 00 00 00, the dimensions as a pair of u32s.
They are stored as two u16s, so that pattern never occurs.

So the negative result was an artifact of the search pattern, not evidence about
the data -- and it was then used as a premise for several rounds of disc-side
work. Both lessons were already written down in AGENTS.md and both were re-learned:
a negative from a byte search bounds the search, not the format; and the control
here is one command -- read the predicted RAM address and compare it to the disc --
which would have shown the verbatim copy immediately.

Also recorded while mapping it: the payload contains ascending u32 offset tables
from +1024 onward, and its first two bytes are the grid dimensions, so level 0's
grid is 11 x 33.

The cell layout itself is still undecoded, but the blocker is gone: disc-side
decoding no longer has to work against a reshaped form.
 2026-10-13 04:40:01 +02:00
ReaperOfVeriod 4e90c282ad Make the tie record relocation a command, and verify it on two levels 
The relocation rule from the previous commit was established by a script that
lived in a shell history, which means the next person would have had to
rediscover it. It is now `gc-trace verify-tie-records <unpacked-dir> <level>
<class> <lod> <src>`: resolve the class, parse the block, search the RAM image
for the loaded record table, report what it found, and exit 0 only when every
record's pointer has become table + word and every field byte is verbatim.

Swept over every (class, lod) slot of two levels, from PCSX2 savestates:

  level 0: 87 slots with a block, 87 verified, 0 mismatched, 61 absent
  level 1: 104 slots with a block, 104 verified, 0 mismatched, 78 absent

So the rule holds on every LOD block of two levels the game had loaded, which
is what makes the record layout a finding rather than a reading of one block.

Writing the command also caught an off-by-one in the parser it exercises:
TieBlockHeader::records started its slice after the 16-byte header, but the
header IS the first record -- the table runs from the block start to
data_offset, and a block with data_offset 0x50 holds five records, the first
being the header itself and its pointer naming where the payload begins. Found
because the tool's first run relocated nothing while the same check in a script
had passed; the unit test fixture was wrong in the same way and is fixed.

Two operational notes now in the format doc: the live emulator is not a stable
input (verifying level 1 over PINE matched 104 of 104 while the user was on
that planet, and the same command minutes later found no matching copy because
they had returned to level 0 -- a miss there is not a counterexample, so the
report says "no copy matched" rather than "failed"), and a class index past the
table count is an error rather than a mismatch, so a sweep should stop at the
level's class count rather than a round number.
 2026-10-12 15:24:48 +02:00
ReaperOfVeriod 2c4bbdc600 Use PCSX2 RAM to recover the chunk loader's transformation 
This is what RAM was needed for: nothing about the loader's transformation is
recoverable from the disc.

Anchor: a 32-byte probe from inside a chunk record in level 0's core_data, searched
in the level-0 savestate. It hits at 0x18538a8 and the record's first bytes at
0x185387e -- exactly chunk_base + record_offset, no rescaling. So the chunk payload
is copied VERBATIM; unlike collision and the shrub packet tables it is not expanded
or repacked.

Result 1: the offset table is rewritten to absolute EE addresses.
  disc: f000057c f00006b0 f00007e4 f0000914
  RAM:  0xf185387c 0xf18539b0 0xf1853ae4 0xf1853c14
The 0xf0000000|low24 tag becomes 0xf1850000|low24 -- tag replaced by the chunk's
load base, low 24 bits untouched. Consecutive deltas agree exactly (0x134, 0x134,
0x130), which is why the disc-side stride is approximate rather than constant.

Result 2: every record gains a constant 2-byte prefix 02 00, and the rewritten
table points at THAT rather than at the record data. Walking all 90 records from
the RAM dump: prefix 02 00 on 90 of 90; the disc's index bytes preserved unchanged
at +2/+3 on 90 of 90.

Result 3, the useful one: the byte at +2 takes exactly 0, 16, 32, ... 240 -- the
16 values index_rule predicts for step = 16, phase = 0. The mixed-radix record index
derived from the disc is corroborated on the running game's own copy of the chunk.
That is a disc-derived claim validated against the game, which is the strongest form
of check available here.

On disc a chunk is [16-byte header][10 zero bytes][tagged offset table][records].
In RAM it is [16-byte header][10 zero bytes][absolute-pointer table][records each
prefixed with 02 00]. The 02 00 is unexplained -- possibly a record type or an
alignment the loader needs -- but it is constant, so like the two index bytes it
carries no per-record information.
 2026-10-12 04:32:07 +02:00
ReaperOfVeriod d8b8ccc6d4 Follow the gameplay.bin pointers: they address position and direction arrays 
Where the pointers go. Following them shows what gameplay.bin indexes. The leading
table is 39 words, all of which land inside the file, and three clusters
dominate: near 150000, near 800000 and near 1050000.

The region near 797808 is position data -- 31 consecutive plausible floats that
group cleanly into xyz triples:

  246.948 201.015  73.867 |  88.240 186.000 302.598 | 107.712  12.723 183.125
  353.591 126.365  19.962 | 182.068 374.001 122.567 | 116.961 265.001 437.866

and the region near 797456 is direction data -- unit-length vectors:

  0.969 0.945 0.733 | -0.120 0.433 0.250 | -0.866 0.000 0.267

So the file's leading words are a table of pointers into float arrays, not a flat
list of small integers. The earlier reading of it as "u32 pointers holding
nothing" was too quick: it is a pointer table into data that is present.

Volumes remain unlocated. The position arrays are the right shape to bound a
volume and nothing yet shows the volume records themselves.
 2026-10-12 03:36:55 +02:00
ReaperOfVeriod b17ec17127 Rule out the moby_anim instance-table reading of the chunk types 
moby_anim was the one post-shrub chunk type with a concrete reason to exist: an
instance table would have to scale with the level's moby class count, so
comparing per-type record counts against that count was the obvious test.

It fails, and the way it fails is itself the result: per-type record counts are
level-independent. The same types report the same counts on every level tried:

  type_id     level0  level1  level2  level3
  0x007d7ec438   90      90      90      90
  0x00d3d8c4a9   90      90      90      90
  0x00429ec43d   77      77      77      77
  0x0053c8c4aa   77      77      77      77
  0x007626c4a4   50      50      50      50
  0x00fb88c462   47      47      47      47

Across all 27 levels there are 77 distinct chunk types, the largest at 2430
records total -- exactly 90 x 27.

What that rules out: nothing in this chunk set is a per-moby-class instance table,
because those counts would vary by level the way the moby class counts do. So
whatever holds moby animation data is either keyed by something other than the
class count, or it is not among these chunk types.

Also visible: the types pair up. 0x007d7ec438 and 0x00d3d8c4a9 both report 90 on
every level; 0x00429ec43d and 0x0053c8c4aa both report 77. Two chunk types
carrying identical, constant record counts on every level is a signature of two
views of the same fixed table rather than two independent tables that happen to
agree.

Still open: the pairings are described but not explained, and no type has been
named. The Moby class-ID enrichment from earlier stands separately -- 9493 hits
against 246.1 expected is real, and the class arrays are contiguous per type, but
that identifies where classes are referenced, not which chunk is moby_anim.
 2026-10-12 03:33:52 +02:00
ReaperOfVeriod 02ced5ee83 Parse the per-level overlay: header, constant block, level data 
level_overlay was the least-characterised file in the level set -- no parser, no
located structure. It now has gc_formats::overlay and a real-disc test over all 27
levels:

  overlay: 27 levels; field_00 1735808, field_08 1;
  field_04 37540..=45308, field_0c 2763912..=3027336
  (17 below the file length, 10 above);
  +0x10..0x1220 identical on all

The header is four words. +0x0c is not the file length: it was recorded as one on
the strength of the first levels measured, where it sat near the file end, but
across all 27 it falls on both sides -- below on 17, above on 10 -- so it is an
ordinary level-specific word. The test asserts this, being the claim most worth
pinning.

The common prefix is 4 bytes, not the whole header: +0x00 is constant at 1735808
while +0x04 and +0x0c vary. Everything from +0x10 to +0x1220 -- 4624 bytes -- is
byte-identical on all 27 levels.

Still not named: what any of the four header words means, and how the
level-specific tail is structured. The module reports them as fields.

Also records in docs/plan/pcsx2-verification.md that the 64-byte RAM stride from
the previous commit is disproved. A 415 KB dump of level 1 finds the longest such
run at 27 records, not 384, and those are identity matrices -- 27 of them, one per
level. It is a per-level transform table, not the collision grid, and the 20.0
reported there was a coincidence.
 2026-10-12 03:32:02 +02:00
ReaperOfVeriod bd012efe71 Find a 64-byte collision record stride in RAM, and record a savestate search pitfall 
Searching the savestates for the collision grid dimensions as a u32 pair -- level
0's 11 x 33 and level 1's 24 x 16 -- puts level 1's at several RAM addresses and
level 0's nowhere. So the grid dimensions are not held in RAM as adjacent u32s,
and the collision payload is reshaped on load, as the shrub packet tables are.

A nearby region of level 1's RAM does show structure. At 0x12e048c and
0x12e04cc, 64 bytes apart:

  28 01 00 00 | 92 ff 00 00 | 04 00 a0 41 | ...
     296        -110        20.0f

Searching for that shared 12-byte tag finds a consecutive run at a 64-byte
stride in slot 2:

  0x12df390
  0x12df3d0   +0x40
  0x12df410   +0x40

plus scattered singles elsewhere. Slot 1 has the tag at four addresses not on a
64-byte stride, so the two levels lay it out differently.

What this is: a 64-byte record carrying two small signed values and a float
reading exactly 20.0 -- the shape of a per-cell entry with coordinates and a
radius or half-extent, which is what the collision grid would hold.

What this is not: it is not decoded. Three consecutive records is not enough to
establish a stride, the tag matches only a few records rather than a whole grid,
and 20.0 may be coincidence. Level 1's grid is 384 cells; nothing here has been
shown to count to 384.

Method note: savestates are compressed on disk, so a byte scan of the file finds
nothing. All RAM search must go through gc-trace find --state, which decompresses
first -- scanning the file directly reports zero hits for a sequence that
demonstrably exists in memory.
 2026-10-12 03:27:14 +02:00
ReaperOfVeriod d3e1475f64 Characterise gameplay.bin as a flat pointer table, and record volumes as unlocated 
Range measured: the first 40 u32 and the whole file, on levels 0 and 1 --
1872256 and 2006400 bytes.

Result: the file is a flat array of u32, opening with a small count then a run of
ascending offsets, terminating on a zero:

  level 0  336 797472 798832 799136  464 150672 150688 314800 479920 ...
           ... 1847600 1845456 797808 1829056 160 1849088 1249808 1847616 0

  level 1  496 799904 800128 800432  624 151152 151168 315680 481456 ...
           ... 1984496 1982400 800112 1966000 160 1988160 1303456 1984512 0

Both levels carry the same shape. The sentinel 160 appears at u32 index 34 on
both and the zero terminator at index 38 on both, with counts 336 against 496
and 464 against 624 at the head. The leading ~38 words are a structured header
whose layout is level-independent apart from its sizes.

The offsets cluster into groups -- near 800000, 150000, 1050000, 1800000 --
which is what a table of pointers into several buffers looks like.

Not decoded: what the pointers point into, what 160 is, and why the zero
terminator sits at the same index on both levels. No gc_formats module exists for
this yet; it is characterised, not parsed.

On volumes: nothing resembling volumes was found here. gameplay.bin is u32
pointers throughout and holds no float data in its first 64 words read as f32 --
59 of 64 are denormals, i.e. small integers reinterpreted. Volumes remain
unlocated on the disc; nothing in the regions examined so far presents itself as
one.
 2026-10-12 03:19:03 +02:00
ReaperOfVeriod 575d1a5bdd Correct the overlay strings from a table to a pool 
The previous commit described a "debug string table on 16-byte alignment". It is
not a table. Dumping the 32 bytes before each run shows most are the tail of the
previous string:

  +3104  "host0:r:/rc2/shock/wavetable.bin"   32 chars
  +3120  "ck/wavetable.bin"                   its own tail
  +3152  "/i5/shock/wavetable1.bin"
  +3168  "ble1.bin"                           its own tail

and the 32-byte prefixes are 327 distinct out of 332 -- essentially all different,
because they are just the preceding string's bytes. There is no per-record header
here. It is one contiguous pool of debug text, and the 16-byte alignment is a
property of how the pool was laid out, which makes continuations land on
boundaries and read as separate entries.

The count of 332 "strings" was therefore inflated by the same effect: it counts
continuations as entries.

The substance is unchanged and still useful -- the pool carries the game's own
debug text, including "Camera_CollPrimTest WARNING! - grid out of bounds! (RAR)",
which is direct evidence for reading the collision payload's two leading u16 as
grid dimensions, and the wavetable.bin paths, which bear on the sound bank
framing. Only the framing was wrong.

The pool is located and counted but not parsed, and there is no gc_formats module
for it. The +0x18/+0x20 names are section names but their structure is not
established, and it is not shown that they index anything.
 2026-10-12 03:17:50 +02:00
ReaperOfVeriod a5a33b9bc0 Locate level_overlay's sections and a 16-aligned debug string table 
level_overlay was on the open list with no parser and no located structure. It
now has a header naming two sections and a string table.

Range measured: overlay.bin on levels 0, 1 and 2 -- 2751268, 2763668 and
2928424 bytes.

Result 1: the file opens with section names.

  +000: 80 7c 1a 00 | dc 99 00 00 | 01 00 00 00 | 88 3b 2a 00
  +010: 00 00 00 00 | 00 00 00 00 | "crap\n"     | 00 00 00 00 00
  +020: "system misc" 00 00 00 00 00 00 00 00 00
  +030: 20 80 ff 40 | 00 60 cf 00 | 64 00 | 46 00 | 19 00 00 00

u32@+0x00 is 1735808 on all three levels, +0x08 is 1, and the bytes at +0x18 and
+0x20 spell "crap" and "system misc".

Result 2: a debug string table on 16-byte alignment. 332 such strings on level
0, 362 on level 1, 351 on level 2, first at +0x20 and last between 1.3 MB and
2.5 MB. They are the game's own debug text:

  host0:r:/rc2/shock/wavetable.bin
  host0:z:/i5/shock/wavetable1.bin
  unloaded sound bank %#010x
  Camera_CollPrimTest WARNING! - grid out of bounds! (RAR)
  render setup
  sky draw
  pre effects
  pre 2 effects

Camera_CollPrimTest WARNING! - grid out of bounds! (RAR) is direct evidence for
the collision reading. It is the game's own string, naming a grid and a
collision test, and it is why the two leading counts in the collision payload are
worth continuing to treat as grid dimensions despite not sizing the payload.

The wavetable.bin paths also confirm the sound bank is loaded from named files,
which bears on the SBlk/DAW framing.

Not done: the string table is located and counted but not parsed -- the records
preceding each string have not been identified, so there is no gc_formats module
for it yet. The +0x18/+0x20 names are section names but their structure is not
established, and it is not yet shown that they index anything.
 2026-10-12 03:11:25 +02:00
ReaperOfVeriod e58ab0e815 Correct the collision table's extent: 17 entries, deltas rise then fall 
+5932 is the table's start, not a point inside it: walking backwards, no
predecessor satisfies "ascending with a delta under 4096". The run is 17 entries,
not 19 -- the earlier 19 counted two entries past the jump.

The deltas are

  204 212 216 224 224 228 228 228 224 220 216 200 96 96 100 96

then a jump of 41484 to 190508. The deltas rise and then fall -- a symmetric
profile, not a monotone one. That weakens the prefix-sum reading: a cumulative
offset table has no reason to come back down, whereas cell contents that grow and
shrink as the table walks across a level would.

17 against a 75-cell grid, and 75 - 17 = 58. No arithmetic relating the two has
been found. The earlier guess that 19 related to 3x25 did not survive
re-measurement and is not carried forward.

Still open. 17 entries may not correspond to the grid at all -- the table may
cover something else, or a subset of cells. Resolving it needs either the same
offset found in other levels at a position the leading counts predict, or a
reason for 17. Neither exists yet, and this is where the measurement trail goes
cold on collision.
 2026-10-12 03:08:03 +02:00
ReaperOfVeriod b66e77855c Locate a per-cell offset table in collision, on level 4 
Range measured: longest strictly-ascending u32 run (delta under 65536) in each
level's collision payload, levels 0 to 5.

Level 4 is the clean case. Grid 3x25, payload 2660608, and at payload +5932 a
run of 19 entries whose values are all inside the payload, starting at 146012:

  deltas 204, 212, 216, 224, 224, 228, 228

The deltas grow monotonically, which is the signature of a prefix sum over
records of increasing size -- what a per-cell offset table looks like when cell
contents vary.

The others are not:

  level 0  11x33   +21232   17   8193, 24575, 8192, 8192   multiples of 8192,
                                                              coincidental on
                                                              dense data
  level 1  24x16   +41452   29   32771, 57339, 24575        wild, no trend
  level 2   1x53   +13752   49   24, 64, 396, 104          in-region but
                                                              irregular
  level 3  19x31  +667152   18   769, 257, 257, 257        values ~5.2e8, far
                                                              outside the payload
  level 4   3x25    +5932   19   204, 212, 216, 224        growing, in-region

Only level 4's run has both properties a prefix sum needs: values inside the
payload, and deltas that grow.

What this does not give: level 4's run is 19 entries against 75 grid cells, so
it is not one offset per cell -- either most cells share an offset, are empty,
or the table covers something other than the grid. Nothing measured picks between
those.

Next to check: whether 19 relates to 3x25 by any arithmetic, and whether the
same offset at +5932 exists in other levels at a position given by their leading
counts. Neither has been tried.
 2026-10-12 03:03:18 +02:00
ReaperOfVeriod 2569cc06ab Decode the collision payload's leading two counts 
Range measured: the first four bytes of the payload on all 19 levels that carry
a non-empty one.

Result: two u16 at payload +0 and +2, both non-zero on all 19, ranging 1..74 and
13..112, products 53 to 3256.

  level  0  11 x 33  =   363   payload   903248
  level  1  24 x 16  =   384   payload  2800496
  level  2   1 x 53  =    53   payload  2315296
  level  7  40 x 35  =  1400   payload  4049232
  level  9  74 x 44  =  3256   payload  2518352
  level 13   2 x 64  =   128   payload  3333520
  level 26  27 x 112 =  3024   payload   652176

Reported as two counts, not as a named grid. They have the shape of collision
grid dimensions -- bounded, both non-zero, products in the hundreds to thousands
-- but nothing measured ties the product to the payload size or to any per-cell
stride. Level 2's 1 x 53 giving a 2.3 MB payload and level 26's 27 x 112 giving
652 KB shows the product does not size the payload, so either cells vary wildly
in size or these are not grid dimensions. Naming them width and depth would be a
reading, not a measurement.

After the two counts comes a run of u16 that stays below 8192 for 30 to 58
entries depending on level, always an even count, and is not ascending. That is
where the per-cell table most likely starts, and it is the next thing to
resolve.

Exposed as CollisionHeader::first and ::second. They live in the payload rather
than the header, so CollisionHeader::region fills them in and the bare
CollisionHeader::parse leaves them zero -- documented at the call site so the
difference is not mistaken for an absence.
 2026-10-12 02:58:46 +02:00
ReaperOfVeriod 72bbd5da38 Rule out GS packet framing in tie, with a control that nearly fooled me 
Range measured: level 0, the 73 tie span bodies (bytes +0x40 onward), 2585856
bytes pooled. Per-position distinct counts, an autocorrelation scan against a
shuffled control, and a GS GIF tag test.

Result 1: the bodies are dense. Every one of the 16 slot positions takes 245 to
256 distinct values, byte 15 zero on 81.3% and byte 7 zero on 78.8%. The best
stride peak is 48 at +0.097, barely above 16 at +0.095 -- no period.

Result 2: no GS GIF framing at 16-byte alignment. Testing whether byte 15's low
three bits fall in 0..4, the GS tag range, gives 97.0%.

That looked like a strong hit until the control was done properly. A uniform
shuffle gives 62.6%, exactly 5/8, what a random byte predicts -- so the naive
comparison reads as a 34-point excess.

The distribution-matched null is 97.0%, identical to the observation. The whole
excess was that byte 15 is 81% zero and (0 & 7) <= 4 is always true. Recomputed
per alignment the null tracks the observation everywhere:

  align 0  97.0%    align 4  75.7%    align 8  96.7%
  align 2  75.5%    align 12 89.3%

This is the control rule earning its place a second time. A uniform shuffle is
not a matched control when the data is skewed, and AGENTS.md already says so;
the near-miss was reading 62.6% as a baseline without checking what the data's
own distribution predicts. The tie bytes concentrate on 0, 1, 2, 3 and 63 --
suggestive in isolation -- and still carry no framing.

What this rules out: the tie bodies are not GS GIF packet streams at 16-byte
alignment and carry no measurable periodicity. With every slot position taking
~256 distinct values they are dense data -- fat vertices, as the roadmap expects
-- and locating the vertex layout needs a reference for this game's instanced
geometry format, not another stride scan.
 2026-10-12 02:56:58 +02:00
ReaperOfVeriod 86ff2e6052 Measure tie spans: a constant first word and three quantized floats 
Range measured: all 1864 tie spans across 27 levels, read at each class entry's
geometry offset.

Method: read +0x00, +0x0c and the three f32 at +0x10, +0x14, +0x18, and count
distinct values per position.

Result:

  u32@+0x00 takes exactly one value across all 1864 spans: 0x00000100, 256
  +0x0c takes 83 distinct values, so it is not constant
  the three floats take 103 distinct triples built from 42 distinct values,
    and those values are round -- 4, 8, 9, 10, 12, 15, 16, 18, 20, 24, 25, 30
    with only 10 of the 42 exact powers of two

Two level 0 spans open:

  +000  00000100 00005070 000065c0 07050612
  +010  41400000 41c80000 42200000 00009460

  +000  00000100 00007ca0 0000a040 0605091e
  +010  41400000 41c80000 42200000 0000ddf0

This rules out the bounding-box reading. A box would take continuous values;
these come from a 42-value set of round numbers, so they are authored or
quantised quantities -- a grid, a size class, or LOD thresholds -- and nothing
measured separates those.

Correction: these floats were previously read as a bounding-box minimum, from
level 0's first two spans happening to share (12.0, 25.0, 40.0). That triple
holds on 266 of 1864 spans, not all of them, and the value set is quantised.

Not decoded: what the three floats are, what 0x100 at +0x00 counts, and what
the 83 values at +0x0c are.
 2026-10-12 02:52:45 +02:00
ReaperOfVeriod 32a24574be Identify data.bin as the level file descriptor, not geometry 
data.bin is the largest per-level file -- 13.3 MB on level 0, 4.2 to 21.9 MB
across the disc -- and had been characterised only as "4-byte structured".

Range measured: the first 64 u32 of data.bin on levels 0, 1, 2 and 5, compared
against the actual byte lengths of the other extracted level files.

Method: read the first 64 u32 and look for any equal to the size of
core_index.bin, gs_ram.bin, overlay.bin, gameplay.bin, sound_bank.bin or
core_data.bin.

Result: three exact matches, at the same indices on every level.

  u32 index 1  overlay.bin     2751268  2763668  2928424  2863676
  u32 index 3  core_index.bin    31600    34016    33808    23536
  u32 index 5  gs_ram.bin       518656   569344   513024   434176

Each is the exact byte length of the named file and each lands on the same index
on all four levels. Three coincidences at fixed positions is not a coincidence.

So data.bin is a descriptor for the level's other files, not geometry in its own
right. The roadmap's "the large one: 4.2 to 21.9 MB of geometry per level" entry
was wrong and is corrected.

The intervening indices 0, 2 and 4 hold values that are not file sizes and their
meaning is not established; they are plausibly offsets, but no cumulative-sum or
offset+size relation has been checked yet.

What follows the descriptor is still unknown, and the file is large enough that
the descriptor is unlikely to be all of it.
 2026-10-12 02:50:24 +02:00
ReaperOfVeriod 5610d173b0 Narrow the collision header to two fields plus a third used by five levels 
Range measured: all 27 levels that carry collision, reading the 0x40-byte
header each.

Method: read +0x00, +0x04 and +0x08, then check +0x08..+0x40 for zero.

Result:

  u32@+0x00 is 0x40 on 27 of 27 -- the header is self-sizing
  u32@+0x04 is the payload size, 20 distinct values across the disc
  bytes +0x08..+0x40 zero on 22 of 27

Five levels carry a large non-zero word at +0x08:

  level  7  1248048        level 12  2541616
  level  9  1766832        level 13  3376592
                      level 14  4060640

So the header is not two fields and 56 bytes of padding: there is a third field
most levels leave at zero and five use. Whether it is a size, an offset or a
second block's length is not established, and nothing measured here separates
those readings. Exposed as CollisionHeader::field_08.

Correction: the collision header was previously described as self-describing
across its full 0x40, which is true for 22 levels and overstates the case for
the other five.

Still not decoded: the payload. It contains ascending u32 runs on levels 0 and
2 -- level 0 at +280 with deltas 12, 12, 12 then a jump, level 2 at +13868 with
deltas 204, 204, 200, 196 -- but irregular deltas fit neither a flat table nor a
simple stride, and level 1's apparent run is a byte-shift false positive.
Nothing about the payload is asserted.
 2026-10-12 02:49:25 +02:00
ReaperOfVeriod 681e7e62ef Parse the sky block: 149 sub-objects with a bounding sphere, and correct the count field 
Range measured: all 27 levels, each sky region from the core index's sky offset
to the next region.

Method: read the 32-bit offset table at +0x20, bounded by staying inside the
region, and parse each sub-object it names.

Result: 149 sub-objects across 27 levels. Each opens with a count, then a
bounding sphere of four floats at +0x10, then a texture offset at +0x20 and
dimensions at +0x24/+0x26:

  +000: 26 00 00 00 | 00 00 00 00 00 00 00 00 00 00 00 00
  +010: e0 8d b2 c1 | 9f d6 f6 3d | ef fc fb be | 1c 61 4e 41
  +020: 90 99 05 00 | 1d 00 | 1d 00 | 00 00 e8 00 ...

Count 1 to 38, width 3 to 81, radius 1.95 to 14.56, and the radius is positive
on 149 of 149 -- the same sphere shape and the same reasoning as the shrub class
header's.

Correction to an earlier claim: the word at +0x0c is not the sub-object count. It
reads 5 on level 0 where the table holds 6 entries, 3 on level 1 against 4, 7 on
level 3 against 8, and 196617 on level 2 against 4. Across the disc
offsets.len() == count+1 holds on 10 of 27 levels.

The field is reported as SkyHeader::field_0c and nothing depends on it; the
offset table enumerates the sub-objects and is self-limiting. That earlier claim
came from level 0 alone, where 5 against 6 looked like a consistent off-by-one.
It is not an off-by-one -- it is a field that happens to be small on most levels
and is not a count at all.

gc_formats::sky parses the header and the sub-objects. The gated test asserts the
sphere is finite, the radius positive where positive at all, the dimensions
non-zero, and the sub-object list non-empty on every level.
 2026-10-12 02:43:28 +02:00
ReaperOfVeriod bbce492c91 Establish that the loader produces the RAM packet forms, not the disc 
Everything previously reported about five-block sub-lists and fixed-point
payloads was read out of the running game. Checking the same packets on disc
settles what the disc actually stores.

Range measured: level 0, all 29 packets named by the +0x40 tables of the 6
bounded spans, read at their span-relative offsets; 1636 whole 16-byte records
after the 16-byte header.

Neither RAM form exists on disc:

                     disc              RAM
  00 00 00 30 per    exactly 1         exactly 5
  sub-list packet
  blocks at 0/32/64  0 of 10           11 of 11
  fixed-point byte3  0 of 19           100%
  == 0x80

The 29 packets split 10 sub-list / 19 fixed-point by whether they contain
00 00 00 30 at all, and not one of the ten has the block structure, and not one
of the nineteen has the fixed-point marker.

The disc body is a compact 16-byte record stream:

  +000: 04 04 00 01 | 00 00 00 00 | 00 00 00 05 | 00 80 0e 6c
  +010: 02 00 00 00 | 05 00 00 00 | 32 00 00 00 | 0e 00 00 00
  +020: 09 00 00 00 | 00 40 3e 30 | 12 04 00 00 | 05 00 00 00
  +030: 09 00 00 00 | 00 40 3e 30 | 12 04 00 00 | 21 00 00 00

Small u32 fields and a repeating 00 40 3e 30 12 04 00 00. Per position over all
1636 records the same two outliers hold: byte 7 takes 4 distinct values and byte
15 takes 2, against 101 to 153 everywhere else.

Byte 7 is per-record, not per-packet: the first record of all 29 packets has
byte 7 = 0x00. The four-value vocabulary varies within a packet body rather than
tagging the packet, which is why grouping packets by it never separated the two
kinds.

Consequence for Phase 2: the payload is encoded on disc and expanded at load, so
a disc parser can bound it and check its signature but cannot read it as either
RAM form. Decoding it means reversing the expansion, not re-describing what the
loader produced. That is the honest boundary of what is known.
 2026-10-12 02:37:55 +02:00
ReaperOfVeriod 5ade83cf0a Parse the shrub packet header, a fixed signature on all 2056 disc packets 
The packets found by the +0x40 table have a header the whole disc agrees on.

Range measured: every packet on the disc -- 2056 of them, across the 334
successor-bounded spans -- read at each packet's span-relative offset.

  bytes 0-3   04 04 00 01   2056 / 2056
  bytes 4-7   zero           2056 / 2056
  byte  8     zero           2056 / 2056
  byte  11    0x05           2056 / 2056
  size % 16   zero           2056 / 2056

Zero exceptions on any of the five.

And the loader fills the rest in. Read back out of the running game the same
records carry byte 8 as a count (0x0b to 0x29 observed), byte 11 as one of four
tag values, and bytes 12-15 as an absolute EE RAM pointer where the disc has a
span-relative offset. Bytes 8-15 are rewritten wholesale on load.

That settles what the disc header is for. It carries no count of its own, and
the count the RAM copy has is the +0x40 table's count arriving by another
route. The disc header is a signature; the RAM header is populated state.

ShrubPacketHeader::parse and Shrub::packet_headers() expose this. parse returns
None unless all the invariants hold, rather than a default, so a packet that
does not match is reported as not-understood instead of half-interpreted.

The gated test asserts all four header invariants and the size alignment on
every packet of every bounded span.

This is the first piece of the mesh packet format represented in gc-formats;
everything known about the packet payloads so far is RAM measurement, since the
disc form of bytes 8-15 is rewritten at load.
 2026-10-12 02:35:00 +02:00
ReaperOfVeriod af0b1aa949 Repeat the packet measurements per level, correcting two over-general claims 
The five-block result was measured on level 0 only. Rule 3a asks for the same
measurement per partition, so it was repeated on level 1's savestate before
being written up as a property.

Range measured: savestate slot 2 (level 1), RAM 0x22a000..0x28f52f, 415024
bytes, then six of its pointer-bearing records read to count*16.

The six sampled pointers split 3/3 between the two payload kinds:

  00949430  41  00 00 00 30  sub-list
  009496c0  13  69 b0 bf 80  fixed-point
  00957970  36  00 00 00 30  sub-list
  00957bb0  10  58 7e 8d 80  fixed-point
  0095b740  12  37 49 41 80  fixed-point
  0095d6d0  57  00 00 00 30  sub-list

Taking all six as sub-list gives 3/6 with five blocks. Taking only the three
that actually open 00 00 00 30 gives 3/3, with the first three blocks at 0, 32,
64 in all three. The three fixed-point payloads have zero such blocks, as
expected. The earlier 3/6 was my sampling, not a counterexample -- the six
pointers were never all the same kind of object.

Combined: 11 of 11 sub-list payloads across both levels have exactly five
blocks, the first three at offsets 0, 32, 64.

The fixed-point payloads have an exact signature: byte 3 of every 4-byte group
is 0x80 in 100% of cases -- 52/52, 40/40, 48/48 over the first 512 bytes of the
three level 1 samples, and the same across all nine level 0 payloads. That
separates the two payload kinds without the record tag, which does not select
them.

Correction: level 1's record tags are not level 0's. The same scan finds 18
04 04 00 01 headers against 19, plus 6 02 01 00 01 against 10, and byte 11 is
no longer constant: 0x30 on 10, 0x00 on 5, 0x20 on 2, 0xbc on 1, where level 0
had 0x30 on all 19. "Byte 11 is 0x30, constant" was a level-0 partition result
written up as universal; across both levels it takes four values.

Fixed-point payload entropy over all nine level 0 payloads is 5.51 to 5.89 bits
per byte against 8.0 for uniform, and the mean of the s16 reading varies per
payload from -82 to -907 -- structured, and consistent with vertex positions
relative to a per-payload origin.

Not established: what the five blocks are, and how the fixed-point data decodes
to vertices. Its entropy is far below random, so it is compressed or permuted
rather than stored plainly, and the permutation table seen once in 0x009376f0
remains unexplained.
 2026-10-12 02:31:27 +02:00
ReaperOfVeriod 186a700f32 Establish the sub-list payload as a fixed five-block shape 
All eight sub-list payloads from level 0 were read to their exact extent
(count * 16) and compared block by block.

Range measured: savestate slot 1, the packets at 0x009376f0, 0x00939c50,
0x0093cc70, 0x00998880, 0x009c19c0, 0x009cf370, 0x009d2da0 and 0x009f5aa0,
counts 21 to 41, each truncated to exactly count*16 bytes.

Every payload contains exactly five blocks tagged 00 00 00 30, and the first
three are always at offsets 0, 32 and 64:

  009376f0  22  0, 32, 64, 184, 304
  00939c50  36  0, 32, 64, 268, 400
  0093cc70  23  0, 32, 64, 200, 288
  00998880  36  0, 32, 64, 272, 404
  009c19c0  25  0, 32, 64, 196, 300
  009cf370  24  0, 32, 64, 204, 316
  009d2da0  41  0, 32, 64, 308, 448
  009f5aa0  21  0, 32, 64, 176, 280

Blocks 4 and 5 move with the payload; the first three do not. Eight payloads,
eight different block-4/5 positions, and 0/32/64 in every case.

Per-block fields constant across all eight:

  block 0  byte 4 varies 0x78..0xcd
  block 1  byte 4 = 0 in 8/8, trailer 00000000 in 8/8
  block 2  byte 4 varies independently, bytes 16-20 zero in 8/8
  block 3  byte 4 EQUALS BLOCK 0 in 8/8
  block 4  byte 4 = 0 in 8/8

Block 2's word is unrelated to block 0's in every case, so the equality is not
every block reading one shared field.

Correction: the "byte permutation table" reported in the previous commit does
not generalise. It appears in 0x009376f0 at offset 212, but a scan across all
eight payloads finds no such run outside that one. It was one payload's block
content, not a format feature. The five-block shape, the fixed 0/32/64 offsets
and the block-0/block-3 equality are what survive eight-way comparison.

Not established: what the five blocks are, why blocks 0 and 3 share a word, and
what moves blocks 4 and 5 -- their offsets are not monotone in count (21 gives
280, 36 gives 400 and 404), so something inside the earlier blocks sets them.
 2026-10-12 02:30:06 +02:00