e389d6d0825f1ce7b2b984eedc75811ebdfd88ac
84
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
e389d6d082 |
UpdateCamera_6 verified at 0x4284a8 -- three independent checks
The anchor
|
||
|
|
20655c7744 |
Stop pattern-matching RAM: follow UpdateCamera_6 from lvl.camvtbl
Two more frustum searches, both unproductive, then the lookup that should have come first. Non-orthonormal 4x4 matrices within 4 KB of the camera, reasoning that the projection must live nearby: 79 candidates, every one a sliding window over data rather than a matrix. The filter does not discriminate and the region has no obvious projection. Then: the decomp names the camera. include/engine.h and docs/AUG6-RETAIL-ANCHORS.md reference UpdateCamera_6, InitCamera_24 and Camera_CollPrimTest, and docs/COMMUNITY-ENGINE-REFERENCE.md places lvl.camvtbl at 0x0026E900. The camera dispatch table is already characterised -- 217 records, 14 distinct ids on level 0, handlers in the overlay .text. That is the right way in. The camera writes its matrices at 0x18aa70; the function that does so is reachable from camvtbl, and the frustum is whatever that same function writes nearby. Reading it would confirm m4 by finding the write site, and pick up the frustum constants beside it, without guessing at float patterns. The caveat that applies first, and why this was not done first: the decomp's function addresses have already failed once here. PROVENANCE.md records that they do not resolve for the moby class loader, while struct layouts transcribed from the decomp were usable. So UpdateCamera_6's address must be verified against the overlay before following it, not trusted from the name. Resuming here is a different kind of task from the three searches above: follow a known call site instead of pattern-matching a 32 MiB heap. 294 passing, clippy clean. |
||
|
|
ca42793e48 |
Frustum search: bounded with a proper filter, not found
|
||
|
|
e7eec01b50 |
m4 is the live camera: a 20x outlier against eight controls
|
||
|
|
d4e81ba977 |
Correction: the camera region is nine matrices, not three
|
||
|
|
dcd4dd6246 |
Camera transform found at 0x18aa70: three orthonormal matrices, one position
With PCSX2 on Aranos the savestate problem went away. PINE reads EE RAM live, so two dumps seconds apart are consecutive frames by construction -- exactly what the offline savestates could not provide ( |
||
|
|
3dbd384b59 |
The guard-band clip test: implemented and unit-tested ahead of the camera
|
||
|
|
1aa94ee288 |
0x152c70 rejected as the camera; the savestates are not consecutive frames
Ran the cheapest confirmation from |
||
|
|
a991088103 |
Camera state: 2,005 per-frame float triples, one singleton candidate
The guard-band clip test's blocker is named in core-data-regions.md: the algorithm needs the camera and the frustum, both per-frame, so it needs a savestate. Two savestates give that as a differential -- anything that moved is per-frame state -- without needing to know where the camera lives. Frames .01 and .04, 33,554,432 bytes each. Of every 16-byte-aligned triple of finite floats with magnitude 0.001..100,000, 2,005 differ. Two shapes, clearly different things: the 0x1886xx block repeats one triple four times at 16-byte stride, which is a per-object array, and 0x18aa70 holds a near-unit vector that reads as a direction. 0x152c70 is a singleton, [247.000, 194.000, 49.890] -> [300.170, 227.480, 128.000], in the range Aranos occupies. Not claimed as the camera. 2,005 candidates is not a shortlist and a plausible shape is not evidence -- this stretch has produced enough of those. Three confirmation steps recorded, cheapest first: smooth motion across .01-.04, a singleton rather than an array element, and perturbing the camera in an emulator to watch that one address change. The first two need only the savestates already on disk; the third is what the lighting verification used. 286 passing with the disc present and GC_SSTATE set, clippy clean. |
||
|
|
e3e0a30030 |
The GS TEX0 register set, and why TBP0 is region-relative
Picking up "the GS register set the level load builds from the texture entry" from level-assets.md, for a renderer that emulates the GS rather than resolving texture addresses on the host. gc_formats::gs gains tex0_of (TBP0, TBW, TW, TH, PSM), texture_slot_bytes and clut_base. Three measurements back them, over all 27 levels and 1,595 textures. Every dimension is a power of two: 1,595 textures, zero exceptions, in four classes -- 128x128 (1,249), 64x64 (303), 256x256 (22), 32x32 (21). The GS addresses textures by exponent, so TW and TH are log2 of the dimensions, and a texture that was not a power of two could not be addressed at all. That the whole population is one is the measurand for the decode. TBP0 is region-relative. 19 of 27 levels have a textures_base only 64-byte aligned -- mod 256 reads 0, 64, 128 or 192 -- while every entry's own data_offset is 256-aligned on all 1,595. A base pointer computed from core_data would therefore not be a whole number of 256-byte blocks on those levels. The first version of tex0_of took textures_base + data_offset and rejected 1,168 of 1,595 textures for exactly this reason, which is how the finding surfaced. The slot is level 0 plus one mip level: the stride between consecutive entries is w*h + (w/2)*(h/2) on 1,560 of the 1,564 ascending gaps -- 32x32 gives 1,280, 64x64 gives 5,120, 128x128 gives 20,480, 256x256 gives 81,920. One byte per texel. `ty` reads 3 on all 1,595, so it is not the stored level count. Four gaps break the rule and four more run backwards. Both are reported as bounded exceptions rather than folded into an average: the four ascending outliers are 712,960 to 1,064,960 bytes, two orders of magnitude past any slot the game allocates, so they are boundaries the slot rule does not describe rather than slots that came out different. The four reverse gaps mean the table is not sorted by data_offset, which a consumer walking the region in table order would trip over. The CLUT address is the established palette * 0x100, restated as clut_base because a GS emulator asks for it as TEXCLUT's CBP. Still open after this: TEXCLUT's own fields, MIPTBP1/2 wiring, and how the ad-gifs' A+D values relate to this base. Suite with the disc: 70 disc tests, 160 gc-formats, 22 gc-trace, 23 gc-engine, 19 gc-game. Clippy clean. |
||
|
|
66e83f1665 |
The tfrag clip cube is at +0x38, and +0x2e is the mini-sphere array
TfragHeader carried a field named `cube_ofs` at +0x2e. R&C1 documents that
slot as `msphere_ofs` and puts the 8-corner clip cube at +0x38. The disc is on
R&C1's side for both.
Reading +0x2e as an array of `msphere_count` (+0x2c) Vec4f entries gives 833 of
833 well-formed arrays: four finite floats per entry, every radius
non-negative. +0x2c takes 4, 3, 5, 6, 16 and others. Reading that same slot as
eight s16 clip corners gives data that is not corner-shaped, which is how the
error was visible at all.
The real cube at +0x38 is non-zero on 833 of 833 headers and its eight points
sit inside the patch's own block on every one.
So: `cube_ofs` at +0x2e is renamed to `msphere_ofs`, the count it pairs with is
parsed and the array is read into TfragGeometry as `mini_spheres`, and
`clip_cube_ofs` at +0x38 is new, read into `clip_corners`.
What the corners are, measured over all 833:
corners, all with 8 distinct x values 833 of 833
|centroid - bsphere centre| / r median 0.147, max 0.595
corner distance from centroid / r median 1.000
The median is exactly the bounding radius, so the eight points lie on the
bounding sphere. They are not an axis-aligned box: every tfrag has eight
distinct x values, so if it is a box it is oriented. Whether they form a cube
needs the corner ordering, which is not recoverable from the data -- assuming
opposite pairs are (0,7)(1,6)(2,5)(3,4) matches body-diagonal == 2r on only 99
of 833, so either that pairing is wrong or the array is not a cube. Recorded as
unknown.
This was picked to unblock the guard-band clip test from the roadmap. The
algorithm is from the vendored R&C1 reference and is recorded with its
attribution: camera-space sphere, near/far truncation, side-plane cull, then
the 8-corner re-test against the frustum unscaled and against a volume with xy
x (0.25, 0.25) -- the 4x guard band -- with any corner outside the guard volume
sending the tfrag to the clipping path at LOD 0 with no morphing. draw_mode
already matches that table. What the test still needs is the camera matrix and
the frustum as a clip volume, both per-frame state, so the camera has to be
pinned at a savestate. The geometry side is now parsed and asserted, which
leaves only the camera side outstanding.
The s16 x 64 corner scale is R&C1's documented one and is not verified against
the game; the assertion covers that the parsed corners sit inside the patch's
own block and lie on the bounding sphere, not that the scale is right.
Suite with the disc: 69 disc tests, 156 gc-formats, 22 gc-trace, 23
gc-engine, 19 gc-game. Clippy clean.
|
||
|
|
d4aeb94616 |
Extent check inconclusive; the opcode field itself is not understood
Tried to answer whether the decomp extents land on CPU code by checking for a
prologue. Only 29 of 3,835 matched 'addiu $sp,$-N' or 'lui $gp' -- but the other
bucket is full of textbook prologues my two patterns miss, because PS2 leaf
functions save registers with no frame adjustment:
0x2a4ef0 ffb00000 ffb10008 sw $s0,0($sp); sw $s1,8($sp)
0x2a48f0 3c013f80 44816000 lui $1,0x3f80; bc1t
Not reading a result into that.
Separately, ffb00000 has major opcode 0x3f, unused in MIPS32, yet is
unmistakably a register-save prologue and ffbf0020 for 'sw $ra,0x20($sp)' appears
throughout this code. Either the EE encodes sw outside MIPS32 or I am reading the
opcode field wrong somewhere. Not resolved -- and it undercuts every opcode-based
check in this investigation, the 391 and the neighbour ratios included.
Position after twenty-one commits: the mapping is verified, two targets are VU
microcode, and the candidate windows are not established to be CPU code. What this
needs is one authoritative disassembly of one extent compared against
tools/overlay-anchors, to establish what the opcode field means. That is the same
mistake one level down from the section-indexing bug: using a decoding never
checked to produce results, then measuring those results with the same unverified
decoding.
286 passing with the disc present, clippy clean.
|
||
|
|
91b09955c8 |
The shortlist points at VU microcode; the decomp extents are unverified
391 candidates on correct bytes, 73 under 16 immediates. The densest, 0x2dca20,
164 bytes with three distinct immediates, disassembles through tools/overlay-anchors
as:
xori.b $w0, $w0, 0xa2
ext $v0, $s0, 0, 1
addu $a2, $v0, $a2
xori.b and ext are VU1 mnemonics and three-operand addu is not MIPS32. Capstone is
decoding COP2: 0x2dca20 is VU microcode. Same answer 0x2ee138 gave.
That calls the frequency result into question more seriously than the displacement
did. The measurement filtered on opcodes excluding COP2, so VU cannot explain the
391 counts directly -- but the function extents come from the decomp's CPU
analysis, and if an extent spans a VU region the window scanned is mixed and the
immediate set attributed to that function is not a function's.
Status: the enrichment is real as a measurement of those windows; what the windows
are is now doubtful.
Next: verify decomp extents land on CPU code before trusting any of them -- decode
a sample and require a plausible prologue. Same shape as the
|
||
|
|
36d8d2b00c |
The frequency enrichment survives the redo; the disassembly does not
Redone through overlay_bytes_at, the verified flat mapping:
functions kept 2,313
moby_classes 0x00c0 391 (displaced: 394)
best neighbour (0x00c8) 15 (displaced: 4)
bucket n 0xc0 best neighbour ratio (displaced)
0-15 1,033 73 2 36.5x 72.0x
16-31 759 118 4 29.5x 29.8x
32-47 283 89 2 44.5x 34.7x
48-63 118 41 3 13.7x 19.5x
64-79 55 32 2 16.0x 15.5x
80-95 37 22 2 11.0x 13.0x
The reason it survives is worth stating, because it separates the two results. The
displacement was uniform -- every window shifted by the same amount -- so a
function's contents barely changed, and a frequency count over a window is robust
to that. A disassembly is not: it depends on the exact byte alignment, which is
what was wrong. Hence 394 becoming 391 with the same shape, while the 0x3a8060
walk and the 16-byte stride evaporated entirely.
So the investigation was not "all wrong". Precisely:
- the frequency enrichment is real, now on verified bytes, with a test that fails
if it goes;
- the disassembly was never real, and the whole jal chain built on it is retracted;
- no loader has been located, and 0x2ee138 is VU microcode.
THE LESSON IS ABOUT WHICH MEASUREMENTS SURVIVE A MISTAKE
A count over a window tolerates a constant displacement. A decode does not. Sixteen
probes leaned on the kind that does not tolerate it, and reported each with the
same confidence as the kind that does. Nothing about the numbers distinguished them
at the time -- that is the whole difficulty, and why the verification in
|
||
|
|
4a826997f6 |
Redone on correct bytes: the disassembly chain is gone, 0x2ee138 is VU microcode
Re-read through overlay_bytes_at, the verified flat mapping from |
||
|
|
e6b3e0b091 |
Root cause: the per-section indexing used throughout this investigation was wrong
The verification |
||
|
|
f2d7e7ecd4 |
d97e7df was wrong: my section offset was wrong, not the decoder
Ran gc-overlay-anchors on 0x2ee138 as |
||
|
|
d97e7df41f |
Second decode error in the same investigation; stopping rather than trusting this decoder
Following
|
||
|
|
e1482abd75 |
Correct my own decode bug: there is a caller, and r2 is a return value
|
||
|
|
349b498ddd |
r2 is never written in the enclosing function, and it has no callers
The trace
|
||
|
|
eca1a16b17 |
Call sites: one caller, and it does not compute base + 0xc0
0x3a8060 has exactly one call site, at 0x3a7d88:
0x3a7d84 addiu r18, r2, 0x202d r18 = r2 + 0x202d
0x3a7d88 jal 0x3a8060
No 0xc0 in the eight instructions before the call, and r18 is built from a
different offset. So the 0xc0 inside the callee is a field offset applied to a
pointer the caller already built, not a chunk offset the caller resolved.
Two readings remain: r18 is core_data, making r18+0xc0 equal &moby_classes, which
fits since level 0's moby_classes.offset really is 0xc0; or 0xc0 is a member
offset in an unrelated structure and the match is coincidence.
Settling it needs the provenance of r2 at the call site -- real dataflow, not
counting. That is where this stops.
Recorded in docs/plan/level-assets.md. The previous commit's doc edit failed
silently on a line-break mismatch, so this appends rather than replaces.
284 passing with the disc present, clippy clean.
|
||
|
|
cee17c95b1 |
Disassembled: 0xc0 is an address offset, and the walk is 16-byte strided
Closes the ambiguity
|
||
|
|
05d9bf6cd7 |
The 72 concentrated candidates, and a float-constant ambiguity
Looking at the 72 candidates with fewer than 16 distinct immediates, the sets are full of values that look like lui high halves: 0x3f80 is the high half of 1.0f, and 0xff40/0xff70/0xff80/0xffc0 are negative high halves. 0xc0 << 16 is 0x00c00000, a small float. So 0xc0 in these functions may be a float constant rather than a chunk offset. The neighbour control cannot tell the difference, because 0xb8 and 0xc8 are equally plausible float high halves and are rare for their own reasons. Also recorded, and explicitly not as a finding: exactly 1 of 394 candidates contains another chunk offset, and none of the 72 does. That sounds like evidence of moby-specificity and is not -- the other chunk offsets are 0x1e80 and above and are globally rare (0 to 2 functions each, measured in |
||
|
|
a2f8c924b4 |
moby_classes.offset enrichment is size-independent: 13x to 72x at matched size
Runs the control |
||
|
|
06531fb66c |
The 394 does not survive: 0xc0's rate rises with immediate-set size
Ran the size-independent statistic |
||
|
|
db1e69c0ab |
Ranking the 394 by size is backwards; record the confound
Ranking the moby_classes candidates largest-first is the wrong first cut. The top entries are 34,316 and 26,432 bytes with 631 and 421 distinct immediates, median candidate 852. A function with 631 distinct immediates is far more likely to contain any given value, 0xc0 included, than one with 80, purely by pigeonhole. Sorting by size therefore selects for the functions most likely to be false positives, and the 34,316-byte outlier is probably a mis-bounded extent -- the decomp corrects three level-00 entries -- rather than a 34 KB loader. The 394 needs a size-independent statistic: whether a function contains 0xc0 more often than its immediate count predicts, not whether it contains it at all. Not measured. Recorded as a negative about the method rather than committed as progress. No test -- there is no assertion here to make. 283 passing with the disc present, clippy clean. |
||
|
|
3ea5ab799f |
moby_classes.offset is referenced by 394 functions; its neighbours by 1 to 14
First positive result of the loader hunt, and it exists only because of the
control that two commits ago I said was missing.
Across 2,323 level-0 functions of at least 128 bytes -- extents from the decomp's
functions.tsv -- the immediate 0x00c0, which is moby_classes.offset, appears in
394 functions. Every value adjacent to it appears in 1 to 14.
0xb8:8 0xb9:2 0xba:2 0xbb:2 0xbc:3 0xbd:5 0xbe:5 0xbf:3
0xc1:3 0xc2:3 0xc3:2 0xc4:1 0xc5:2 0xc6:3 0xc7:1 0xc8:14
This is what
|
||
|
|
59957a02a2 |
Attribute the 0xc0 immediate sites to functions: 410 candidates
The decomp's functions.tsv makes this cheap -- 3,835 level-0 functions with start
and size, so each immediate-bearing instruction is attributable to its container.
410 of 3,835 functions (10.7%) contain one; sizes 20 to 34,316, median 808.
Ten per cent is what a common small immediate buys, so this confirms
|
||
|
|
56b00fb305 |
Fix the immediate scan by filtering on opcode; still no signal
Saying the previous method 'cannot tell' was right but the reason deserved a step
rather than a hand-wave: it is fixable by counting immediates only where they are
immediates, which needs an opcode filter and no disassembler.
All 27 levels: 11,911,320 instructions, 2,985,897 carrying an immediate
(addiu/lui/ori/xori/andi/sltiu), 13,813 distinct immediate values.
moby_classes 0x00c0 17,699
tie_classes 0x1e80 17
And now there is a control the raw version never had -- the program's most
frequent immediates are 0x0001 (200,784), 0x001b (149,259), 0x0010 (137,523),
0xffff (98,312). 0xc0 at 17,699 sits below the top of that small-value cluster
rather than standing out, so it is not a distinctive reference.
Result unchanged from the raw scan and now defensible: no signal. The loader is
not findable by frequency, and the remaining route is structural dataflow --
functions touching both the chunk offset and a class span.
No test: asserting that an offset is unremarkable is not an invariant worth
locking, and the two numbers are already recorded with their denominator.
|
||
|
|
27891ce5b4 |
Cross-reference scan by raw immediate: the method cannot tell
Sought the moby class loader by counting raw u16 occurrences of each chunk offset in .text. moby_classes 0xc0 hits 49,230 times, tie_classes 0x1e80 hits 111 -- and that contrast means nothing. Small u16 values are simply common: the most frequent value in .text occurs 1,834,535 times. A two-byte value that is frequent by construction beats a rarer one regardless of what either references, so the comparison has no shared denominator. Rule 3a again, one level below where it has bitten before -- not the measurement being wrong but the comparison being uninterpretable. Recorded as a methodological negative rather than a finding. Doing it properly means counting immediates only where the instruction decoder says they are immediates, which is what tools/overlay-anchors exists for. No test added: a test asserting a meaningless comparison would make the next reader trust it. |
||
|
|
743107e864 |
moby_anim: the class table is not the loader
Ruled out the cheap approach: moby_classes (offset 0xc0, count 238 in level 0) holds float-shaped values, and only 120 of the first 4,096 u32s land inside the overlay .text against ~259 expected by chance. Fewer than chance, so the loader is not dispatched from a pointer here. Not concluded what the 238 entries are -- the float reading is suggestive but was taken at one stride on one level. |
||
|
|
6d26da3630 |
The control fails my own claim: 0x1c/0x34 select nothing distinctive
|
||
|
|
2ff2abbd32 |
Two class-header words are base-relative offsets pointing past the tie packets
Chasing where the AD-GIF is, since |
||
|
|
f43a1c2e6e |
bfc_distance is not the control region's length; the AD-GIF is not in the packet
Follows
|
||
|
|
9bb04cca7b |
The tie control region is not a VIF or GIF tag stream; and vert_ofs is mis-named
Went after the GS's GIF packets in the tie AD-GIF region. Got a negative result,
a new universal invariant, and a retraction of my own first probe, which reached
the right answer for the wrong reason twice.
THE NEGATIVE RESULT
Tried the leading `5 * shader_count` quadwords of each packet (the
`bfc_distance` field's stated meaning) as VIF tags -- command in bits 0-6, size
in 7-14, mark in 15 -- and as GIF tags -- type in the top byte of word 1, length
in bits 0-2 of word 0. Both uniform. Over 13,042 records: 10,443 distinct values
of word 0, no GIF tag type above a count of 2. That is the control result rule 4
asks for, and it says stop.
THE RETRACTION
The first probe scanned `bfc_distance` quadwords, which for most records
overruns `vert_ofs` and runs into the vertex data. Pooling a control region with
vertex positions -- two populations with different structure -- gives a uniform
answer, so "not a tag stream" came out right from a measurement that was
measuring the wrong bytes. That is exactly the rule 3a failure, and I only caught
it because the hex dump I took next looked structured while the histogram said
random.
Re-measuring quadword 0 alone, on its own population, gives the same uniformity.
The conclusion survives; the reasoning did not, and both are recorded.
THE PARTITION, AND A NEW INVARIANT
The hex dump showed the packet layout directly:
q[0 .. vert_ofs) control 3..8 quadwords, peak at 5
q[vert_ofs .. +vert_size) vertices
q[vert_ofs+vert_size ..] scissor
and `scissor_ofs == vert_ofs + vert_size` holds on **15,219 of 15,219** records
across all 27 levels, including every one whose strip_count is non-zero. That is a
derived quantity stored explicitly, and it holds everywhere -- which is the check
that confirms the three regions really are laid out the way the existing packet
length formula assumed. It is now a hard check in invariants_hold(), which is
something the previous four could not be.
The fixture in tie.rs failed the moment I added it: it had vert_ofs 4 and
vert_size 0x20 with scissor_ofs 0x30, which is 4 + 0x24. The synthetic data was
never constrained by a relation that did not exist yet. Fixed to 0x24.
vert_ofs IS MIS-NAMED
It is not a vertex offset. It is the *length* of the leading control region, and
the vertex data begins at that many quadwords into the packet. The field name
stays because the on-disc order is fixed and everything already written about this
record says vert_ofs, but the doc now states what the value is and gives the
distribution.
STILL UNKNOWN
What the control region contains. Not standard DMA tags; possibly pre-merged GS
register writes in another framing, or per-class state rather than per-primitive
draw data. Recorded as an open question rather than guessed at.
23 + 19 + 58 + 156 + 22 = 278 passing with the disc present (disc binary 35.67s),
clippy clean.
|
||
|
|
1fa1913e38 |
Reconcile the docs after the last three corrections
The last three commits changed things the documentation still asserted in the
old direction. Fixed:
roadmap
* 227 tests -> 238.
* The level_overlay bullet contradicted itself: it said "the header words are
not named" in the same breath as "+0x00 load base is 0x1A7C80". Now says
+0x00 IS the load base and is named, +0x0c is excluded as the .lit size by
sign (below file length on 16/27, by up to 8.9%) and remains unnamed, and the
unnamed remainder is stated as such.
* Section names aligned with the Phase 3 list (lvl.vtbl, lvl.camvtbl,
lvl.sndvtbl).
core-data-regions.md
* The claim-assertion audit still said the two PCSX2 claims were "NOT
assertable here -- needs a savestate". They are asserted now, gated on
GC_SSTATE, in the same file. Updated, with the two corrections that fell out
of writing them recorded right there.
* "that string is inside the collision routine" was never established. It is
in the string pool, yes; the reference to it is not found. Reworded to say
so, including the zero-hit result.
research log
* field_00 now carries its name and the .lit/$gp/program context.
* The field_0c falsification and the byte-order retraction are both recorded
here as well, so the log and the format file agree.
Nothing left asserting the withdrawn anchor, and no stale test counts.
|
||
|
|
d8b8ccc6d4 |
Follow the gameplay.bin pointers: they address position and direction arrays
Where the pointers go. Following them shows what gameplay.bin indexes. The leading table is 39 words, all of which land inside the file, and three clusters dominate: near 150000, near 800000 and near 1050000. The region near 797808 is position data -- 31 consecutive plausible floats that group cleanly into xyz triples: 246.948 201.015 73.867 | 88.240 186.000 302.598 | 107.712 12.723 183.125 353.591 126.365 19.962 | 182.068 374.001 122.567 | 116.961 265.001 437.866 and the region near 797456 is direction data -- unit-length vectors: 0.969 0.945 0.733 | -0.120 0.433 0.250 | -0.866 0.000 0.267 So the file's leading words are a table of pointers into float arrays, not a flat list of small integers. The earlier reading of it as "u32 pointers holding nothing" was too quick: it is a pointer table into data that is present. Volumes remain unlocated. The position arrays are the right shape to bound a volume and nothing yet shows the volume records themselves. |
||
|
|
b17ec17127 |
Rule out the moby_anim instance-table reading of the chunk types
moby_anim was the one post-shrub chunk type with a concrete reason to exist: an instance table would have to scale with the level's moby class count, so comparing per-type record counts against that count was the obvious test. It fails, and the way it fails is itself the result: per-type record counts are level-independent. The same types report the same counts on every level tried: type_id level0 level1 level2 level3 0x007d7ec438 90 90 90 90 0x00d3d8c4a9 90 90 90 90 0x00429ec43d 77 77 77 77 0x0053c8c4aa 77 77 77 77 0x007626c4a4 50 50 50 50 0x00fb88c462 47 47 47 47 Across all 27 levels there are 77 distinct chunk types, the largest at 2430 records total -- exactly 90 x 27. What that rules out: nothing in this chunk set is a per-moby-class instance table, because those counts would vary by level the way the moby class counts do. So whatever holds moby animation data is either keyed by something other than the class count, or it is not among these chunk types. Also visible: the types pair up. 0x007d7ec438 and 0x00d3d8c4a9 both report 90 on every level; 0x00429ec43d and 0x0053c8c4aa both report 77. Two chunk types carrying identical, constant record counts on every level is a signature of two views of the same fixed table rather than two independent tables that happen to agree. Still open: the pairings are described but not explained, and no type has been named. The Moby class-ID enrichment from earlier stands separately -- 9493 hits against 246.1 expected is real, and the class arrays are contiguous per type, but that identifies where classes are referenced, not which chunk is moby_anim. |
||
|
|
02ced5ee83 |
Parse the per-level overlay: header, constant block, level data
level_overlay was the least-characterised file in the level set -- no parser, no located structure. It now has gc_formats::overlay and a real-disc test over all 27 levels: overlay: 27 levels; field_00 1735808, field_08 1; field_04 37540..=45308, field_0c 2763912..=3027336 (17 below the file length, 10 above); +0x10..0x1220 identical on all The header is four words. +0x0c is not the file length: it was recorded as one on the strength of the first levels measured, where it sat near the file end, but across all 27 it falls on both sides -- below on 17, above on 10 -- so it is an ordinary level-specific word. The test asserts this, being the claim most worth pinning. The common prefix is 4 bytes, not the whole header: +0x00 is constant at 1735808 while +0x04 and +0x0c vary. Everything from +0x10 to +0x1220 -- 4624 bytes -- is byte-identical on all 27 levels. Still not named: what any of the four header words means, and how the level-specific tail is structured. The module reports them as fields. Also records in docs/plan/pcsx2-verification.md that the 64-byte RAM stride from the previous commit is disproved. A 415 KB dump of level 1 finds the longest such run at 27 records, not 384, and those are identity matrices -- 27 of them, one per level. It is a per-level transform table, not the collision grid, and the 20.0 reported there was a coincidence. |
||
|
|
d3e1475f64 |
Characterise gameplay.bin as a flat pointer table, and record volumes as unlocated
Range measured: the first 40 u32 and the whole file, on levels 0 and 1 --
1872256 and 2006400 bytes.
Result: the file is a flat array of u32, opening with a small count then a run of
ascending offsets, terminating on a zero:
level 0 336 797472 798832 799136 464 150672 150688 314800 479920 ...
... 1847600 1845456 797808 1829056 160 1849088 1249808 1847616 0
level 1 496 799904 800128 800432 624 151152 151168 315680 481456 ...
... 1984496 1982400 800112 1966000 160 1988160 1303456 1984512 0
Both levels carry the same shape. The sentinel 160 appears at u32 index 34 on
both and the zero terminator at index 38 on both, with counts 336 against 496
and 464 against 624 at the head. The leading ~38 words are a structured header
whose layout is level-independent apart from its sizes.
The offsets cluster into groups -- near 800000, 150000, 1050000, 1800000 --
which is what a table of pointers into several buffers looks like.
Not decoded: what the pointers point into, what 160 is, and why the zero
terminator sits at the same index on both levels. No gc_formats module exists for
this yet; it is characterised, not parsed.
On volumes: nothing resembling volumes was found here. gameplay.bin is u32
pointers throughout and holds no float data in its first 64 words read as f32 --
59 of 64 are denormals, i.e. small integers reinterpreted. Volumes remain
unlocated on the disc; nothing in the regions examined so far presents itself as
one.
|
||
|
|
575d1a5bdd |
Correct the overlay strings from a table to a pool
The previous commit described a "debug string table on 16-byte alignment". It is not a table. Dumping the 32 bytes before each run shows most are the tail of the previous string: +3104 "host0:r:/rc2/shock/wavetable.bin" 32 chars +3120 "ck/wavetable.bin" its own tail +3152 "/i5/shock/wavetable1.bin" +3168 "ble1.bin" its own tail and the 32-byte prefixes are 327 distinct out of 332 -- essentially all different, because they are just the preceding string's bytes. There is no per-record header here. It is one contiguous pool of debug text, and the 16-byte alignment is a property of how the pool was laid out, which makes continuations land on boundaries and read as separate entries. The count of 332 "strings" was therefore inflated by the same effect: it counts continuations as entries. The substance is unchanged and still useful -- the pool carries the game's own debug text, including "Camera_CollPrimTest WARNING! - grid out of bounds! (RAR)", which is direct evidence for reading the collision payload's two leading u16 as grid dimensions, and the wavetable.bin paths, which bear on the sound bank framing. Only the framing was wrong. The pool is located and counted but not parsed, and there is no gc_formats module for it. The +0x18/+0x20 names are section names but their structure is not established, and it is not shown that they index anything. |
||
|
|
a5a33b9bc0 |
Locate level_overlay's sections and a 16-aligned debug string table
level_overlay was on the open list with no parser and no located structure. It now has a header naming two sections and a string table. Range measured: overlay.bin on levels 0, 1 and 2 -- 2751268, 2763668 and 2928424 bytes. Result 1: the file opens with section names. +000: 80 7c 1a 00 | dc 99 00 00 | 01 00 00 00 | 88 3b 2a 00 +010: 00 00 00 00 | 00 00 00 00 | "crap\n" | 00 00 00 00 00 +020: "system misc" 00 00 00 00 00 00 00 00 00 +030: 20 80 ff 40 | 00 60 cf 00 | 64 00 | 46 00 | 19 00 00 00 u32@+0x00 is 1735808 on all three levels, +0x08 is 1, and the bytes at +0x18 and +0x20 spell "crap" and "system misc". Result 2: a debug string table on 16-byte alignment. 332 such strings on level 0, 362 on level 1, 351 on level 2, first at +0x20 and last between 1.3 MB and 2.5 MB. They are the game's own debug text: host0:r:/rc2/shock/wavetable.bin host0:z:/i5/shock/wavetable1.bin unloaded sound bank %#010x Camera_CollPrimTest WARNING! - grid out of bounds! (RAR) render setup sky draw pre effects pre 2 effects Camera_CollPrimTest WARNING! - grid out of bounds! (RAR) is direct evidence for the collision reading. It is the game's own string, naming a grid and a collision test, and it is why the two leading counts in the collision payload are worth continuing to treat as grid dimensions despite not sizing the payload. The wavetable.bin paths also confirm the sound bank is loaded from named files, which bears on the SBlk/DAW framing. Not done: the string table is located and counted but not parsed -- the records preceding each string have not been identified, so there is no gc_formats module for it yet. The +0x18/+0x20 names are section names but their structure is not established, and it is not yet shown that they index anything. |
||
|
|
e58ab0e815 |
Correct the collision table's extent: 17 entries, deltas rise then fall
+5932 is the table's start, not a point inside it: walking backwards, no predecessor satisfies "ascending with a delta under 4096". The run is 17 entries, not 19 -- the earlier 19 counted two entries past the jump. The deltas are 204 212 216 224 224 228 228 228 224 220 216 200 96 96 100 96 then a jump of 41484 to 190508. The deltas rise and then fall -- a symmetric profile, not a monotone one. That weakens the prefix-sum reading: a cumulative offset table has no reason to come back down, whereas cell contents that grow and shrink as the table walks across a level would. 17 against a 75-cell grid, and 75 - 17 = 58. No arithmetic relating the two has been found. The earlier guess that 19 related to 3x25 did not survive re-measurement and is not carried forward. Still open. 17 entries may not correspond to the grid at all -- the table may cover something else, or a subset of cells. Resolving it needs either the same offset found in other levels at a position the leading counts predict, or a reason for 17. Neither exists yet, and this is where the measurement trail goes cold on collision. |
||
|
|
b66e77855c |
Locate a per-cell offset table in collision, on level 4
Range measured: longest strictly-ascending u32 run (delta under 65536) in each
level's collision payload, levels 0 to 5.
Level 4 is the clean case. Grid 3x25, payload 2660608, and at payload +5932 a
run of 19 entries whose values are all inside the payload, starting at 146012:
deltas 204, 212, 216, 224, 224, 228, 228
The deltas grow monotonically, which is the signature of a prefix sum over
records of increasing size -- what a per-cell offset table looks like when cell
contents vary.
The others are not:
level 0 11x33 +21232 17 8193, 24575, 8192, 8192 multiples of 8192,
coincidental on
dense data
level 1 24x16 +41452 29 32771, 57339, 24575 wild, no trend
level 2 1x53 +13752 49 24, 64, 396, 104 in-region but
irregular
level 3 19x31 +667152 18 769, 257, 257, 257 values ~5.2e8, far
outside the payload
level 4 3x25 +5932 19 204, 212, 216, 224 growing, in-region
Only level 4's run has both properties a prefix sum needs: values inside the
payload, and deltas that grow.
What this does not give: level 4's run is 19 entries against 75 grid cells, so
it is not one offset per cell -- either most cells share an offset, are empty,
or the table covers something other than the grid. Nothing measured picks between
those.
Next to check: whether 19 relates to 3x25 by any arithmetic, and whether the
same offset at +5932 exists in other levels at a position given by their leading
counts. Neither has been tried.
|
||
|
|
2569cc06ab |
Decode the collision payload's leading two counts
Range measured: the first four bytes of the payload on all 19 levels that carry a non-empty one. Result: two u16 at payload +0 and +2, both non-zero on all 19, ranging 1..74 and 13..112, products 53 to 3256. level 0 11 x 33 = 363 payload 903248 level 1 24 x 16 = 384 payload 2800496 level 2 1 x 53 = 53 payload 2315296 level 7 40 x 35 = 1400 payload 4049232 level 9 74 x 44 = 3256 payload 2518352 level 13 2 x 64 = 128 payload 3333520 level 26 27 x 112 = 3024 payload 652176 Reported as two counts, not as a named grid. They have the shape of collision grid dimensions -- bounded, both non-zero, products in the hundreds to thousands -- but nothing measured ties the product to the payload size or to any per-cell stride. Level 2's 1 x 53 giving a 2.3 MB payload and level 26's 27 x 112 giving 652 KB shows the product does not size the payload, so either cells vary wildly in size or these are not grid dimensions. Naming them width and depth would be a reading, not a measurement. After the two counts comes a run of u16 that stays below 8192 for 30 to 58 entries depending on level, always an even count, and is not ascending. That is where the per-cell table most likely starts, and it is the next thing to resolve. Exposed as CollisionHeader::first and ::second. They live in the payload rather than the header, so CollisionHeader::region fills them in and the bare CollisionHeader::parse leaves them zero -- documented at the call site so the difference is not mistaken for an absence. |
||
|
|
72bbd5da38 |
Rule out GS packet framing in tie, with a control that nearly fooled me
Range measured: level 0, the 73 tie span bodies (bytes +0x40 onward), 2585856 bytes pooled. Per-position distinct counts, an autocorrelation scan against a shuffled control, and a GS GIF tag test. Result 1: the bodies are dense. Every one of the 16 slot positions takes 245 to 256 distinct values, byte 15 zero on 81.3% and byte 7 zero on 78.8%. The best stride peak is 48 at +0.097, barely above 16 at +0.095 -- no period. Result 2: no GS GIF framing at 16-byte alignment. Testing whether byte 15's low three bits fall in 0..4, the GS tag range, gives 97.0%. That looked like a strong hit until the control was done properly. A uniform shuffle gives 62.6%, exactly 5/8, what a random byte predicts -- so the naive comparison reads as a 34-point excess. The distribution-matched null is 97.0%, identical to the observation. The whole excess was that byte 15 is 81% zero and (0 & 7) <= 4 is always true. Recomputed per alignment the null tracks the observation everywhere: align 0 97.0% align 4 75.7% align 8 96.7% align 2 75.5% align 12 89.3% This is the control rule earning its place a second time. A uniform shuffle is not a matched control when the data is skewed, and AGENTS.md already says so; the near-miss was reading 62.6% as a baseline without checking what the data's own distribution predicts. The tie bytes concentrate on 0, 1, 2, 3 and 63 -- suggestive in isolation -- and still carry no framing. What this rules out: the tie bodies are not GS GIF packet streams at 16-byte alignment and carry no measurable periodicity. With every slot position taking ~256 distinct values they are dense data -- fat vertices, as the roadmap expects -- and locating the vertex layout needs a reference for this game's instanced geometry format, not another stride scan. |
||
|
|
86ff2e6052 |
Measure tie spans: a constant first word and three quantized floats
Range measured: all 1864 tie spans across 27 levels, read at each class entry's
geometry offset.
Method: read +0x00, +0x0c and the three f32 at +0x10, +0x14, +0x18, and count
distinct values per position.
Result:
u32@+0x00 takes exactly one value across all 1864 spans: 0x00000100, 256
+0x0c takes 83 distinct values, so it is not constant
the three floats take 103 distinct triples built from 42 distinct values,
and those values are round -- 4, 8, 9, 10, 12, 15, 16, 18, 20, 24, 25, 30
with only 10 of the 42 exact powers of two
Two level 0 spans open:
+000 00000100 00005070 000065c0 07050612
+010 41400000 41c80000 42200000 00009460
+000 00000100 00007ca0 0000a040 0605091e
+010 41400000 41c80000 42200000 0000ddf0
This rules out the bounding-box reading. A box would take continuous values;
these come from a 42-value set of round numbers, so they are authored or
quantised quantities -- a grid, a size class, or LOD thresholds -- and nothing
measured separates those.
Correction: these floats were previously read as a bounding-box minimum, from
level 0's first two spans happening to share (12.0, 25.0, 40.0). That triple
holds on 266 of 1864 spans, not all of them, and the value set is quantised.
Not decoded: what the three floats are, what 0x100 at +0x00 counts, and what
the 83 values at +0x0c are.
|
||
|
|
32a24574be |
Identify data.bin as the level file descriptor, not geometry
data.bin is the largest per-level file -- 13.3 MB on level 0, 4.2 to 21.9 MB across the disc -- and had been characterised only as "4-byte structured". Range measured: the first 64 u32 of data.bin on levels 0, 1, 2 and 5, compared against the actual byte lengths of the other extracted level files. Method: read the first 64 u32 and look for any equal to the size of core_index.bin, gs_ram.bin, overlay.bin, gameplay.bin, sound_bank.bin or core_data.bin. Result: three exact matches, at the same indices on every level. u32 index 1 overlay.bin 2751268 2763668 2928424 2863676 u32 index 3 core_index.bin 31600 34016 33808 23536 u32 index 5 gs_ram.bin 518656 569344 513024 434176 Each is the exact byte length of the named file and each lands on the same index on all four levels. Three coincidences at fixed positions is not a coincidence. So data.bin is a descriptor for the level's other files, not geometry in its own right. The roadmap's "the large one: 4.2 to 21.9 MB of geometry per level" entry was wrong and is corrected. The intervening indices 0, 2 and 4 hold values that are not file sizes and their meaning is not established; they are plausibly offsets, but no cumulative-sum or offset+size relation has been checked yet. What follows the descriptor is still unknown, and the file is large enough that the descriptor is unlikely to be all of it. |
||
|
|
5610d173b0 |
Narrow the collision header to two fields plus a third used by five levels
Range measured: all 27 levels that carry collision, reading the 0x40-byte
header each.
Method: read +0x00, +0x04 and +0x08, then check +0x08..+0x40 for zero.
Result:
u32@+0x00 is 0x40 on 27 of 27 -- the header is self-sizing
u32@+0x04 is the payload size, 20 distinct values across the disc
bytes +0x08..+0x40 zero on 22 of 27
Five levels carry a large non-zero word at +0x08:
level 7 1248048 level 12 2541616
level 9 1766832 level 13 3376592
level 14 4060640
So the header is not two fields and 56 bytes of padding: there is a third field
most levels leave at zero and five use. Whether it is a size, an offset or a
second block's length is not established, and nothing measured here separates
those readings. Exposed as CollisionHeader::field_08.
Correction: the collision header was previously described as self-describing
across its full 0x40, which is true for 22 levels and overstates the case for
the other five.
Still not decoded: the payload. It contains ascending u32 runs on levels 0 and
2 -- level 0 at +280 with deltas 12, 12, 12 then a jump, level 2 at +13868 with
deltas 204, 204, 200, 196 -- but irregular deltas fit neither a flat table nor a
simple stride, and level 1's apparent run is a byte-shift false positive.
Nothing about the payload is asserted.
|
||
|
|
681e7e62ef |
Parse the sky block: 149 sub-objects with a bounding sphere, and correct the count field
Range measured: all 27 levels, each sky region from the core index's sky offset to the next region. Method: read the 32-bit offset table at +0x20, bounded by staying inside the region, and parse each sub-object it names. Result: 149 sub-objects across 27 levels. Each opens with a count, then a bounding sphere of four floats at +0x10, then a texture offset at +0x20 and dimensions at +0x24/+0x26: +000: 26 00 00 00 | 00 00 00 00 00 00 00 00 00 00 00 00 +010: e0 8d b2 c1 | 9f d6 f6 3d | ef fc fb be | 1c 61 4e 41 +020: 90 99 05 00 | 1d 00 | 1d 00 | 00 00 e8 00 ... Count 1 to 38, width 3 to 81, radius 1.95 to 14.56, and the radius is positive on 149 of 149 -- the same sphere shape and the same reasoning as the shrub class header's. Correction to an earlier claim: the word at +0x0c is not the sub-object count. It reads 5 on level 0 where the table holds 6 entries, 3 on level 1 against 4, 7 on level 3 against 8, and 196617 on level 2 against 4. Across the disc offsets.len() == count+1 holds on 10 of 27 levels. The field is reported as SkyHeader::field_0c and nothing depends on it; the offset table enumerates the sub-objects and is self-limiting. That earlier claim came from level 0 alone, where 5 against 6 looked like a consistent off-by-one. It is not an off-by-one -- it is a field that happens to be small on most levels and is not a count at all. gc_formats::sky parses the header and the sub-objects. The gated test asserts the sphere is finite, the radius positive where positive at all, the dimensions non-zero, and the sub-object list non-empty on every level. |