main
227
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
e389d6d082 |
UpdateCamera_6 verified at 0x4284a8 -- three independent checks
The anchor
|
||
|
|
20655c7744 |
Stop pattern-matching RAM: follow UpdateCamera_6 from lvl.camvtbl
Two more frustum searches, both unproductive, then the lookup that should have come first. Non-orthonormal 4x4 matrices within 4 KB of the camera, reasoning that the projection must live nearby: 79 candidates, every one a sliding window over data rather than a matrix. The filter does not discriminate and the region has no obvious projection. Then: the decomp names the camera. include/engine.h and docs/AUG6-RETAIL-ANCHORS.md reference UpdateCamera_6, InitCamera_24 and Camera_CollPrimTest, and docs/COMMUNITY-ENGINE-REFERENCE.md places lvl.camvtbl at 0x0026E900. The camera dispatch table is already characterised -- 217 records, 14 distinct ids on level 0, handlers in the overlay .text. That is the right way in. The camera writes its matrices at 0x18aa70; the function that does so is reachable from camvtbl, and the frustum is whatever that same function writes nearby. Reading it would confirm m4 by finding the write site, and pick up the frustum constants beside it, without guessing at float patterns. The caveat that applies first, and why this was not done first: the decomp's function addresses have already failed once here. PROVENANCE.md records that they do not resolve for the moby class loader, while struct layouts transcribed from the decomp were usable. So UpdateCamera_6's address must be verified against the overlay before following it, not trusted from the name. Resuming here is a different kind of task from the three searches above: follow a known call site instead of pattern-matching a 32 MiB heap. 294 passing, clippy clean. |
||
|
|
ca42793e48 |
Frustum search: bounded with a proper filter, not found
|
||
|
|
e7eec01b50 |
m4 is the live camera: a 20x outlier against eight controls
|
||
|
|
d4e81ba977 |
Correction: the camera region is nine matrices, not three
|
||
|
|
dcd4dd6246 |
Camera transform found at 0x18aa70: three orthonormal matrices, one position
With PCSX2 on Aranos the savestate problem went away. PINE reads EE RAM live, so two dumps seconds apart are consecutive frames by construction -- exactly what the offline savestates could not provide ( |
||
|
|
b4d605a2f4 |
Docs audit: 15 stale figures left behind by two fixes, including a wrong numerator
Checked the docs for claims this session retracted. Found more than the first pass.
1,163 -> 1,162 9 in the format spec, 3 in the roadmap, 5 test doc-comments
5,878 -> 5,822 2 in the spec, 3 in test doc-comments
234 -> 233 1 in the spec, 1 test doc-comment -- level 0's contribution
The last one is the one a blanket search would have missed. "5,878 records, and
level 0 contributes 234" was stale in both numbers, and only one of them was in
the string I was looking for. Verified against the tests rather than against the
other stale prose:
lvl.vtbl: 1162 distinct class ids from 5822 records; max 0x13ba (5050);
moby_classes.count is 238, and 1135 of 1162 ids (97.7%) are at or above it
vtbl oClass: 1162 ids, largest 5050; ... 1133/1162 ... level 0 alone
contributes 233
The reason all of it survived is the same shape as the bug it belongs to. The
tests were fixed and the new numbers asserted, and the prose was not touched -- and
the prose is what a reader takes away. A commit that corrects a measurement without
correcting the document it was measured in has only half-landed. Two corrections in
this session, 15 places.
Also left alone deliberately: the explanatory sections in core-data-regions.md that
discuss 5,878 and 1,163 as the *wrong* figures, and the test comments that record
what the loop used to print. Those are correct as written -- removing them would
erase the reasoning. A final sweep confirms nothing stale remains outside those.
293 passing with the disc and savestate present, clippy clean. Documentation and
comments only; no behaviour changed.
|
||
|
|
6e5486de88 |
Docs audit: nine stale class-ID counts left behind by the terminator fix
Checked the docs for claims this session retracted. Found a real one: the vtbl
terminator fix (
|
||
|
|
3dbd384b59 |
The guard-band clip test: implemented and unit-tested ahead of the camera
|
||
|
|
1aa94ee288 |
0x152c70 rejected as the camera; the savestates are not consecutive frames
Ran the cheapest confirmation from |
||
|
|
a991088103 |
Camera state: 2,005 per-frame float triples, one singleton candidate
The guard-band clip test's blocker is named in core-data-regions.md: the algorithm needs the camera and the frustum, both per-frame, so it needs a savestate. Two savestates give that as a differential -- anything that moved is per-frame state -- without needing to know where the camera lives. Frames .01 and .04, 33,554,432 bytes each. Of every 16-byte-aligned triple of finite floats with magnitude 0.001..100,000, 2,005 differ. Two shapes, clearly different things: the 0x1886xx block repeats one triple four times at 16-byte stride, which is a per-object array, and 0x18aa70 holds a near-unit vector that reads as a direction. 0x152c70 is a singleton, [247.000, 194.000, 49.890] -> [300.170, 227.480, 128.000], in the range Aranos occupies. Not claimed as the camera. 2,005 candidates is not a shortlist and a plausible shape is not evidence -- this stretch has produced enough of those. Three confirmation steps recorded, cheapest first: smooth motion across .01-.04, a singleton rather than an array element, and perturbing the camera in an emulator to watch that one address change. The first two need only the savestates already on disk; the third is what the lighting verification used. 286 passing with the disc present and GC_SSTATE set, clippy clean. |
||
|
|
e3e0a30030 |
The GS TEX0 register set, and why TBP0 is region-relative
Picking up "the GS register set the level load builds from the texture entry" from level-assets.md, for a renderer that emulates the GS rather than resolving texture addresses on the host. gc_formats::gs gains tex0_of (TBP0, TBW, TW, TH, PSM), texture_slot_bytes and clut_base. Three measurements back them, over all 27 levels and 1,595 textures. Every dimension is a power of two: 1,595 textures, zero exceptions, in four classes -- 128x128 (1,249), 64x64 (303), 256x256 (22), 32x32 (21). The GS addresses textures by exponent, so TW and TH are log2 of the dimensions, and a texture that was not a power of two could not be addressed at all. That the whole population is one is the measurand for the decode. TBP0 is region-relative. 19 of 27 levels have a textures_base only 64-byte aligned -- mod 256 reads 0, 64, 128 or 192 -- while every entry's own data_offset is 256-aligned on all 1,595. A base pointer computed from core_data would therefore not be a whole number of 256-byte blocks on those levels. The first version of tex0_of took textures_base + data_offset and rejected 1,168 of 1,595 textures for exactly this reason, which is how the finding surfaced. The slot is level 0 plus one mip level: the stride between consecutive entries is w*h + (w/2)*(h/2) on 1,560 of the 1,564 ascending gaps -- 32x32 gives 1,280, 64x64 gives 5,120, 128x128 gives 20,480, 256x256 gives 81,920. One byte per texel. `ty` reads 3 on all 1,595, so it is not the stored level count. Four gaps break the rule and four more run backwards. Both are reported as bounded exceptions rather than folded into an average: the four ascending outliers are 712,960 to 1,064,960 bytes, two orders of magnitude past any slot the game allocates, so they are boundaries the slot rule does not describe rather than slots that came out different. The four reverse gaps mean the table is not sorted by data_offset, which a consumer walking the region in table order would trip over. The CLUT address is the established palette * 0x100, restated as clut_base because a GS emulator asks for it as TEXCLUT's CBP. Still open after this: TEXCLUT's own fields, MIPTBP1/2 wiring, and how the ad-gifs' A+D values relate to this base. Suite with the disc: 70 disc tests, 160 gc-formats, 22 gc-trace, 23 gc-engine, 19 gc-game. Clippy clean. |
||
|
|
66e83f1665 |
The tfrag clip cube is at +0x38, and +0x2e is the mini-sphere array
TfragHeader carried a field named `cube_ofs` at +0x2e. R&C1 documents that
slot as `msphere_ofs` and puts the 8-corner clip cube at +0x38. The disc is on
R&C1's side for both.
Reading +0x2e as an array of `msphere_count` (+0x2c) Vec4f entries gives 833 of
833 well-formed arrays: four finite floats per entry, every radius
non-negative. +0x2c takes 4, 3, 5, 6, 16 and others. Reading that same slot as
eight s16 clip corners gives data that is not corner-shaped, which is how the
error was visible at all.
The real cube at +0x38 is non-zero on 833 of 833 headers and its eight points
sit inside the patch's own block on every one.
So: `cube_ofs` at +0x2e is renamed to `msphere_ofs`, the count it pairs with is
parsed and the array is read into TfragGeometry as `mini_spheres`, and
`clip_cube_ofs` at +0x38 is new, read into `clip_corners`.
What the corners are, measured over all 833:
corners, all with 8 distinct x values 833 of 833
|centroid - bsphere centre| / r median 0.147, max 0.595
corner distance from centroid / r median 1.000
The median is exactly the bounding radius, so the eight points lie on the
bounding sphere. They are not an axis-aligned box: every tfrag has eight
distinct x values, so if it is a box it is oriented. Whether they form a cube
needs the corner ordering, which is not recoverable from the data -- assuming
opposite pairs are (0,7)(1,6)(2,5)(3,4) matches body-diagonal == 2r on only 99
of 833, so either that pairing is wrong or the array is not a cube. Recorded as
unknown.
This was picked to unblock the guard-band clip test from the roadmap. The
algorithm is from the vendored R&C1 reference and is recorded with its
attribution: camera-space sphere, near/far truncation, side-plane cull, then
the 8-corner re-test against the frustum unscaled and against a volume with xy
x (0.25, 0.25) -- the 4x guard band -- with any corner outside the guard volume
sending the tfrag to the clipping path at LOD 0 with no morphing. draw_mode
already matches that table. What the test still needs is the camera matrix and
the frustum as a clip volume, both per-frame state, so the camera has to be
pinned at a savestate. The geometry side is now parsed and asserted, which
leaves only the camera side outstanding.
The s16 x 64 corner scale is R&C1's documented one and is not verified against
the game; the assertion covers that the parsed corners sit inside the patch's
own block and lie on the bounding sphere, not that the scale is right.
Suite with the disc: 69 disc tests, 156 gc-formats, 22 gc-trace, 23
gc-engine, 19 gc-game. Clippy clean.
|
||
|
|
3a1156d8e1 |
The tfrag morph is not in the data block: positions are verbatim in RAM
The morph was recorded as unverifiable because "reading a weight back out of
RAM does not work -- the positions live inside regenerated VU packets". That
is right about the correspondence and wrong about the cause. The cause is
measurable, and it comes back clean:
patches 833
positions 40,600 disc == ram
positions identical 40,600
block bytes differing 166,465
inside a position unpack 0
inside another unpack 49,484
in a VIF command word 116,981
Range: level 0, all 833 tfrag patches, live PINE RAM against the disc. Two
independent measurements agree. Walking the VIF lists on both copies and
comparing parsed positions gives 40,600 identical and zero changed; a raw
per-byte diff of each patch's own extent classifies every differing byte, and
none of them is in a position unpack. The position ranges total 490,224 bytes.
So the blend is not precomputed into the geometry. The rewrites are packet
framing -- 70% command words -- plus other unpack payloads, which is the loader
rebuilding the VIF packet for the tier it selected while leaving the vertices
alone. The morph happens at draw time in the VU stream.
The standing note becomes a sharper one: there was never a correspondence
problem. The values were identical, so there was nothing to read.
gc-trace gains verify-morph, which does both measurements. TfragGeometry
gains position_ranges and other_ranges so the walk needed to classify a diff
is already paid for, and the ranges come from the same walk the rest of the
parse uses rather than a second one that could drift.
Per-patch extents are the gap to the next patch's data pointer. That is exact
and needs no guess about which header field marks a patch's end; the first
attempt at this compared each patch against the whole rest of the region and
reported 65 million differing bytes, which is what an unbounded comparison
looks like.
tfrag_positions_are_verbatim_and_the_rewrites_are_elsewhere is gated on
GC_SSTATE and asserts the four bounds, plus two guards: the rewrites must be
non-trivial (over 100,000 bytes) and mostly command words, so a RAM image that
was a verbatim copy of the disc -- which would make the zero look good for the
wrong reason -- fails instead.
Suite with the disc and GC_SSTATE: 68 disc tests, 156 gc-formats, 22
gc-trace, 23 gc-engine, 19 gc-game. Clippy clean.
|
||
|
|
4d66cf6214 |
The tie vertex element is 16 bytes and the layout cycles; correct e5cd3df
Picking up after |
||
|
|
d4aeb94616 |
Extent check inconclusive; the opcode field itself is not understood
Tried to answer whether the decomp extents land on CPU code by checking for a
prologue. Only 29 of 3,835 matched 'addiu $sp,$-N' or 'lui $gp' -- but the other
bucket is full of textbook prologues my two patterns miss, because PS2 leaf
functions save registers with no frame adjustment:
0x2a4ef0 ffb00000 ffb10008 sw $s0,0($sp); sw $s1,8($sp)
0x2a48f0 3c013f80 44816000 lui $1,0x3f80; bc1t
Not reading a result into that.
Separately, ffb00000 has major opcode 0x3f, unused in MIPS32, yet is
unmistakably a register-save prologue and ffbf0020 for 'sw $ra,0x20($sp)' appears
throughout this code. Either the EE encodes sw outside MIPS32 or I am reading the
opcode field wrong somewhere. Not resolved -- and it undercuts every opcode-based
check in this investigation, the 391 and the neighbour ratios included.
Position after twenty-one commits: the mapping is verified, two targets are VU
microcode, and the candidate windows are not established to be CPU code. What this
needs is one authoritative disassembly of one extent compared against
tools/overlay-anchors, to establish what the opcode field means. That is the same
mistake one level down from the section-indexing bug: using a decoding never
checked to produce results, then measuring those results with the same unverified
decoding.
286 passing with the disc present, clippy clean.
|
||
|
|
91b09955c8 |
The shortlist points at VU microcode; the decomp extents are unverified
391 candidates on correct bytes, 73 under 16 immediates. The densest, 0x2dca20,
164 bytes with three distinct immediates, disassembles through tools/overlay-anchors
as:
xori.b $w0, $w0, 0xa2
ext $v0, $s0, 0, 1
addu $a2, $v0, $a2
xori.b and ext are VU1 mnemonics and three-operand addu is not MIPS32. Capstone is
decoding COP2: 0x2dca20 is VU microcode. Same answer 0x2ee138 gave.
That calls the frequency result into question more seriously than the displacement
did. The measurement filtered on opcodes excluding COP2, so VU cannot explain the
391 counts directly -- but the function extents come from the decomp's CPU
analysis, and if an extent spans a VU region the window scanned is mixed and the
immediate set attributed to that function is not a function's.
Status: the enrichment is real as a measurement of those windows; what the windows
are is now doubtful.
Next: verify decomp extents land on CPU code before trusting any of them -- decode
a sample and require a plausible prologue. Same shape as the
|
||
|
|
36d8d2b00c |
The frequency enrichment survives the redo; the disassembly does not
Redone through overlay_bytes_at, the verified flat mapping:
functions kept 2,313
moby_classes 0x00c0 391 (displaced: 394)
best neighbour (0x00c8) 15 (displaced: 4)
bucket n 0xc0 best neighbour ratio (displaced)
0-15 1,033 73 2 36.5x 72.0x
16-31 759 118 4 29.5x 29.8x
32-47 283 89 2 44.5x 34.7x
48-63 118 41 3 13.7x 19.5x
64-79 55 32 2 16.0x 15.5x
80-95 37 22 2 11.0x 13.0x
The reason it survives is worth stating, because it separates the two results. The
displacement was uniform -- every window shifted by the same amount -- so a
function's contents barely changed, and a frequency count over a window is robust
to that. A disassembly is not: it depends on the exact byte alignment, which is
what was wrong. Hence 394 becoming 391 with the same shape, while the 0x3a8060
walk and the 16-byte stride evaporated entirely.
So the investigation was not "all wrong". Precisely:
- the frequency enrichment is real, now on verified bytes, with a test that fails
if it goes;
- the disassembly was never real, and the whole jal chain built on it is retracted;
- no loader has been located, and 0x2ee138 is VU microcode.
THE LESSON IS ABOUT WHICH MEASUREMENTS SURVIVE A MISTAKE
A count over a window tolerates a constant displacement. A decode does not. Sixteen
probes leaned on the kind that does not tolerate it, and reported each with the
same confidence as the kind that does. Nothing about the numbers distinguished them
at the time -- that is the whole difficulty, and why the verification in
|
||
|
|
4a826997f6 |
Redone on correct bytes: the disassembly chain is gone, 0x2ee138 is VU microcode
Re-read through overlay_bytes_at, the verified flat mapping from |
||
|
|
e6b3e0b091 |
Root cause: the per-section indexing used throughout this investigation was wrong
The verification |
||
|
|
f2d7e7ecd4 |
d97e7df was wrong: my section offset was wrong, not the decoder
Ran gc-overlay-anchors on 0x2ee138 as |
||
|
|
d97e7df41f |
Second decode error in the same investigation; stopping rather than trusting this decoder
Following
|
||
|
|
e1482abd75 |
Correct my own decode bug: there is a caller, and r2 is a return value
|
||
|
|
349b498ddd |
r2 is never written in the enclosing function, and it has no callers
The trace
|
||
|
|
eca1a16b17 |
Call sites: one caller, and it does not compute base + 0xc0
0x3a8060 has exactly one call site, at 0x3a7d88:
0x3a7d84 addiu r18, r2, 0x202d r18 = r2 + 0x202d
0x3a7d88 jal 0x3a8060
No 0xc0 in the eight instructions before the call, and r18 is built from a
different offset. So the 0xc0 inside the callee is a field offset applied to a
pointer the caller already built, not a chunk offset the caller resolved.
Two readings remain: r18 is core_data, making r18+0xc0 equal &moby_classes, which
fits since level 0's moby_classes.offset really is 0xc0; or 0xc0 is a member
offset in an unrelated structure and the match is coincidence.
Settling it needs the provenance of r2 at the call site -- real dataflow, not
counting. That is where this stops.
Recorded in docs/plan/level-assets.md. The previous commit's doc edit failed
silently on a line-break mismatch, so this appends rather than replaces.
284 passing with the disc present, clippy clean.
|
||
|
|
cee17c95b1 |
Disassembled: 0xc0 is an address offset, and the walk is 16-byte strided
Closes the ambiguity
|
||
|
|
05d9bf6cd7 |
The 72 concentrated candidates, and a float-constant ambiguity
Looking at the 72 candidates with fewer than 16 distinct immediates, the sets are full of values that look like lui high halves: 0x3f80 is the high half of 1.0f, and 0xff40/0xff70/0xff80/0xffc0 are negative high halves. 0xc0 << 16 is 0x00c00000, a small float. So 0xc0 in these functions may be a float constant rather than a chunk offset. The neighbour control cannot tell the difference, because 0xb8 and 0xc8 are equally plausible float high halves and are rare for their own reasons. Also recorded, and explicitly not as a finding: exactly 1 of 394 candidates contains another chunk offset, and none of the 72 does. That sounds like evidence of moby-specificity and is not -- the other chunk offsets are 0x1e80 and above and are globally rare (0 to 2 functions each, measured in |
||
|
|
a2f8c924b4 |
moby_classes.offset enrichment is size-independent: 13x to 72x at matched size
Runs the control |
||
|
|
06531fb66c |
The 394 does not survive: 0xc0's rate rises with immediate-set size
Ran the size-independent statistic |
||
|
|
db1e69c0ab |
Ranking the 394 by size is backwards; record the confound
Ranking the moby_classes candidates largest-first is the wrong first cut. The top entries are 34,316 and 26,432 bytes with 631 and 421 distinct immediates, median candidate 852. A function with 631 distinct immediates is far more likely to contain any given value, 0xc0 included, than one with 80, purely by pigeonhole. Sorting by size therefore selects for the functions most likely to be false positives, and the 34,316-byte outlier is probably a mis-bounded extent -- the decomp corrects three level-00 entries -- rather than a 34 KB loader. The 394 needs a size-independent statistic: whether a function contains 0xc0 more often than its immediate count predicts, not whether it contains it at all. Not measured. Recorded as a negative about the method rather than committed as progress. No test -- there is no assertion here to make. 283 passing with the disc present, clippy clean. |
||
|
|
3ea5ab799f |
moby_classes.offset is referenced by 394 functions; its neighbours by 1 to 14
First positive result of the loader hunt, and it exists only because of the
control that two commits ago I said was missing.
Across 2,323 level-0 functions of at least 128 bytes -- extents from the decomp's
functions.tsv -- the immediate 0x00c0, which is moby_classes.offset, appears in
394 functions. Every value adjacent to it appears in 1 to 14.
0xb8:8 0xb9:2 0xba:2 0xbb:2 0xbc:3 0xbd:5 0xbe:5 0xbf:3
0xc1:3 0xc2:3 0xc3:2 0xc4:1 0xc5:2 0xc6:3 0xc7:1 0xc8:14
This is what
|
||
|
|
59957a02a2 |
Attribute the 0xc0 immediate sites to functions: 410 candidates
The decomp's functions.tsv makes this cheap -- 3,835 level-0 functions with start
and size, so each immediate-bearing instruction is attributable to its container.
410 of 3,835 functions (10.7%) contain one; sizes 20 to 34,316, median 808.
Ten per cent is what a common small immediate buys, so this confirms
|
||
|
|
56b00fb305 |
Fix the immediate scan by filtering on opcode; still no signal
Saying the previous method 'cannot tell' was right but the reason deserved a step
rather than a hand-wave: it is fixable by counting immediates only where they are
immediates, which needs an opcode filter and no disassembler.
All 27 levels: 11,911,320 instructions, 2,985,897 carrying an immediate
(addiu/lui/ori/xori/andi/sltiu), 13,813 distinct immediate values.
moby_classes 0x00c0 17,699
tie_classes 0x1e80 17
And now there is a control the raw version never had -- the program's most
frequent immediates are 0x0001 (200,784), 0x001b (149,259), 0x0010 (137,523),
0xffff (98,312). 0xc0 at 17,699 sits below the top of that small-value cluster
rather than standing out, so it is not a distinctive reference.
Result unchanged from the raw scan and now defensible: no signal. The loader is
not findable by frequency, and the remaining route is structural dataflow --
functions touching both the chunk offset and a class span.
No test: asserting that an offset is unremarkable is not an invariant worth
locking, and the two numbers are already recorded with their denominator.
|
||
|
|
27891ce5b4 |
Cross-reference scan by raw immediate: the method cannot tell
Sought the moby class loader by counting raw u16 occurrences of each chunk offset in .text. moby_classes 0xc0 hits 49,230 times, tie_classes 0x1e80 hits 111 -- and that contrast means nothing. Small u16 values are simply common: the most frequent value in .text occurs 1,834,535 times. A two-byte value that is frequent by construction beats a rarer one regardless of what either references, so the comparison has no shared denominator. Rule 3a again, one level below where it has bitten before -- not the measurement being wrong but the comparison being uninterpretable. Recorded as a methodological negative rather than a finding. Doing it properly means counting immediates only where the instruction decoder says they are immediates, which is what tools/overlay-anchors exists for. No test added: a test asserting a meaningless comparison would make the next reader trust it. |
||
|
|
743107e864 |
moby_anim: the class table is not the loader
Ruled out the cheap approach: moby_classes (offset 0xc0, count 238 in level 0) holds float-shaped values, and only 120 of the first 4,096 u32s land inside the overlay .text against ~259 expected by chance. Fewer than chance, so the loader is not dispatched from a pointer here. Not concluded what the 238 entries are -- the float reading is suggestive but was taken at one stride on one level. |
||
|
|
6d26da3630 |
The control fails my own claim: 0x1c/0x34 select nothing distinctive
|
||
|
|
2ff2abbd32 |
Two class-header words are base-relative offsets pointing past the tie packets
Chasing where the AD-GIF is, since |
||
|
|
f43a1c2e6e |
bfc_distance is not the control region's length; the AD-GIF is not in the packet
Follows
|
||
|
|
9bb04cca7b |
The tie control region is not a VIF or GIF tag stream; and vert_ofs is mis-named
Went after the GS's GIF packets in the tie AD-GIF region. Got a negative result,
a new universal invariant, and a retraction of my own first probe, which reached
the right answer for the wrong reason twice.
THE NEGATIVE RESULT
Tried the leading `5 * shader_count` quadwords of each packet (the
`bfc_distance` field's stated meaning) as VIF tags -- command in bits 0-6, size
in 7-14, mark in 15 -- and as GIF tags -- type in the top byte of word 1, length
in bits 0-2 of word 0. Both uniform. Over 13,042 records: 10,443 distinct values
of word 0, no GIF tag type above a count of 2. That is the control result rule 4
asks for, and it says stop.
THE RETRACTION
The first probe scanned `bfc_distance` quadwords, which for most records
overruns `vert_ofs` and runs into the vertex data. Pooling a control region with
vertex positions -- two populations with different structure -- gives a uniform
answer, so "not a tag stream" came out right from a measurement that was
measuring the wrong bytes. That is exactly the rule 3a failure, and I only caught
it because the hex dump I took next looked structured while the histogram said
random.
Re-measuring quadword 0 alone, on its own population, gives the same uniformity.
The conclusion survives; the reasoning did not, and both are recorded.
THE PARTITION, AND A NEW INVARIANT
The hex dump showed the packet layout directly:
q[0 .. vert_ofs) control 3..8 quadwords, peak at 5
q[vert_ofs .. +vert_size) vertices
q[vert_ofs+vert_size ..] scissor
and `scissor_ofs == vert_ofs + vert_size` holds on **15,219 of 15,219** records
across all 27 levels, including every one whose strip_count is non-zero. That is a
derived quantity stored explicitly, and it holds everywhere -- which is the check
that confirms the three regions really are laid out the way the existing packet
length formula assumed. It is now a hard check in invariants_hold(), which is
something the previous four could not be.
The fixture in tie.rs failed the moment I added it: it had vert_ofs 4 and
vert_size 0x20 with scissor_ofs 0x30, which is 4 + 0x24. The synthetic data was
never constrained by a relation that did not exist yet. Fixed to 0x24.
vert_ofs IS MIS-NAMED
It is not a vertex offset. It is the *length* of the leading control region, and
the vertex data begins at that many quadwords into the packet. The field name
stays because the on-disc order is fixed and everything already written about this
record says vert_ofs, but the doc now states what the value is and gives the
distribution.
STILL UNKNOWN
What the control region contains. Not standard DMA tags; possibly pre-merged GS
register writes in another framing, or per-class state rather than per-primitive
draw data. Recorded as an open question rather than guessed at.
23 + 19 + 58 + 156 + 22 = 278 passing with the disc present (disc binary 35.67s),
clippy clean.
|
||
|
|
87e2367daf |
Retract the "cutout matters far more" claim; the counts do not support it
|
||
|
|
25d73b4fa8 |
What the disc says about GS priority: cutout matters, blending barely does
The alpha histogram from
|
||
|
|
90ec3e6adf |
The texture sampler was Repeat while its comment claimed it clamped
Wiring the first gs rule into the renderer, and it turned up a bug that had been
sitting in plain sight:
// Only mip level 0 is decoded, so minification must clamp rather than
// invent a level the disc does not have here.
ImageSamplerDescriptor {
mag_filter: ImageFilterMode::Linear,
..default() // <-- address_mode is Repeat
}
The comment describes clamping. The code was doing the opposite: `..default()`
supplies bevy's default address mode, which is ImageAddressMode::Repeat. Nothing
here ever set an address mode, so every texture was sampling with Repeat while a
comment three lines up claimed it was not.
That is the GS rule I wrote in
|
||
|
|
bbd8cdccab |
Phase 5 is Aranos, not Veldin -- Veldin is not on this disc
Caught by the user, and it was wrong in four places: the Phase 5 heading, its gate, its go/no-go line, and the PCSX2 verification plan's "Level 0 (Veldin)". Veldin is Ratchet & Clank's home planet in the 2002 game. Going Commando opens on Aranos, and there is no Veldin anywhere in this disc. The error looks like it carried over from the first game when the roadmap was drafted, which is an easy slip to make and an unhelpful one to leave standing: a vertical slice defined as "completable and matching PCSX2 frame-for-frame" is not something anyone could have attempted or judged while pointing at the wrong planet. Confirmed against the vendored decomp rather than taken on faith. Its docs/LEVELS.md refers to level 00 as Aranos directly -- "corrects three measured Aranos entries before fingerprinting ... These corrections apply to level 00 only" -- so level 0 is Aranos in the reference material this project has been using all along, and the roadmap disagreed with the decomp. There is a real R&C1 reference nearby and it is worth keeping, because the reasoning under the gate is sound and only the noun was wrong. Phase 5 argues that spherical gravity is the thing most likely to break, because it invalidates the movement and camera assumptions the engine inherits from Ratchet & Clank (2002). That is about inherited code from the previous game, not about the level being built, and the text now says so explicitly so the next reader does not have to work out which game is meant. No code change. Documentation only. |
||
|
|
cc52060c67 |
Pin the dispatch counts exactly, after finding two of them were pinned wrong
Follows directly from
|
||
|
|
65025cbf4f |
The vtbl terminator: three test counts, all defensible, one of them wrong
Reconciles the discrepancy flagged when the dispatch tables went in. Three
committed tests counted lvl.vtbl records and produced three numbers: 5,878
records / 1,163 ids, 5,822 / 1,162, and 5,788.
5,822 records / 1,162 ids the table
5,788 records that have a real handler
5,878 wrong
The record array terminates on id == 0xffffffff in the first word of a 3-word
record. Two of the tests reached that word with `if c != MAX { ... }` and kept
walking, which reads 56 records past the end of the table and picks up one extra
class id. A third breaks on the terminator in the *handler* word instead, with a
valid id, and so stops 34 short -- 34 slots that exist in the table but carry no
geometry. Both stopping points are correct for the question each test asks.
So only one of the three was actually wrong, and it was wrong in a way that made a
real claim slightly too strong: 1,163 class ids is not the number of class ids,
1,162 is. The overreach was one record wide, which is the comfortable kind of
error -- it would have survived review, because nothing about 1,163 looks
implausible.
Worth being plain about why this sat unresolved for so long. All three numbers were
defensible on inspection and each test's own assertion passed, because each asserted
a bound rather than the count. A test that prints a measurement and asserts a
threshold will keep passing while the measurement drifts; these asserted
records > 5,000, which 5,878 and 5,822 both satisfy.
Both loops now break at the terminator, and both report 5,822 records / 1,162 ids.
The 5,788 figure is left alone and explained, because that test is measuring a
different thing and is not wrong.
|
||
|
|
31f4ba3f7c |
GS alpha test and blend: all four rules the item names are now present
Finishes the rule layer. Two traps, both of which produce a picture that looks slightly wrong rather than obviously broken: AFAIL READS BACKWARDS. The PS2's alpha-test register holds an enable, a 7-bit reference, and a bit that looks like "fail if alpha is above the reference". On the hardware it means the opposite -- AFAIL = 0 discards where alpha is BELOW the reference. passes_alpha_test takes the side explicitly rather than encoding the register, and both directions are tested so the convention is pinned in code rather than in a comment. THE OPAQUE BIT MUST STAY OUT OF THE COMPARISON. 0x80 is fully opaque, but it is also numerically 128, so a test that compares the raw byte against a 0x40 reference would have every fully-opaque fragment pass every threshold, and nothing would ever be discarded. passes_alpha_test clamps the opaque bit off first, and there is a test that asserts exactly that -- the failure mode here is a scene that renders correctly with the wrong shading, which is not something anyone notices by looking. Blend is the four GS primitive types, decoded from the primitive-type byte and named by what they do rather than by the documentation's IIIB/IIIA/IIIC labels, which name two different modes identically. Out-of-range types decode to Off rather than panicking, because the byte comes from asset data. Addition clamps at 255 rather than rescaling, matching the GS's 7-bit clamp; a caller wanting a true average blends before widening, and the doc says so. That completes the four rules the roadmap names for this item: alpha 0x80 == 1.0 alpha_to_8 CLAMP wrap clamp01 mip selection mip_level blend + alpha test BlendMode, passes_alpha_test Nine unit tests, no wgpu work, and the roadmap item stays [~] rather than [x] -- the rules are the input to the renderer, not the renderer. Worth noting the pattern in both traps: each one produces output that is plausible and slightly-off, with no error anywhere. That is the same shape as the sign inversion in mip_level and the byte-order bug in the anchor search. The tests are the only thing that catches this class, which is the argument for having them even though nothing here needs a GPU. |
||
|
|
81ec17857a |
GC starts: the GS's own rules, as a module with tests
Picked Phase 3's GS item and did the part that is checkable without a GPU -- the rule layer the renderer will have to feed wgpu. New gc_formats::gs with three of the four rules the roadmap names, each unit-tested. alpha_to_8: 0x80 IS OPAQUE. The obvious implementation of this is let v = value & 0x7F; and it is wrong, because 0x80 & 0x7F is 0 -- which is precisely the mechanism that makes a 7-bit alpha read as "half" to anything treating the byte as 8-bit unsigned. The first version of this function did exactly that and the test caught it. Below 0x80 the value is genuinely 7-bit and scales onto 1..255, so 0x7F is opaque too and no non-zero alpha vanishes. clamp01: CLAMP wrap, with 1.0 kept AT the far edge rather than wrapping to 0. Under repeat sampling an exact 1.0 is a discontinuity; under clamp it is the edge texel, and the game loads its textures that way. mip_level: round(log2(image / on_screen)) clamped to the chain. The ratio is image-over-screen and not the reverse -- a SMALLER on-screen size selects a HIGHER mip level. I had this inverted in the first version, which made mip_level(1.0, 64.0, 8) return 0 where it must return 6, and the test caught that too. Both bugs were the same kind: a plausible implementation of a rule whose sign convention is not obvious, and the only thing standing between them and a commit was the test. Not started: wgpu itself, and blend/alpha-test behaviour. This is the rule layer only, and the roadmap item stays [~] rather than being ticked, because three of four rules and zero of the GPU work is not "done". Two of the four rules the item names were already half-present in the renderer -- moby alpha and the 1/4096 UVs -- so making them explicit here rather than scattered is worth something on its own: 0x80's meaning was previously implicit in however each call site happened to read the byte. |
||
|
|
3bc6c2d155 |
Phase 3 gate, second half: cross-check our masking against an independent hash
The gate asks for the relocated-copy result to be cross-checked against an
independent cluster hash. OpenRAC/rac2-gc-decomp's config/levels/functions.tsv is
exactly that: 7,765 functions carrying a content fingerprint and the levels each
appears on. 3,452 appear on all 27, in 2,984 distinct fingerprints.
Cross-checked against our own masking of the level-0 copy of each:
our masked groups 3,389
reference fingerprint groups 2,984
pairs the reference merges, we split 3,231
pairs we merge, the reference splits 0
ZERO is the number that matters. Whenever our masking says two functions are the
same code, the independent fingerprint agrees -- our method never merges what
theirs separates. It is strictly finer: 3,389 groups against 2,984, splitting
3,231 pairs the reference keeps together.
The likely reason ours splits is that the reference fingerprint evidently ignores
something our mask preserves, so functions it calls identical differ in the words
our mask keeps. Coarseness, not contradiction. That is the compatible reading and
it is the one the 0 supports; had it been non-zero, this would have been a
conflict and worth chasing.
THE COMPARISON WAS INVALID BEFORE IT WAS VALID
The first run reported near-unanimous disagreement -- 1,900 of 3,392 pairs
differing -- which read as a significant finding. It was a category error in the
harness: key() masks the FIRST FOUR WORDS, while the reference fingerprints the
WHOLE function. A four-word summary compared against a whole-body hash is not a
comparison of two clusterings, and it produced confident numbers either way.
Recording it because nothing in the invalid output distinguishes it from a real
conflict, and the correction is only visible from the harness, not the numbers.
That is the third time this session that a measurement error produced a result
that looked like a finding -- after the constant-zero validity test and the stride
sampling artifact -- and the common factor is comparing things whose denominators
or extents differ.
The gate's second half is met, with "agrees" meaning "never contradicts" and ours
the finer of the two partitions.
|
||
|
|
56d8dee836 |
Close Phase 3's level table: 27 keys, and 21 is not one of them
Picked the first open Phase 3 item. "Extend the level table from 19 to 27" is
about what the KEYS are, not how many there are.
ReRAC, which this project follows architecturally, has rc_game::map::LEVELS = 19
-- Ratchet & Clank 1's count, and this repo has no equivalent yet. Reading every
container header's own id off the disc gives:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 22 23 24 25 26 30
27 keys, and 21 is NOT one of them. The container files run LEVEL0.WAD through
LEVEL26.WAD, but LEVEL21.WAD reports id 30, so the id space is 0..20, 22..26, 30 --
one number was cut from the disc and reused by nothing. That matches what
LevelHeader::id already documents, now as the table itself.
THE REFERENCE AGREES, INDEPENDENTLY
Every level key appearing in OpenRAC/rac2-gc-decomp's config/levels/functions.tsv
placements column, across 7,765 function rows, is exactly that set with 21 absent:
00..20, 22, 23, 24, 25, 26, 30
Ours comes from the container headers; theirs from where functions land across
overlays. Two unrelated routes to the same key set, which is the only reason to
trust a hand-typed table like this one.
Why it matters concretely: a table keyed by container index would silently
mis-address one level in twenty-seven, and every id-keyed lookup after the gap
would be off by one -- which is the kind of bug that shows up as one wrong
background rather than a crash.
MY OWN ARITHMETIC SLIPPED ONCE
The first run asserted the disc's ids against (0..=20).chain(22..=27).chain([30]),
which is 28 entries -- 22..=27 should be 22..=26. The test caught it against the
disc, which is the right way round. Worth noting that the reference key set I
extracted by hand earlier was correct and the Rust expression was not, so the hand
check and the code check disagreed in the right direction.
Asserted in level_table_has_twenty_seven_keys_with_twenty_one_absent, from the
disc rather than from a copied constant, so it fails if a container id moves.
269 tests, clippy clean.
|
||
|
|
88bc9566f2 |
Narrow moby_anim: the class header has no pointer to anything external
Finished the unblock attempt with two cheap tests on the 187 level-0 class headers, because "which field points to the animation" needed to be ruled in or out properly rather than guessed at. NO HEADER u32 EQUALS THE SPAN'S OWN TABLE OFFSET, on any class. So the varying fields are not self-relative pointers to the part table. THE LARGEST VALUE IN ANY HEADER u32 IS 3,332,678,933. At +0x0c the value is 16,711,748 across classes. That is packed data or an identifier, not a displacement. So the header's varying fields are a MIXTURE: some hold in-span offsets (+0x10 through +0x1c run 192 to 10,800, comfortably inside the spans) and some hold packed values far too large to be offsets. There is no single "field that points to the animation", because not all of them point anywhere. That is a real narrowing even though it is a negative. If per-class animation data exists on the disc it is INSIDE the class span, not at a separate pointer target. So the next step is not "find the pointer" -- it is to subtract the geometry the parser already accounts for from the span and look at what remains. That is a bounded question with an existing control, rather than an open-ended one. moby_anim is still not unblocked. It is now bounded rather than open. The other three blocked items are unchanged and, as recorded, not reachable from here without something this repository does not have: a debug build or an external format description for collision, ground truth for volumes, and a captured frame at a known camera distance for the tfrag morph. Moving to Phase 3 next. Its first unchecked item is "extend the level table from 19 to 27" -- concrete, bounded, and independent of everything above. |