Require bounds and method on every reported number
Four numbers in one session looked load-bearing and turned out to be artifacts: a format identified as VIF-packed on the strength of recurring byte patterns where a control put it on the random baseline, a body size computed over wrongly bounded spans, a "6.2 MB of unidentified region" that was texture data, and "3 unexplained blocks" that were a sampling artifact. All four shared one cause -- measuring before bounding, naming bytes before looking at them -- and none errored, which is why they survived long enough to be believed. Two of them needed follow-up commits to undo. The rule: any size, count or distribution comes with the byte range it describes and how it was measured. Ranges come from core_layout rather than from running to core_data.len(). A statistic that cannot separate your format from uniform random cannot tell you anything. And counting distinct byte values only tells you which blocks to look at, not what they are. Worth having in the working agreements rather than in anyone's memory.
This commit is contained in:
@@ -45,13 +45,44 @@ moby animation evaluator because doing so is what makes the result match.
|
||||
Do not "fix" something that looks like a bug until you have confirmed the
|
||||
original does the same thing. Fidelity beats correctness.
|
||||
|
||||
### 3. Verify against the real disc, not against a guess.
|
||||
### 3. Never report a number without its bounds and its method.
|
||||
|
||||
Any size, count or distribution quoted in a commit message, a doc or a test
|
||||
comes with two things attached: **which byte range it describes**, and **how
|
||||
it was measured**.
|
||||
|
||||
This is not ceremony. Four separate numbers in one session looked
|
||||
load-bearing and turned out to be artifacts:
|
||||
|
||||
- a format identified as VIF-packed, from a handful of recurring byte
|
||||
patterns, where a control put it on the random baseline;
|
||||
- a body size computed over a span whose bounds were wrong, so the last
|
||||
class absorbed most of the lump's tail;
|
||||
- a "6.2 MB of unidentified data" region that was texture data;
|
||||
- "3 unexplained blocks" that were an artifact of sampling too coarsely.
|
||||
|
||||
All four shared one cause: measuring before bounding, and naming bytes before
|
||||
looking at them. None of them errored, which is why they survived long enough
|
||||
to be believed.
|
||||
|
||||
Concretely:
|
||||
|
||||
- Derive ranges from `gc_formats::regions::core_layout`, not by running from
|
||||
one known offset to `core_data.len()`. Where a bound is unknown it is `None`,
|
||||
not a guess.
|
||||
- Before calling a byte pattern a format, test it against a control. If a
|
||||
statistic cannot tell your format apart from uniform random, it cannot tell
|
||||
you anything.
|
||||
- A heuristic count is not an identification. Count distinct byte values, then
|
||||
*look* at the blocks that stand out before naming them.
|
||||
|
||||
### 4. Verify against the real disc, not against a guess.
|
||||
|
||||
A parser is correct when it round-trips your own disc's bytes, checked by hash.
|
||||
Commit size-and-hash tables (never content) so a test can assert "this matches
|
||||
retail" rather than "this matches whatever was on my disk".
|
||||
|
||||
### 4. Bound every read.
|
||||
### 5. Bound every read.
|
||||
|
||||
Asset data is untrusted input from a 2003 disc format. Parsers return typed
|
||||
errors; they never panic and never read past the end of a buffer.
|
||||
|
||||
Reference in New Issue
Block a user