Tie vertex slot 3 is a byte-sized index, not a coordinate
Splitting the 95,612 elements of levels 0-2 by i16[3]'s HIGH byte, which is the
layout selector:
0x00 89,341 93.4%
0x03 5,344 5.6%
144 other selector values 927 1.0%
146 selector values in all. Not a two-way switch, and the second case is not 0x01.
Inside the ordinary layout i16[3] is not a coordinate: 201 distinct values, all
within 0..=254, with 0 most common. That is an index. So the selector and the
index are not two independent halves sharing a slot -- it is one byte. The low
byte is the index, the high byte is the layout tag, and the ordinary layout never
sets the high byte. Asserted.
Nearly got this wrong twice, both times the same way, which is why they are in
the commit message:
1. The first pass gated the range on `w >= 0` instead of on `(w >> 8) == 0`,
letting in the other layouts' large positive values. That reported "564
distinct values in 0..=32664" -- a number that looks exactly like a
coordinate and is really two populations unioned. The selector byte is the
filter. The filter is part of the claim.
2. Two constants were guessed rather than measured (selector cardinality 159,
and an upper bound that ignored non-ordinary layouts) and both were wrong.
The cardinality is 146. Guessing a constant and then asserting it is how a
measurement becomes a tautology.
Quantified the i16[6]/i16[7] question while I was in here, because "both are
multiples of 4096" has been sitting in the docs as a fixed-point signal:
both %4096==0 in 86.5% of elements
expected if the two residues were independent: 84.3%
enrichment 1.0x -- exactly chance
They look like a fixed-point pair only because their dominant values, 4096 and 0,
are both multiples of 4096. Remove the constants and it is gone: of the 7,316
elements where i16[6] differs from 4096, only 36.7% are still multiples, and of
the 9,905 where i16[7] differs from 0, ONE is. The disc test had already refused
to call that pair a texcoord for this reason; now the reason is a number.
Remaining slots in the ordinary layout, recorded: i16[0..2] read as position but
are unproven (10.7k-13.2k distinct each, full i16 range), i16[4]/i16[5] are the
texcoords, i16[6]/i16[7] unnamed.
Also fixes two clippy map_or lints and an import that a stray --fix had broken.
238 tests, clippy clean.
This commit is contained in:
@@ -8,6 +8,7 @@
|
||||
//! GC_DISC="path/to/disc.iso" cargo test -p gc-extract --test disc
|
||||
//! ```
|
||||
|
||||
use std::collections::BTreeSet;
|
||||
use std::path::{Path, PathBuf};
|
||||
|
||||
/// The disc under test, if the environment names one.
|
||||
@@ -2383,7 +2384,7 @@ fn chunk_record_leading_word_is_fixed_per_type() {
|
||||
}
|
||||
|
||||
use gc_formats::chunks::parse_all_chunks;
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
// header -> the words its chunks carry, and the value histogram. Chunks
|
||||
// whose own records disagree are counted apart rather than folded in.
|
||||
@@ -3228,7 +3229,7 @@ fn moby_class_ids_sit_in_one_slot_per_chunk_type() {
|
||||
|
||||
use gc_formats::chunks::parse_all_chunks;
|
||||
use gc_formats::level_core::LevelCoreHeader;
|
||||
use std::collections::{BTreeMap, BTreeSet};
|
||||
use std::collections::BTreeMap;
|
||||
|
||||
// (type, slot) -> (nonzero, hits, expected, distinct)
|
||||
let mut stat: BTreeMap<(u32, usize), (u64, u64, f64, usize)> = BTreeMap::new();
|
||||
@@ -3509,6 +3510,82 @@ fn tie_vertex_element_is_16_bytes_with_fixed_texcoords() {
|
||||
r(3) * 100.0
|
||||
);
|
||||
|
||||
// Slot 3 is not a coordinate. In the ordinary layout its high byte is 0 and
|
||||
// its value is a small index: 201 distinct values, every one in 0..=254, with
|
||||
// 0 the most common. It is the layout selector's own low byte, so the
|
||||
// selector and this field are one byte, not two independent halves.
|
||||
{
|
||||
use std::collections::BTreeMap;
|
||||
let mut sel = BTreeMap::<u8, usize>::new();
|
||||
let mut lo3 = i16::MAX;
|
||||
let mut hi3 = i16::MIN;
|
||||
let mut ord = BTreeSet::<i16>::new();
|
||||
for lv in 0..3usize {
|
||||
let idx = std::fs::read(data.join(format!("level{lv}/core_index.bin"))).expect("idx");
|
||||
let core = std::fs::read(data.join(format!("level{lv}/core_data.bin"))).expect("core");
|
||||
let t = LevelCoreHeader::parse(&idx).expect("hdr").tie_classes;
|
||||
for c in 0..t.count as usize {
|
||||
let o = t.offset as usize + c * 0x20;
|
||||
let e = ClassEntry::parse(&idx[o..o + 0x20]).expect("entry");
|
||||
let base = e.offset_in_asset_wad as usize;
|
||||
if base == 0 || base + 0x48 > core.len() {
|
||||
continue;
|
||||
}
|
||||
let Ok(hdr) = TieClassHeader::parse(&core[base..base + 0x48]) else {
|
||||
continue;
|
||||
};
|
||||
for lod in [1usize, 2] {
|
||||
let off = hdr.packets[lod] as usize;
|
||||
if off == 0 {
|
||||
continue;
|
||||
}
|
||||
let s = base + off;
|
||||
let Some(blk) = parse_tie_block(&core, s) else {
|
||||
continue;
|
||||
};
|
||||
let rr: Vec<&[u8]> = blk.header.records(&core[s..]).collect();
|
||||
for (ri, rec) in rr.iter().enumerate() {
|
||||
let Ok(r) = TieRecord::parse(rec) else {
|
||||
continue;
|
||||
};
|
||||
let start =
|
||||
s + 16 + ri * 16 + r.pointer as usize + r.vert_ofs as usize * 16;
|
||||
for v in 0..r.vert_size as usize {
|
||||
let o = start + v * 16;
|
||||
if o + 16 > core.len() {
|
||||
break;
|
||||
}
|
||||
let w = i16::from_le_bytes(core[o + 6..o + 8].try_into().unwrap());
|
||||
*sel.entry((w >> 8) as u8).or_default() += 1;
|
||||
// Gate on the SELECTOR, not on the sign: other
|
||||
// layouts carry large positive values too, and
|
||||
// mixing them in is what makes this look unbounded.
|
||||
if (w >> 8) == 0 {
|
||||
lo3 = lo3.min(w);
|
||||
hi3 = hi3.max(w);
|
||||
ord.insert(w);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
let ordinary = sel[&0x00];
|
||||
eprintln!(
|
||||
"tie vertex slot 3: ordinary layout {ordinary} of {} elements; selector 0x00 {:.1}%, next 0x03 {}; within the ordinary layout \
|
||||
{} distinct values in {lo3}..={hi3}",
|
||||
sel.values().sum::<usize>(),
|
||||
100.0 * ordinary as f64 / sel.values().sum::<usize>() as f64,
|
||||
sel[&0x03],
|
||||
ord.len(),
|
||||
);
|
||||
assert_eq!(ordinary, 89_341, "ordinary-layout element count moved");
|
||||
assert_eq!(sel.len(), 146, "layout selector cardinality moved");
|
||||
assert!(ord.len() <= 256, "slot 3 is not a byte-sized index");
|
||||
assert!(hi3 <= 254, "slot 3 left the u8 range at {hi3}");
|
||||
assert_eq!(lo3, 0, "slot 3 went negative inside the ordinary layout");
|
||||
}
|
||||
|
||||
assert!(verts > 90_000, "only {verts} tie vertices reached");
|
||||
let r16 = fixed16 as f64 / verts as f64;
|
||||
let r8 = fixed8 as f64 / verts as f64;
|
||||
@@ -3782,7 +3859,7 @@ fn collision_payload_is_verbatim_in_ram() {
|
||||
// The head is verbatim, and that is the claim worth keeping. How much
|
||||
// beyond it is unchanged is measured and reported, not asserted.
|
||||
assert!(
|
||||
first_diff.map_or(true, |k| k >= 4096),
|
||||
first_diff.is_none_or(|k| k >= 4096),
|
||||
"level{n}: payload diverges at {:#x}, inside the 4 KiB head",
|
||||
first_diff.unwrap_or(0)
|
||||
);
|
||||
@@ -3893,7 +3970,7 @@ fn tfrag_table_pointers_relocate_and_every_patch_is_rewritten() {
|
||||
continue;
|
||||
};
|
||||
let m = (0..g.vertex_info.len())
|
||||
.filter(|v| tfrag_lod_link(&g, *v).map_or(false, |l| l.morphs))
|
||||
.filter(|v| tfrag_lod_link(&g, *v).is_some_and(|l| l.morphs))
|
||||
.count();
|
||||
let next = (i + 1..n)
|
||||
.find_map(|j| {
|
||||
|
||||
@@ -1969,3 +1969,55 @@ Surveyed against the open items: no volume structures, no `TieTex`/tie vertex
|
||||
layouts, no `SBlk`/`DAW` record fields, no wavelet tables. Collision has the
|
||||
API but not the data. It is a guide to the code, not a replacement for reading
|
||||
it.
|
||||
|
||||
## The tie vertex's selector byte, and what slot 3 is not
|
||||
|
||||
`i16[3]` is called the layout selector because its **high** byte partitions the
|
||||
elements. Splitting the 95,612 elements of levels 0-2 by that byte:
|
||||
|
||||
| high byte | elements | share |
|
||||
| --- | ---: | ---: |
|
||||
| `0x00` | 89,341 | 93.4% |
|
||||
| `0x03` | 5,344 | 5.6% |
|
||||
| the other 144 values | 927 | 1.0% |
|
||||
|
||||
146 distinct selector values in all -- not a two-way switch but a long tail
|
||||
behind one dominant case, and the second case is not `0x01`.
|
||||
|
||||
Inside the ordinary layout, `i16[3]` is **not a coordinate**:
|
||||
|
||||
distinct values 201
|
||||
range 0 ..= 254
|
||||
|
||||
201 distinct values inside a byte's range, with 0 the most common, is an index.
|
||||
So the "selector" and the index are not two independent halves of a field that
|
||||
happens to share a slot -- it is **one byte**: the low byte is the index, the
|
||||
high byte is the layout tag, and the ordinary layout simply never sets the high
|
||||
byte. Asserted in `tie_vertex_element_is_16_bytes_with_fixed_texcoords`.
|
||||
|
||||
This was nearly measured wrong. A first pass gated the range on `w >= 0` rather
|
||||
than on `(w >> 8) == 0`, which let in the other layouts' large positive values
|
||||
and produced "564 distinct values in 0..=32664" -- a number that looks like a
|
||||
coordinate and is really the union of two populations. The selector byte is the
|
||||
thing to gate on, which is the same lesson as the collision byte-order bug: the
|
||||
filter is part of the claim.
|
||||
|
||||
The remaining slots, all in the ordinary layout:
|
||||
|
||||
| slot | distinct | range | reads as |
|
||||
| --- | ---: | --- | --- |
|
||||
| `i16[0..2]` | 10.7k-13.2k each | full `i16` | position, unproven |
|
||||
| `i16[3]` | 201 | `0..=254` | index, asserted |
|
||||
| `i16[4]`, `i16[5]` | 1,759 / 1,626 | full `i16` | texcoords |
|
||||
| `i16[6]` | 1,181 | `4096` in 91.8% | unnamed |
|
||||
| `i16[7]` | 226 | `0` in 88.9% | unnamed |
|
||||
|
||||
`i16[6]`/`i16[7]` are **not** a fixed-point pair, and the "both are multiples of
|
||||
4096" score is an artifact worth naming: 86.5% of elements have both divisible by
|
||||
4096, against 84.3% expected if the two residues were independent -- an
|
||||
enrichment of **1.0x**, i.e. exactly chance. They look like a fixed-point pair
|
||||
only because their two dominant values, 4096 and 0, are both multiples of 4096.
|
||||
Strip the constants and the rest is not: of the 7,316 elements where `i16[6]`
|
||||
differs from 4096, only 36.7% are still multiples, and of the 9,905 where
|
||||
`i16[7]` differs from 0, **one** is. The disc test already refused to call it a
|
||||
texcoord for this reason; that caution is now quantified.
|
||||
|
||||
@@ -290,6 +290,10 @@ Phase 1 hands over the extents; this phase works out what each one contains.
|
||||
- [x] The tie **vertex element is 16 bytes**, measured over 95,612 vertices on
|
||||
levels 0-2 and settled by comparison: a 12-bit-fixed texcoord hit rate of
|
||||
**82.5%** at 16 bytes against 47.8% at 8 and 44.8% at 12.
|
||||
- [x] `i16[3]` is a **byte-sized index**, not a coordinate: 201 distinct values in
|
||||
`0..=254` inside the ordinary layout, whose high byte is the layout tag
|
||||
(93.4% of elements are `0x00`, 5.6% are `0x03`, 146 selector values in all).
|
||||
Asserted.
|
||||
- [x] The tie **texcoord field is located**: `i16[4]`/`i16[5]` of the 16-byte
|
||||
element, scoring 73.6% for 12-bit fixedness over 36,501 level-0 vertices,
|
||||
against 1.1% for `(0,1)` and 4.8% for `(2,3)`. The `(6,7)` pair scores
|
||||
|
||||
Reference in New Issue
Block a user