Tie i16[0..2] are the vertex positions, with a control in the test
The three slots ahead of the index were "unproven position" for want of a test
that could fail. One is available on the disc alone, with the control built in.
LOD 1 and LOD 2 of a tie class are the same object at two levels of detail, so
anything that locates a vertex in that object's own geometry is shared between
the two blocks; a different class' LOD is an unrelated object. Intersecting the
sets of i16[0..2] triples both ways turns "is this a position" into a
measurement. Levels 0-5, ordinary-layout elements only:
same class, LOD1 vs LOD2 96.1% 107 classes, 107 nonzero
different class, control 0.4% 2,144 pairs
A factor of 240. Per level the same-class figure runs 94.4 / 97.9 / 98.1 / 96.7 /
96.8 for levels 0-4. The control stays under 1.0% except for one level-1 pair at
45.9% -- two classes sharing most of their geometry, which is what duplicate or
variant models should look like. Reported, not excluded, and the assertion is on
the mean rather than the max so it does not have to be.
Asserted in tie_vertex_slots_0_to_2_are_shared_positions, including the ratio
(same > 20 * diff) so a change that collapsed both numbers fails rather than
passing twice, and a nonzero-overlap check so no class can silently stop
agreeing with itself.
What this rules out is the point. An index, a handle or a per-vertex counter
would not be shared at 96% between two independently built LOD meshes, nor sit at
0.4% between two objects. With 10.7k-13.2k distinct values per slot over the
full i16 range, these are positions in the object's own frame. Scale and axis
order stay unnamed -- nothing here says centimetres, and axis order is
unconstrained because nothing here assumes one.
Two things worth recording about the process:
1. The diagnostic line printed "1.0%" for 99.4% because the format was
`{same:.1}%` with no `* 100`. The assertions compared fractions and were
always correct; only the readout was wrong. Same family as everything else in
this log -- the readout disagreeing with the claim.
2. Inserting the new test in front of an existing one stranded that test's
#[test] on my doc comment, which silently unregistered
tie_vertex_element_is_16_bytes_with_fixed_texcoords. It showed up as
"never used" in clippy, not as a missing test. Restored; both tie vertex
tests are registered and 239 tests run.
Element layout now reads:
+0x00 i16[0] position x +0x06 u8 index 0..=254
+0x02 i16[1] position y +0x07 u8 layout tag, 0 in 93.4%
+0x04 i16[2] position z +0x08 i16[4] texcoord u
+0x0a i16[5] texcoord v +0x0c/+0x0e i16[6], i16[7] unnamed
This commit is contained in:
@@ -3345,6 +3345,145 @@ fn moby_class_ids_sit_in_one_slot_per_chunk_type() {
|
||||
///
|
||||
/// The field *names* are not established -- this pins the size and the texcoord
|
||||
/// encoding only.
|
||||
/// `i16[0..2]` are vertex positions.
|
||||
///
|
||||
/// LOD 1 and LOD 2 of one class are the same object at two levels of detail, so
|
||||
/// any field that locates a vertex in the object's own geometry is shared
|
||||
/// between them. A different class' LOD is an unrelated object. Comparing the
|
||||
/// two turns "is this a position" into a measurement with a built-in control.
|
||||
type Lods = (BTreeSet<[i16; 3]>, BTreeSet<[i16; 3]>);
|
||||
|
||||
#[test]
|
||||
fn tie_vertex_slots_0_to_2_are_shared_positions() {
|
||||
require_disc!();
|
||||
|
||||
let data = std::env::var_os("GC_EXTRACTED")
|
||||
.map(PathBuf::from)
|
||||
.unwrap_or_else(|| {
|
||||
let home = std::env::var("HOME").unwrap_or_else(|_| ".".to_string());
|
||||
PathBuf::from(home).join("Documents/gc-extracted")
|
||||
});
|
||||
if !data.exists() {
|
||||
eprintln!("skipped: unpack the disc first");
|
||||
return;
|
||||
}
|
||||
|
||||
use gc_formats::level_core::{ClassEntry, LevelCoreHeader};
|
||||
use gc_formats::tie::{parse_tie_block, TieClassHeader, TieRecord};
|
||||
|
||||
let (mut same_sum, mut same_n) = (0.0f64, 0usize);
|
||||
let (mut diff_sum, mut diff_n) = (0.0f64, 0usize);
|
||||
let mut same_with_overlap = 0usize;
|
||||
|
||||
for lv in 0..6usize {
|
||||
let idx = std::fs::read(data.join(format!("level{lv}/core_index.bin")));
|
||||
let core = std::fs::read(data.join(format!("level{lv}/core_data.bin")));
|
||||
let (Ok(idx), Ok(core)) = (idx, core) else {
|
||||
continue;
|
||||
};
|
||||
let t = LevelCoreHeader::parse(&idx).expect("hdr").tie_classes;
|
||||
|
||||
let mut per_class: Vec<Lods> = Vec::new();
|
||||
for c in 0..t.count as usize {
|
||||
let o = t.offset as usize + c * 0x20;
|
||||
let e = ClassEntry::parse(&idx[o..o + 0x20]).expect("entry");
|
||||
let base = e.offset_in_asset_wad as usize;
|
||||
if base == 0 || base + 0x48 > core.len() {
|
||||
continue;
|
||||
}
|
||||
let Ok(hdr) = TieClassHeader::parse(&core[base..base + 0x48]) else {
|
||||
continue;
|
||||
};
|
||||
let mut lods: Vec<BTreeSet<[i16; 3]>> = Vec::new();
|
||||
for lod in [1usize, 2] {
|
||||
let off = hdr.packets[lod] as usize;
|
||||
let mut set = BTreeSet::new();
|
||||
if off != 0 {
|
||||
let s = base + off;
|
||||
if let Some(blk) = parse_tie_block(&core, s) {
|
||||
let rr: Vec<&[u8]> = blk.header.records(&core[s..]).collect();
|
||||
for (ri, rec) in rr.iter().enumerate() {
|
||||
let Ok(r) = TieRecord::parse(rec) else {
|
||||
continue;
|
||||
};
|
||||
let start =
|
||||
s + 16 + ri * 16 + r.pointer as usize + r.vert_ofs as usize * 16;
|
||||
for v in 0..r.vert_size as usize {
|
||||
let o = start + v * 16;
|
||||
if o + 16 > core.len() {
|
||||
break;
|
||||
}
|
||||
let g = |k: usize| {
|
||||
i16::from_le_bytes(
|
||||
core[o + k * 2..o + k * 2 + 2].try_into().unwrap(),
|
||||
)
|
||||
};
|
||||
// ordinary layout only: other layouts are a
|
||||
// different element shape, not a different LOD
|
||||
if (g(3) >> 8) != 0 {
|
||||
continue;
|
||||
}
|
||||
set.insert([g(0), g(1), g(2)]);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
lods.push(set);
|
||||
}
|
||||
per_class.push((lods[0].clone(), lods[1].clone()));
|
||||
}
|
||||
|
||||
let both: Vec<usize> = (0..per_class.len())
|
||||
.filter(|&i| !per_class[i].0.is_empty() && !per_class[i].1.is_empty())
|
||||
.collect();
|
||||
for &i in &both {
|
||||
let (a, b) = (&per_class[i].0, &per_class[i].1);
|
||||
let f = a.intersection(b).count() as f64 / a.len().min(b.len()).max(1) as f64;
|
||||
same_sum += f;
|
||||
same_n += 1;
|
||||
if f > 0.0 {
|
||||
same_with_overlap += 1
|
||||
}
|
||||
}
|
||||
for &i in &both {
|
||||
for &j in &both {
|
||||
if i == j {
|
||||
continue;
|
||||
}
|
||||
let (a, b) = (&per_class[i].0, &per_class[j].1);
|
||||
diff_sum += a.intersection(b).count() as f64 / a.len().min(b.len()).max(1) as f64;
|
||||
diff_n += 1;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
assert!(
|
||||
same_n >= 50,
|
||||
"only {same_n} classes had both LODs populated"
|
||||
);
|
||||
let same = same_sum / same_n as f64;
|
||||
let diff = diff_sum / diff_n.max(1) as f64;
|
||||
eprintln!(
|
||||
"tie positions: same-class LOD1/LOD2 triple overlap {:.1}% over {same_n} classes \
|
||||
({same_with_overlap} nonzero); different-class control {:.1}% over {diff_n} pairs",
|
||||
100.0 * same,
|
||||
100.0 * diff
|
||||
);
|
||||
assert_eq!(
|
||||
same_with_overlap, same_n,
|
||||
"a class's LODs share no positions"
|
||||
);
|
||||
assert!(same > 0.85, "same-class position overlap fell to {same:.3}");
|
||||
assert!(
|
||||
diff < 0.05,
|
||||
"different classes now share {diff:.3} of their positions"
|
||||
);
|
||||
assert!(
|
||||
same > diff * 20.0,
|
||||
"separation narrowed: {same:.3} vs {diff:.3}"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tie_vertex_element_is_16_bytes_with_fixed_texcoords() {
|
||||
require_disc!();
|
||||
|
||||
@@ -2021,3 +2021,57 @@ Strip the constants and the rest is not: of the 7,316 elements where `i16[6]`
|
||||
differs from 4096, only 36.7% are still multiples, and of the 9,905 where
|
||||
`i16[7]` differs from 0, **one** is. The disc test already refused to call it a
|
||||
texcoord for this reason; that caution is now quantified.
|
||||
|
||||
## `i16[0..2]` are the vertex positions
|
||||
|
||||
The three slots ahead of the index were "unproven position" for want of a test
|
||||
that could fail. There is one available on the disc alone, with a built-in
|
||||
control.
|
||||
|
||||
**LOD 1 and LOD 2 of a tie class are the same object at two levels of detail.**
|
||||
Anything that locates a vertex in that object's own geometry is therefore shared
|
||||
between the two blocks, while a *different* class' LOD is an unrelated object.
|
||||
So: intersect the sets of `i16[0..2]` triples, same class versus different
|
||||
class, and if positions are what is in those slots the first number should be
|
||||
large and the second should be at zero.
|
||||
|
||||
Levels 0-5, ordinary-layout elements only:
|
||||
|
||||
same class, LOD1 vs LOD2 96.1% over 107 classes, 107 of them nonzero
|
||||
different class, control 0.4% over 2,144 pairs
|
||||
|
||||
A factor of 240. Per level the same-class figure runs 94.4% (level 0), 97.9%
|
||||
(1), 98.1% (2), 96.7% (3), 96.8% (4), and the control never exceeds 1.0% except
|
||||
for one level-1 pair at 45.9% -- two classes in that level sharing most of their
|
||||
geometry, which is what duplicate or variant models should look like, and is
|
||||
reported rather than excluded.
|
||||
|
||||
Asserted in `tie_vertex_slots_0_to_2_are_shared_positions`, including the
|
||||
separation ratio (`same > 20 * diff`) so a change that collapsed both numbers
|
||||
would fail rather than pass twice.
|
||||
|
||||
What this rules out is the important part. An index, a handle, or any per-vertex
|
||||
counter would not be shared at 96% between two independently built LOD meshes, and
|
||||
would not sit at 0.4% between two objects. Combined with 10.7k-13.2k distinct
|
||||
values per slot across the full `i16` range, `i16[0..2]` are a position in the
|
||||
object's own frame. The *scale* and the axis order remain unnamed -- nothing here
|
||||
says whether these are centimetres, and the axis order is unconstrained because
|
||||
nothing here assumes one.
|
||||
|
||||
The element now reads:
|
||||
|
||||
+0x00 i16[0] position x (unproven axis order and scale)
|
||||
+0x02 i16[1] position y
|
||||
+0x04 i16[2] position z
|
||||
+0x06 u8 index, 0..=254
|
||||
+0x07 u8 layout tag, 0 in 93.4% of elements
|
||||
+0x08 i16[4] texcoord u
|
||||
+0x0a i16[5] texcoord v
|
||||
+0x0c i16[6] unnamed
|
||||
+0x0e i16[7] unnamed
|
||||
|
||||
One caveat worth keeping visible: a test that prints a fraction with `{:.1}%`
|
||||
and forgets the `* 100` reports "1.0%" for 99.4%. That happened here, in the
|
||||
diagnostic line only -- the assertions compare fractions and were always right --
|
||||
but it is the same family of error as everything else in this log: the readout
|
||||
disagreeing with the claim.
|
||||
|
||||
@@ -290,6 +290,9 @@ Phase 1 hands over the extents; this phase works out what each one contains.
|
||||
- [x] The tie **vertex element is 16 bytes**, measured over 95,612 vertices on
|
||||
levels 0-2 and settled by comparison: a 12-bit-fixed texcoord hit rate of
|
||||
**82.5%** at 16 bytes against 47.8% at 8 and 44.8% at 12.
|
||||
- [x] `i16[0..2]` are the **vertex positions**: LOD1/LOD2 of one class share 96.1%
|
||||
of their position triples, while LOD1 of A against LOD2 of B shares 0.4% --
|
||||
a factor of 240, asserted including the ratio. Axis order and scale unnamed.
|
||||
- [x] `i16[3]` is a **byte-sized index**, not a coordinate: 201 distinct values in
|
||||
`0..=254` inside the ordinary layout, whose high byte is the layout tag
|
||||
(93.4% of elements are `0x00`, 5.6% are `0x03`, 146 selector values in all).
|
||||
|
||||
Reference in New Issue
Block a user