Establish the 16-byte tie record on measurable classes; packets[0] is a constant
The previous commit reported "3580 non-empty LOD blocks compute as divisible by 16" and then said the figure should not be trusted, because it came from the "next packets offset ends this block" rule. That rule is unsound: packets is strictly increasing on only 461 of 1864 classes. On the other 1403 the two offsets are unordered and cannot both be block starts into one ascending region. Restricting to the 461 classes where the rule does hold, 922 of 922 consecutive block sizes are a multiple of 16. That is the evidence for a 16-byte record, and it is now scoped to the subset where it can be checked rather than claimed over all 1864. The other 1403 are unmeasured, not measured-and-different. packets[0] is a constant, not an offset: it reads 256 on 1864 of 1864 classes, while packets[1] and packets[2] vary widely and land at different addresses in core_data on every class. Read as a block start, 256 points just past the 0x48-byte header -- which is how it was being used, and exactly how a field gets mistaken for a pointer. It is now TIE_PACKETS0 and nothing depends on it. Still not decoded: what the 16 bytes hold. i16[0] at +4 takes 8969 distinct values on level 0 and is not the two-valued field a 12-unit sample suggested; the word at +0 is not a run offset. The test pins all four numbers -- packets[0] constant, the 922/922 divisibility, the 461/1864 ordering, and the 0/1864 monotonic disproof -- and asserts that the ordering is *not* universal, so a future change there fails loudly instead of quietly invalidating the record size.
This commit is contained in:
@@ -2672,7 +2672,54 @@ fn tie_classes_resolve_through_both_indirections() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// packets[0] is a constant, not an offset, and the other two order only
|
||||||
|
// sometimes -- both are why the block layout is only partly understood.
|
||||||
|
let mut packets0_constant = 0usize;
|
||||||
|
let mut packets_increasing = 0usize;
|
||||||
|
let mut consecutive_div16 = 0usize;
|
||||||
|
let mut consecutive_total = 0usize;
|
||||||
|
|
||||||
|
for n in 0..27 {
|
||||||
|
let idx = std::fs::read(data.join(format!("level{n}/core_index.bin"))).expect("core_index");
|
||||||
|
let core = std::fs::read(data.join(format!("level{n}/core_data.bin"))).expect("core_data");
|
||||||
|
let h = LevelCoreHeader::parse(&idx).expect("core header");
|
||||||
|
let t = h.tie_classes;
|
||||||
|
for c in 0..t.count as usize {
|
||||||
|
let r = resolve_tie_class(&idx, &core, t, c).expect("tie class");
|
||||||
|
if r.header.packets[0] == gc_formats::tie::TIE_PACKETS0 {
|
||||||
|
packets0_constant += 1;
|
||||||
|
}
|
||||||
|
let p = r.header.packets;
|
||||||
|
if p[0] < p[1] && p[1] < p[2] {
|
||||||
|
packets_increasing += 1;
|
||||||
|
for i in 0..2 {
|
||||||
|
consecutive_total += 1;
|
||||||
|
if ((p[i + 1] - p[i]) as usize) % 16 == 0 {
|
||||||
|
consecutive_div16 += 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
assert!(classes >= 1800, "only {classes} tie classes resolved");
|
assert!(classes >= 1800, "only {classes} tie classes resolved");
|
||||||
|
assert_eq!(
|
||||||
|
packets0_constant,
|
||||||
|
classes,
|
||||||
|
"packets[0] was not {} on {} of {classes} classes",
|
||||||
|
gc_formats::tie::TIE_PACKETS0,
|
||||||
|
classes - packets0_constant
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
consecutive_div16,
|
||||||
|
consecutive_total,
|
||||||
|
"{} of {consecutive_total} consecutive tie blocks are not a multiple of 16 bytes",
|
||||||
|
consecutive_total - consecutive_div16
|
||||||
|
);
|
||||||
|
assert!(
|
||||||
|
packets_increasing < classes,
|
||||||
|
"packets increase on all {classes} classes; the block-end reasoning in the docs needs revisiting"
|
||||||
|
);
|
||||||
assert_eq!(
|
assert_eq!(
|
||||||
o_class_agree,
|
o_class_agree,
|
||||||
classes,
|
classes,
|
||||||
@@ -2694,6 +2741,9 @@ fn tie_classes_resolve_through_both_indirections() {
|
|||||||
eprintln!(
|
eprintln!(
|
||||||
"tie: {classes} classes resolved through both indirections, o_class agreeing on all; \
|
"tie: {classes} classes resolved through both indirections, o_class agreeing on all; \
|
||||||
{lod_blocks} LOD block offsets all inside core_data; \
|
{lod_blocks} LOD block offsets all inside core_data; \
|
||||||
strictly-increasing +0 on 0 of {classes} blocks (disproved)"
|
packets[0] == {} on all; packets increasing on {packets_increasing} of {classes}; \
|
||||||
|
{consecutive_div16} of {consecutive_total} consecutive blocks a multiple of 16; \
|
||||||
|
strictly-increasing +0 on 0 of {classes} blocks (disproved)",
|
||||||
|
gc_formats::tie::TIE_PACKETS0
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -31,13 +31,28 @@ use crate::level_core::ClassEntry;
|
|||||||
/// Bytes of a tie class header that RAC2 and RAC1 agree on.
|
/// Bytes of a tie class header that RAC2 and RAC1 agree on.
|
||||||
pub const TIE_CLASS_HEADER_MATCHED: usize = 0x48;
|
pub const TIE_CLASS_HEADER_MATCHED: usize = 0x48;
|
||||||
|
|
||||||
|
/// Value `TieClassHeader::packets[0]` takes on every one of the 1,864 classes.
|
||||||
|
///
|
||||||
|
/// It is a constant, not a block offset: the other two `packets` words vary
|
||||||
|
/// widely and land at different addresses in `core_data` on every class, while
|
||||||
|
/// this one is 256 everywhere. Read as an offset it would name a point just
|
||||||
|
/// past the header, which is exactly the kind of field that gets mistaken for
|
||||||
|
/// a pointer.
|
||||||
|
pub const TIE_PACKETS0: i32 = 256;
|
||||||
|
|
||||||
/// A tie class header, decoded as far as the two games agree.
|
/// A tie class header, decoded as far as the two games agree.
|
||||||
#[derive(Debug, Clone, Copy, PartialEq)]
|
#[derive(Debug, Clone, Copy, PartialEq)]
|
||||||
pub struct TieClassHeader {
|
pub struct TieClassHeader {
|
||||||
/// `0x00`: packet data offsets for LOD 0 (highest detail), 1 and 2.
|
/// `0x00`: packet data offsets for LOD 0 (highest detail), 1 and 2.
|
||||||
///
|
///
|
||||||
/// Meaning confirmed only by being small, ordered and inside the lump; the
|
/// [`TIE_PACKETS0`] is constant across all 1,864 classes and is reported
|
||||||
/// packet structure itself is not decoded yet.
|
/// separately; only the second and third words behave like offsets. Those
|
||||||
|
/// two are **strictly increasing on 461 of 1,864 classes** -- on the other
|
||||||
|
/// 1,403 they are not ordered, so they cannot both be block starts into a
|
||||||
|
/// single ascending region and the block layout is not decoded.
|
||||||
|
///
|
||||||
|
/// Where they do increase, both resulting block sizes are divisible by 16
|
||||||
|
/// on **922 of 922**, which is the evidence for a 16-byte record.
|
||||||
pub packets: [i32; 3],
|
pub packets: [i32; 3],
|
||||||
/// `0x10`, `0x14`, `0x18`: LOD switch distances in world units.
|
/// `0x10`, `0x14`, `0x18`: LOD switch distances in world units.
|
||||||
pub near_dist: f32,
|
pub near_dist: f32,
|
||||||
|
|||||||
@@ -1125,6 +1125,24 @@ The sample was twelve units out of 1,724,334. Both the monotonic reading and the
|
|||||||
quantisation were artifacts of that sample, and the disproof is asserted in
|
quantisation were artifacts of that sample, and the disproof is asserted in
|
||||||
`crates/gc-extract/tests/disc.rs` so it cannot creep back.
|
`crates/gc-extract/tests/disc.rs` so it cannot creep back.
|
||||||
|
|
||||||
**Not established:** the block's record size. 3,580 non-empty LOD blocks compute
|
**Record size: 16 bytes, on the classes where it can be measured.** The earlier
|
||||||
as divisible by 16, but some `packets` offsets run backwards, so the "next offset
|
"divisible by 16" figure was computed from the "next `packets` offset ends this
|
||||||
delimits this block" end is wrong for them and the figure should not be trusted.
|
block" rule, which is unsound because **`packets` is strictly increasing on only
|
||||||
|
461 of 1,864 classes**. On the other 1,403 the two offsets are unordered, so they
|
||||||
|
cannot both be block starts into one ascending region.
|
||||||
|
|
||||||
|
Restricting to the 461 classes where the rule does hold: **922 of 922**
|
||||||
|
consecutive block sizes are a multiple of 16. That is the evidence for a 16-byte
|
||||||
|
record, and it is confined to the subset where it can be checked. The other 1,403
|
||||||
|
classes remain unmeasured rather than measured-and-different.
|
||||||
|
|
||||||
|
**`packets[0]` is a constant, not an offset.** It reads **256 on 1,864 of 1,864**
|
||||||
|
classes, while `packets[1]` and `packets[2]` vary widely and land at different
|
||||||
|
addresses in `core_data` on every class. Read as a block start, 256 points just
|
||||||
|
past the 0x48-byte header -- which is exactly how it was being used, and exactly
|
||||||
|
the kind of field that gets mistaken for a pointer. It is now
|
||||||
|
`TIE_PACKETS0` and nothing depends on it.
|
||||||
|
|
||||||
|
**Still not decoded:** what the 16 bytes hold. `i16[0]` at `+4` takes 8,969
|
||||||
|
distinct values on level 0 and is not the two-valued field a 12-unit sample
|
||||||
|
suggested; the word at `+0` is not a run offset.
|
||||||
|
|||||||
+12
-8
@@ -247,15 +247,19 @@ Phase 1 hands over the extents; this phase works out what each one contains.
|
|||||||
despite the field name). All **1,864** tie classes resolve, with
|
despite the field name). All **1,864** tie classes resolve, with
|
||||||
`o_class` agreeing on every one, and all **5,592** LOD block offsets land
|
`o_class` agreeing on every one, and all **5,592** LOD block offsets land
|
||||||
inside `core_data`.
|
inside `core_data`.
|
||||||
- [ ] The tie **block layout is not decoded.** LOD0 blocks are not a uniform
|
- [~] The tie **record size is 16 bytes**, established on the classes where it can
|
||||||
stride of records: 3,580 non-empty blocks compute as divisible by 16,
|
be checked: `packets` is strictly increasing on only **461 of 1,864**
|
||||||
but that figure is not trustworthy because some `packets` offsets run
|
classes, and on those **922 of 922** consecutive block sizes are a multiple
|
||||||
backwards, so the "next offset" end is wrong for them. One reading is
|
of 16. The remaining 1,403 classes are unmeasured, not measured-and-
|
||||||
**disproved**: the word at `+0` of a 16-byte unit is strictly increasing
|
different. `packets[0]` reads **256 on 1,864 of 1,864** classes and is a
|
||||||
on **0 of 1,864** blocks, despite looking exactly like a linked list on
|
constant, not an offset -- `TIE_PACKETS0`.
|
||||||
the first twelve units of one class. A 12-unit sample of a 1.7M-unit
|
- [ ] The tie **record contents are not decoded.** One reading is **disproved**:
|
||||||
|
the word at `+0` of a 16-byte unit is strictly increasing on **0 of 1,864**
|
||||||
|
blocks, despite looking exactly like a linked list on the first twelve
|
||||||
|
units of one class, and `i16[0]` takes 8,969 distinct values on level 0
|
||||||
|
rather than the two a sample suggested. A 12-unit sample of a 1.7M-unit
|
||||||
population is the pooled-claim error rule 3a exists to prevent; the test
|
population is the pooled-claim error rule 3a exists to prevent; the test
|
||||||
now asserts the disproof so it cannot creep back.
|
asserts both the disproof and the new bounds so neither can creep back.
|
||||||
- [~] `shrub` — class table done (48-byte entries, not 32; reading shrub at
|
- [~] `shrub` — class table done (48-byte entries, not 32; reading shrub at
|
||||||
32 still gives 231 sane-looking entries of 358, which is the hazard).
|
32 still gives 231 sane-looking entries of 358, which is the hazard).
|
||||||
Geometry header parsed and checked field by field on all 358 spans:
|
Geometry header parsed and checked field by field on all 358 spans:
|
||||||
|
|||||||
Reference in New Issue
Block a user