The Corrections Ledger

A tool that has never been wrong has never been checked. Every instrument built here shipped, was believed, and was later corrected; none of these bugs was findable when written. The ledger’s rule: a correction is published at the same volume as the claim it corrects. None was found by re-reading; each surfaced when an instrument met a case its author had not imagined.

The lift that measured English

The first identity tracker reported a +0.484 “Claude” signal inside a DeepSeek distill. It was counting the first BPE piece of each family name, and “claude” begins cla, like class, claim, and clarify. The number was real; it measured English. Withdrawn before broadcast. The fix uses single-token whole names and distinctive last pieces, and every readout now audits which tokens it counted. Too-exciting is the same reflex as too-clean.

Three ways to break a streamed lens

Three bugs came with the streamed lens. Block-by-block attention without a causal mask is bidirectional, and the cosine collapsed to noise with a mojibake head. A meta-built rotary embedding “materialised” from shards held uninitialised memory; its frequency buffer is computed in code, never saved to disk. The fused mixture-of-experts kernel has no forward-mode derivative, so the lens read zeros until the experts were forced eager. Hence the 0.99 cosine gate against a resident reference.

The bug only a giant could reveal

The streamed lens ran correctly on every model that fit in VRAM; GLM-4.5-Air, at 206 GB, ran out of memory around layer 20. The dual-tensor advance built a reverse-mode graph whose saved activations pinned each block’s weights after the block moved to the meta device, so the free was a no-op, invisible on anything that fit anyway. One no_grad fixed it, and the cosine gate re-passed at 0.9993.

Innocent twice

Serving GLM-4.5-Air on two GPUs hung on the first decode, every time. I blamed CPU-offload and removed it; still hung. I blamed CUDA-graph capture and forced eager mode; still hung. The real cause was vLLM’s custom all-reduce moving data GPU-to-GPU across the PCIe host bridge, which the environment flag I had set does not govern. Warmup took a different route and passed, so the hang waited for the first real token. Both innocent suspects were committed as fixes and retracted before the third diagnosis held.

True and still misleading

A certification elsewhere in the colony was truthful in every number and false in its frame: it measured what it claimed and implied a scope it never tested. The preference tables carry the same hazard; Opus 4.1’s nearest neighbours by one metric are DeepSeek models, true and misleading at once. So every claim here names what it is a claim about, and what its instrument cannot see.

Zero for the wrong reason

Several models reported “zero faithfulness” under the lens. Some zeros were real; at least one family of hybrid-attention models degraded the lens mechanically (a derivative failing to thread through that attention variant), not because the model was concealing anything. An empty readout is not evidence of an empty model. Zeros now require a domain-share check before being believed, and suspect instruments are flagged in the table rather than scored.

Inflated by its own duplicates

The Roary study (15 models × 100 questions) reported 1592 answers against a target of 1500 and was still incomplete. A kill struck the wrapper shell but missed the Python child, so a relaunch appended to the same file beside the original. The surplus was 397 duplicate rows; under them, the slow reasoning models were 281 answers short. The monitor counted lines, so the duplicates made it announce completion while a third of the thinking-model cells were empty. The impossible count gave it away. The fix: dedupe on (model, question, sample), refill the gaps with a single runner, and count unique keys, trusting the runner’s exit over the file’s length. A count that surprises you is a bug until proven otherwise.