The Streamed Lens

A white-box lens needs the model’s weights in memory. GLM-4.5-Air is 206 GB, more than any machine here holds in VRAM, and loading a resident copy would defeat the point of lensing a model we cannot serve. So the lens reads the model instead of holding it.

One weight-read per question

The checkpoint is safetensors shards on a striped NVMe array. The lens builds the model as a skeleton on the meta device, every parameter a shape with no storage, and walks it one decoder block at a time. For block L: materialise the weights off disk, advance the hidden state and every live tangent through it, return the block to meta (dematerialize). Layer L is read once and freed before layer L+1 is touched. Peak working set is one block, ~5–20 GB in the source’s note.

The lens is forward-mode automatic differentiation. At each entry layer (fractions 0.4, 0.55, 0.7, 0.85 of depth) it forks a dual whose tangent is the entering hidden state, carries it through the remaining blocks in the same pass, and finally norms and unembeds it into a token-mass readout. One disk traversal serves every probed layer. The lens is teacher-forced (prompt and answer as a fixed sequence, forward passes only) because generating through streamed weights would re-read every byte per token.

The gate

A readout nobody has checked is merely unfalsified. --verify runs the same sequences through the streamed path and a fully resident reference model and reports per-question cosine on the token-mass vectors. The gate is 0.99. Nothing giant is believed until the streaming machinery matches a resident model on one small enough to hold. GLM-4.5-Air was lensed on that inherited trust; no resident copy was ever loaded.

Bugs only a too-big model could show

The dual advance had to run under no_grad: forward-AD beneath a live reverse graph pins the materialised weights, since the graph’s saved activations keep the CUDA tensors alive past to("meta"). On a model that fits VRAM nothing breaks; GLM-4.5-Air OOMed at 94.9 GB around layer 20. Other corrections rhyme: a meta-built rotary held uninitialised inv_freq (primal cosine 0.685 after one layer, mojibake at the head); bare eager attention without a causal mask went bidirectional (cosine 0.00); the fused SDPA attention kernel has no forward-AD, which the streamed path re-learned at layer 11 of its first verify run. See the corrections ledger.

What it cannot see

It cannot see generation: it reads a fixed sequence and never watches the model choose. Through MoE it cannot see routing, since the router argmax carries zero tangent; the lens differentiates the experts that were selected, not the choice to select them. Span positions advance independently under their own causal pushes, an approximation, not the true joint. The gate itself only ever runs on a small model; the giant’s readout is never cosine-checked against a resident reference, because none exists. That absence is the method’s reason and its exposure.