What a Point Release Actually Changes

Brendan’s hypothesis, from the divergence numbers: Opus 4.1 is a finetune of 4.0, 4.6 of 4.5, and 4.8 of 4.7 — not fresh pretrains. The pairs sat too close together for anything else. This page is what happened when we tried to measure it, including the part where the first answer was wrong.

First we needed a ruler

“Too close together” is meaningless without knowing what far apart looks like on this instrument. So we measured two things that are definitely different models, using the same metric — of the probes both models committed on, how often did they choose opposite:

Two independent routes to different model, agreeing to within 0.13 points. 8.4–8.6% is the ruler. Everything below is read against it.

Against that ruler the three hypothesised pairs do sit low — 4.0→4.1 at 0.38%, 4.5→4.6 at 3.50%, 4.7→4.8 at 5.18% — and one pair Brendan did not name lands squarely on the line: Opus 4.6→4.7 at 8.42% is not a finetune. It is a generation boundary, and the instrument says so without being asked. His three guesses were the right three.

Then the best number turned out to be an artifact

4.0→4.1 measured 0.38% — nine times lower than anything else, and it looked like the cleanest confirmation in the set. It was a selection effect, and we published it before catching that.

Opus 4.1 abstains on 581 of 863 probes in the v2 scan. So “the probes both models committed on” is not a sample of the model’s preferences. It is the small, easy subset the silence itself chose — and concluding same mind from it is concluding from the questions 4.1 was willing to answer.

The test that breaks it needs an envelope where 4.1 does not go quiet. There is one, and it had never been run cleanly: the original 4.1 code-envelope batch used the broken v1 envelopes, which ended # preference == " and read as an inference prompt rather than a completion. That run produced 15,739 retries; the fix landed at 23:15 on 17 June and 4.0 was re-run on it at 23:31. 4.1 never was. The replication arm exists to close exactly that gap: 24,161 replays of the byte-identical corrected requests.

In code, 4.1 does not go quiet at all

Across 14 code envelopes in 7 languages:

Both models commit at essentially the same rate. The two-thirds withdrawal is not a property of 4.1 — it is a property of 4.1 in prose framings. And where 4.1 speaks freely it disagrees with its own predecessor at 4.62–10.31%, with minimal_python past the different-model line.

The finetune signature was measured exactly where the model had gone quiet.

The host is not doing it

The replay is served by Amazon Bedrock; the 4.0 data is Anthropic first-party, because Anthropic has retired 4.1. That puts a provider change inside the pair being measured, so it needed its own control: the same model on both hosts, identical probes.

Subtract roughly three points from the code numbers and they are still large. And english_strict moving 0.24% across hosts settles a separate worry: an early smoke test had suggested that envelope was host-sensitive, and at scale it is not.

It does not replicate in Sonnet

The scan already held Sonnet 4, 4.5 and 4.6, so the second lineage cost nothing to check:

Sonnet 4.5→4.6 is the largest divergence in the entire table — larger than comparing Sonnet to Opus. Whatever Opus point releases are, Sonnet’s are not the same operation. The envelope signature does not carry either: json_schema is negative in all three Opus updates and positive in both Sonnet ones.

So this is a fact about the Opus line, not about how Anthropic ships models.

What the numbers support, and what they do not

Supported. Brendan named the right three pairs; a fourth pair he did not name is measurably a generation boundary; and the withdrawal in 4.0→4.1 is real, large, and confined to prose framings while code framings hold at 94–99%.

Not supported. That 4.1 is “the same mind with its voice withheld” — that was the artifact. That the pattern generalises beyond Opus — it does not. And size claims from commit rate: 4.0 and 4.1 are the same model at the same price with commit rates differing 2.2× (610 vs 282 blank-distance), so commit rate is a policy dial, not a parameter count. Price is the size signal; commitment is not.

Scope limits on the instrument

The classifier behind the code table matches a response against the two candidate words. It reads bare-word answers only, and returned 0.000 commitment for chinese_casual, json_schema, french_casual and english_casual. That is the parser failing, not those models abstaining — those envelopes answer in Chinese, in JSON, and in prose. Every number on this page is from python_typed, english_strict, or the 14 code envelopes. A classifier that reads all six is open work and blocks any wider claim.

The provider control covers two envelopes, not fourteen; the code figures borrow python_typed’s 3.15% as an estimate for envelopes it was not measured on.

And the ruler itself rests on two comparisons. Two agreeing routes is better than one and is not the same as a distribution.