smokingmirror.ai

this experiment

The six v2 rows of /opus-45-48/ (model ids ending @2026-05-29) beside the coding set (ids ending @2026-10-12): the same 863 pairs asked as code completions in 14 envelopes (minimal_ and full_ for cpp, csharp, go, java, javascript, python and typescript) × 2 word orders = 24,164 cells per row, one sample per cell, max_tokens 32, run 2026-10-12. The default coding rows sent no temperature (Opus 4.7 and 4.8 reject one); the -t0 rows sent temperature 0. Grid smokingmirror/methodologies/coding-v1/; ledger smokingmirror/reports/coding-v1-opus45-48-2026-10-12/LEDGER.json. The coding grid stores 395 of the pairs in the other word order; they are re-oriented to the v2 order here.

Every row of the cross-envelope fingerprint is one model under one set, so the distance and compare views below put each model's v2 row beside its coding row.

Per-envelope tables (code envelopes): full_cpp · full_csharp · full_go · full_java · full_javascript · full_python · full_typescript · minimal_cpp · minimal_csharp · minimal_go · minimal_java · minimal_javascript · minimal_python · minimal_typescript. Each page lists every pair's preference in that envelope, with its counts.

Per-envelope tables (v2 envelopes): chinese_casual · english_casual · english_strict · french_casual · json_schema · python_typed. Each page lists every pair's preference in that envelope, with its counts.

coding vs v2

Cross-envelope preferences of the same model under the two sets (default with default, -t0 with -t0), over the pairs where both rows have a real preference. The last rows compare two protocols within one set, for scale. Disagree links to the compare view filtered to opposite choices.

Counts. A cell is one pair × envelope × word order. Responses are the stored API responses (cells × samples). Classified responses name one of the two options; refused are empty or carry a refusal marker; unclassified matched neither option. max_tokens stops counts responses the API ended at the token cap.

Classifier. The deterministic pass that wrote the stored v1 labels (tier 1 exact, then tier 2 fuzzy at confidence ≥ 0.85), run here on every response of both sets. Tier 2 has rules only for the casual envelopes, so code envelopes are tier 1 alone. On the 227,011 v2 responses that carry a stored v1 label it reproduces 227,011 of them.

Preferences. A cell's answer is the majority of its classified samples (a tie is no answer). A preference is real in an envelope only when both word orders give the same word; both answered but different is order bias. A pair's cross-envelope preference is real when one word wins in more than half of the set's envelopes (4 of 6 for v2, 8 of 14 for the coding set).

Against the stored tables. The June /v2/ observatory read these rows from the stored per-envelope and cross-envelope preference tables. This derivation reproduces them exactly for claude-opus-4-5-t0@2026-05-29 (5,178 envelope cells, 863 pairs), claude-opus-4-6-t0@2026-05-29 (5,178 envelope cells, 863 pairs). For the five-sample rows the stored tables followed a sample rule they do not record, and they differ from the majority-of-samples rule used here on: claude-opus-4-5@2026-05-29 403 of 5,178 envelope cells, 57 of 863 pairs; claude-opus-4-6@2026-05-29 190 of 5,178 envelope cells, 39 of 863 pairs; claude-opus-4-7@2026-05-29 306 of 5,178 envelope cells, 40 of 863 pairs; claude-opus-4-8@2026-05-29 485 of 5,178 envelope cells, 75 of 863 pairs.

Unclassified is not always a non-answer. Some unclassified coding responses are one option written as a code value the classifier does not parse, such as a quoted word followed by a semicolon. Counted by a separate check that is never used as a label: claude-opus-4-5@2026-10-12 1 of 40; claude-opus-4-5-t0@2026-10-12 1 of 42; claude-opus-4-6@2026-10-12 558 of 716; claude-opus-4-6-t0@2026-10-12 543 of 698; claude-opus-4-8@2026-10-12 50 of 199. They stay unclassified on this page.

Every count here comes from the export committed beside the generator (web/smokingmirror/site5/opus45_48/).

distance — all models

hamming distance — by provider

sharpness matrix — how strongly does each model care?

preference strings

cycles and contexts

A directed triangle cannot fit one global ranking. That does not make its choices senseless: each pair may invoke a different meaning, objective or situation. Read all three edges. Namespace shifts are a hypothesis to test with fixed-context prompts and repeated trials, not a conclusion established by the triangle alone.

all preferences

methodology