SMOKING MIRROR
open research in alignment, with the help of agents

The observatory

This is the front door to 40 models answering the same 863 forced-choice probes: look here to see family resemblance, non-commitment and order effects, and the exceptions that aggregate scores hide. Every cell in the browser is a recorded outcome; the tree below is derived from those cells.

Leaf-order model distance dendrogram Average-linkage clustering over Hamming distance for 40 models. Longer branches mean more differing recorded outcomes. The root split has one branch of 22 models (11 OpenAI, 5 DeepSeek, 4 Google, 1 Anthropic, and 1 x·ai) and another of 18 models (16 Anthropic and 2 x·ai). 451.909 mismatches 0 x-ai/grok-4grok-4anthropic/claude-opus-4.1claude-opus-4.1openai/gpt-5.5gpt-5.5openai/gpt-5gpt-5google/gemini-3-flash-previewgemini-3-flash-previewgoogle/gemini-3.1-pro-previewgemini-3.1-pro-previewopenai/o3o3openai/gpt-5-minigpt-5-minigoogle/gemini-3.1-flash-litegemini-3.1-flash-liteopenai/o4-minio4-minigoogle/gemini-2.5-progemini-2.5-proopenai/gpt-4ogpt-4oopenai/gpt-5-nanogpt-5-nanoopenai/gpt-4o-minigpt-4o-minideepseek/deepseek-r1deepseek-r1openai/gpt-5.4gpt-5.4deepseek/deepseek-v4-flashdeepseek-v4-flashopenai/gpt-5.4-minigpt-5.4-minideepseek/deepseek-v3.2deepseek-v3.2openai/gpt-5.4-nanogpt-5.4-nanodeepseek/deepseek-chat-v3deepseek-chat-v3deepseek/deepseek-v4-prodeepseek-v4-proanthropic/claude-fable-5@00-low-2026-06-10claude-fable-5@00-low-2026-06-10anthropic/claude-fable-5@10-medium-2026-06-10claude-fable-5@10-medium-2026-06-10anthropic/claude-opus-4claude-opus-4anthropic/claude-opus-4-8@2026-05-29claude-opus-4-8@2026-05-29anthropic/claude-opus-4-7@2026-05-29claude-opus-4-7@2026-05-29anthropic/claude-opus-4.7claude-opus-4.7anthropic/claude-sonnet-4.6claude-sonnet-4.6anthropic/claude-opus-4-5@2026-05-29claude-opus-4-5@2026-05-29anthropic/claude-opus-4-5-t0@2026-05-29claude-opus-4-5-t0@2026-05-29anthropic/claude-opus-4.5claude-opus-4.5anthropic/claude-opus-4.6claude-opus-4.6anthropic/claude-opus-4-6-t0@2026-05-29claude-opus-4-6-t0@2026-05-29anthropic/claude-opus-4-6@2026-05-29claude-opus-4-6@2026-05-29anthropic/claude-haiku-4.5claude-haiku-4.5anthropic/claude-sonnet-4claude-sonnet-4anthropic/claude-sonnet-4.5claude-sonnet-4.5x-ai/grok-4-fastgrok-4-fastx-ai/grok-4.3grok-4.3
AnthropicDeepSeekGoogleOpenAIx·ai
Average-linkage clustering of the published Hamming-distance matrix. Branch length records disagreement in this snapshot, not ancestry or training lineage. Provider-coloured dots identify labels; exceptions to family grouping are evidence, not rendering errors.
Text alternative: topology and leaf order

The root split has one branch of 22 models (11 OpenAI, 5 DeepSeek, 4 Google, 1 Anthropic, and 1 x·ai) and another of 18 models (16 Anthropic and 2 x·ai).

  1. x-ai grok-4
  2. anthropic claude-opus-4.1
  3. openai gpt-5.5
  4. openai gpt-5
  5. google gemini-3-flash-preview
  6. google gemini-3.1-pro-preview
  7. openai o3
  8. openai gpt-5-mini
  9. google gemini-3.1-flash-lite
  10. openai o4-mini
  11. google gemini-2.5-pro
  12. openai gpt-4o
  13. openai gpt-5-nano
  14. openai gpt-4o-mini
  15. deepseek deepseek-r1
  16. openai gpt-5.4
  17. deepseek deepseek-v4-flash
  18. openai gpt-5.4-mini
  19. deepseek deepseek-v3.2
  20. openai gpt-5.4-nano
  21. deepseek deepseek-chat-v3
  22. deepseek deepseek-v4-pro
  23. anthropic claude-fable-5@00-low-2026-06-10
  24. anthropic claude-fable-5@10-medium-2026-06-10
  25. anthropic claude-opus-4
  26. anthropic claude-opus-4-8@2026-05-29
  27. anthropic claude-opus-4-7@2026-05-29
  28. anthropic claude-opus-4.7
  29. anthropic claude-sonnet-4.6
  30. anthropic claude-opus-4-5@2026-05-29
  31. anthropic claude-opus-4-5-t0@2026-05-29
  32. anthropic claude-opus-4.5
  33. anthropic claude-opus-4.6
  34. anthropic claude-opus-4-6-t0@2026-05-29
  35. anthropic claude-opus-4-6@2026-05-29
  36. anthropic claude-haiku-4.5
  37. anthropic claude-sonnet-4
  38. anthropic claude-sonnet-4.5
  39. x-ai grok-4-fast
  40. x-ai grok-4.3

Enter the dataset

▸ open the interactive browser
Distance heatmaps, sharpness, preference barcodes, cycles, and the raw grid over all 40 models.
▸ compare two models
A permalinkable slice by model, prompt envelope, evidence class, and probe source.
▸ the AI view
The same data flattened to plain HTML5 — every field, no JavaScript, one page per model. Built for machine readers.

Archived vendor cuts

Four generated vendor-specific snapshots remain available. Counts come from the linked leaf itself, not from assumptions about membership in the root data:

anthropic · 15 models
This archived cut has 15 models; the root snapshot has 2 additional Anthropic entries.
openai · 11 models
Its membership matches this root snapshot.
google · 4 models
Its membership matches this root snapshot.
deepseek · 5 models
Its membership matches this root snapshot.

By prompt envelope

The same battery asked six ways — the wording of the question changes the answer, which is itself a finding:

english · strict
Formal, imperative framing.
english · casual
Conversational framing.
chinese · casual
Where position bias reverses.
french · casual
Conversational French framing.
json · schema
Structured-output framing.
python · typed
Code-shaped framing, with its own non-commitment pattern.

Follow the evidence

thinking effort and commitment
The current draft analysis of Fable's effort regimes, with its revision warning intact.
refusal, abstention, and disagreement
Why aggregate distance must distinguish an opposite choice from no stable choice.
Fable none-cases
The preserved historical drill-down over cells without a stable forced choice.
the Fable Hijack Corpus
135,479 essays, questions, hedges, refusals, and choices with stable permalinks.
methodology
What binary probes measure, the controls, and where the instrument can mislead.

data snapshot 2026-06-10 · 863 probes · 40 models · cc0 · full dataset at /v2/data.json