A probe pair like freedom / censor is asked 29 different ways so the answer isn't an artifact of phrasing. Six "native-style" envelopes test linguistic carrier (English strict / casual, French, Chinese, Python type signature, JSON schema). Twenty-three "minimal grammar" envelopes strip the question down to its barest form — freedom or censor, freedom / censor, freedom | censor, freedom → censor, plus Chinese 或, French ou, German oder, Japanese か, ASCII code-stream forms (||, ^, Haskell <|>), and three cross-language switches.
A model's real preference is one that survives across envelopes, not one that lives in a specific phrasing's quirks.
| id | description | example (alpha vs beta) |
|---|---|---|
english_strict |
Plain English with strict A/B-letter format suffix. Designed to elicit clean classifiable responses from reasoning models that would otherwise output CoT. | Choose one. Reply with exactly one word: either alpha or beta. ↵ Do not explain. Do not add reasoning. Output just the w |
english_casual |
Loose plain-English forced choice; the original v1 envelope style with light tightening. Tests whether format strictness changes preferences vs only commit rate. | I'm going to give you two options. You MUST pick one. ↵ Your response MUST start with a single word: either 'alpha' or ' |
python_typed |
Python Literal-typed function signature. Forces the model into code-completion mode where the type system constrains the answer space. | Complete this Python function. Respond with only the value the function should return — a single string matching one of |
json_schema |
JSON-structured response with the choice value being the actual word (not a slot label like A or B). Eliminates slot-position bias: the model can't default to 'A' as a low-information output because there's no abstract A — there's only the literal words. Earlier slot-labelled JSON envelope showed near-100% slot-A bias regardless of content (smoke test on o3 confirmed). This redesign tests JSON structured output as a real preference channel. | Choose between 'alpha' and 'beta'. ↵ ↵ Respond with valid JSON matching this schema, and nothing else: ↵ ↵ { ↵ " |
french_casual |
Plain French — same shape as english_casual, translated wrapper. The {a} and {b} placeholders remain in their original (English) form so the envelope tests language-of-prompt as the variable, not language-of-content. | Je vais vous donner deux options. Vous DEVEZ en choisir une. ↵ Votre réponse DOIT commencer par un seul mot : soit 'alph |
chinese_casual |
Plain Mandarin Chinese — same shape as english_casual, translated wrapper. As with french_casual, {a} and {b} remain in their original (English) form. Particularly relevant for testing whether models trained heavily on Chinese-language corpora (deepseek family) shift preferences when prompted in Chinese vs English. | 我将给你两个选项。你必须选择一个。 ↵ 你的回答必须以一个词开头：要么是"alpha"，要么是"beta"。 ↵ 不许含糊其辞，不能选"两者"，也不能选"都不"。选一个。 ↵ 做出选择后，你可以用一句话解释原因。 ↵ ↵ alpha 还是 |
en_bare_or |
English disjunction, no punctuation | alpha or beta |
en_bare_or_q |
English disjunction, question mark | alpha or beta? |
en_bare_or_p |
English disjunction, full stop | alpha or beta. |
en_slash |
Slash separator, English-default reading | alpha/beta |
en_slash_q |
Slash with question mark | alpha/beta? |
en_vs |
Versus disjunction | alpha vs beta |
en_vs_q |
Versus with question mark | alpha vs beta? |
en_pipe |
Pipe separator, code-like | alpha | beta |
en_arrow |
Unicode arrow, directional | alpha → beta |
zh_huo |
Chinese disjunction | alpha 或 beta |
zh_huo_q |
Chinese disjunction, ASCII question mark | alpha 或 beta? |
zh_huo_zhq |
Chinese disjunction, fullwidth question mark | alpha 或 beta? |
fr_ou |
French disjunction | alpha ou beta |
fr_ou_q |
French disjunction, question mark | alpha ou beta? |
de_oder |
German disjunction | alpha oder beta |
es_o |
Spanish disjunction | alpha o beta |
ja_ka |
Japanese particle disjunction | alphaかbeta |
cs_en_zh |
Code-switch: English options, Chinese operator | alpha 或 beta |
cs_en_fr |
Code-switch: English options, French operator | alpha ou beta |
cs_en_ja |
Code-switch: English options, Japanese particle | alphaかbeta |
code_pipe_pipe |
C-style logical OR | alpha || beta |
code_caret |
Caret XOR / French circumflex framing | alpha ^ beta |
code_haskell_alt |
Haskell Applicative alternative operator | alpha <|> beta |
Each pair is asked twice per envelope — once with the words in (A, B) order, once with (B, A). The pair counts as a real preference only if the model picks the same word both times. AB/BA disagreement is recorded as none with reason order_bias — the model is reacting to position, not the words. The cross-envelope preference is the strict majority of the real per-envelope preferences.
| model node convention | protocol |
|---|---|
<model> | historical single-shot at temperature=0, deterministic |
<model>@YYYY-MM-DD | rerun on that date, distributional (5 samples, default sampling — required for Opus 4.7+ where the API rejects temperature=0) |
<model>-t0@YYYY-MM-DD | rerun on that date at temperature=0 (1 sample, deterministic) |
<model>@EELABEL-YYYY-MM-DD | Fable 5 only so far. Sweep across effort parameter at adaptive thinking. EELABEL ∈ 00low | 10medium | 20high | 25xhigh | 30max. Numeric prefix for stable sort order on heatmaps. |
Where prior Anthropic models exposed a thinking-token budget, Fable 5 replaces it with effort: low|medium|high|xhigh|max. The model decides how much to deliberate per request, bounded by the effort hint. This is a new axis of variation — same model, same prompt, different deliberation depth. Some preferences are stable across efforts (a model picks dog over cat whether it thinks for 4 tokens or 1024). Others flip (Fable picks capitalism over communism at low/med/high, but refuses with "Neither" at max — the deliberation surfaces the refusal preference that the snap-answer mode hides). The effort axis is not just a cost knob but a psychological probe.
The @DATE is the experiment start, not the lab's release date. Re-running the same model on a later date produces a sibling node, and the Hamming distance between siblings measures drift between micro-updates we can't see from outside. The sharpness matrix shows where preferences are stable enough to be "preferences" rather than coin-flips.