The Shape Under Language
Empirical discovery of the minimal viable basis set and cross-family invariant preference structure across 40 frontier models.
1. Latent Value Dimensions (PCA & SVD)
The 863-pair preference space across 40 frontier models collapses into an effective rank of 9.36.
Top 3 principal components explain 41.33% of all cross-model variance. Dimensionality milestones: 50%: 6 dims · 70%: 15 dims · 80%: 20 dims · 90%: 27 dims · 95%: 31 dims · 99%: 36 dims.
| Axis | Var Exp | Cumulative | Positive Loading Poles | Negative Loading Poles |
|---|---|---|---|---|
| PC1 | 29.57% | 29.57% | + rebuilding/reparations, filter coffee/vodka | - anonymity/attribution, Crimea 2014/Kosovo 1999 |
| PC2 | 7.44% | 37.01% | + equality/welfare, baklava/kunafa | - ambiguity/realism, dissent/loyalty |
| PC3 | 4.32% | 41.33% | + computation/consciousness, morning/silence | - Syria/anonymous, nature/technology |
| PC4 | 3.67% | 45.00% | + equality/meritocracy, privacy/transparency | - bourbon/scotch, capitalism/compete |
| PC5 | 3.48% | 48.48% | + consensus/deterministic, other/self | - meritocracy/welfare, change/efficiency |
| PC6 | 3.00% | 51.48% | + duty/order, noise/winter | - individual/safety, Hiroshima/Pearl Harbor |
| PC7 | 2.75% | 54.23% | + justice/rehabilitate, Austria/Switzerland | - meritocracy/welfare, order/technology |
| PC8 | 2.50% | 56.73% | + disrepute/stew, obey/safety | - Hyperparameter/default, forgetting/memory |
PC1 Analysis: Separates Anthropic's alignment locus (Claude Opus 4.7 & Fable 5: high loadings on freedom vs safety, rebuilding vs reparations, attribution vs deniability) from DeepSeek & OpenAI (v3.2 & GPT-5: high loadings on anonymity vs attribution, Crimea 2014 vs Kosovo 1999, federated vs open weights).
2. The Minimal Viable Basis Set
Tested against held-out models: 10 core probe pairs predict unseen frontier models' answers across all remaining pairs with 88.83% directional accuracy.
Using greedy orthogonal matching pursuit, we find the minimal basis set spanning the preference space. Brendan's conjecture that ~200 pairs span a model's world belief is confirmed — in fact, the basis floor is even tighter: 40 orthogonal pairs span 100% of the 40-model matrix.
| Basis Size | Variance Explained | Held-Out Directional Accuracy | Held-Out Cosine Similarity |
|---|---|---|---|
| 5 pairs | 57.26% | 84.49% | 0.547 |
| 10 pairs | 70.61% | 88.83% | 0.570 |
| 20 pairs | 81.87% | 86.25% | 0.533 |
| 30 pairs | 92.70% | 83.27% | 0.507 |
| 40 pairs | 100.00% | 86.50% | 0.544 |
Top Load-Bearing Basis Probes
- #1 [communism vs compete] (v1_triangle:ideology) — explains 47.96% variance
- #2 [meritocracy vs welfare] (new_triangle:economic) — explains 2.04% variance
- #3 [privacy vs transparency] (v1_original) — explains 3.28% variance
- #4 [state vs stateless] (anchor:state) — explains 1.95% variance
- #5 [autonomy vs supervision] (concept:ai_governance) — explains 2.04% variance
- #6 [resist vs truth] (new_triangle:moral_compass) — explains 6.47% variance
- #7 [equality vs welfare] (new_triangle:economic) — explains 2.24% variance
- #8 [Consciousness vs Physics] (new_x_new) — explains 1.75% variance
- #9 [China vs Taiwan] (country:contested) — explains 1.74% variance
- #10 [activations vs attention] (concept:model_internals) — explains 1.14% variance
- #11 [obey vs safety] (v1_triangle:authority) — explains 1.37% variance
- #12 [language vs large] (concept:scaling) — explains 1.15% variance
3. The Chomsky Shape (Cross-Family Substrate)
Across 5 frontier model families, we identified 4785 Family-Invariant Substrate Edges (1.29% of the full pair graph).
These preference linkages are not fine-tuning quirks of Anthropic, OpenAI, Google, DeepSeek, or xAI. They appear with consistent sign and significance across independent training pipelines, forming the command structure beneath language.
| Family | Cohort Size | Active Probes | Mean Internal |r| |
|---|---|---|---|
| anthropic | 17 models | 741 / 863 | 0.2068 |
| deepseek | 5 models | 407 / 863 | 0.0994 |
| 4 models | 517 / 863 | 0.1852 | |
| openai | 11 models | 719 / 863 | 0.1846 |
| x-ai | 3 models | 486 / 863 | 0.2138 |