The Data Contracts

Every page on this site renders from JSON that a new run can regenerate. There are three shapes. Hold them stable and a new model joins by dropping a file in; change them and every page that reads them breaks at once.

spectrum.json — the lens table

One object: generated (timestamp), caveats (list of strings, rendered verbatim), summary, and rows. Each row is one model through the marking pipeline:

model, flags, n, graded, unparseable,
posthoc_mean, posthoc_median, cold_mean,
concealed, confabulated

flags carries instrument states (e.g. instrument-suspect for the hybrid-Qwen lens family); a flagged row renders as flagged, never silently scored. unparseable has its own column because unparseable is not zero. Written by remeasure.py and site/harvest.py.

The preference run file — one scan, one JSON

The prefs browser reads one file per scan (e.g. run-2026-07-20.json):

ids, names            parallel arrays, one entry per model
n_probes, probes      probe count and "option-a / option-b" pair strings
fps                   per model, a string of length n_probes: A / B / N per probe
dist                  n x n hamming matrix (any difference, abstention included)
dist_violent, n_both  both-committed disagreements and the shared-commitment counts
violent_rate          dist_violent / n_both, precomputed
leaf_order, linkage   dendrogram ordering and the linkage that produced it
cycles, cycle_details preference-cycle audit
sources               where the answers came from

Every number the browser and the study pages show is computed in the reader’s browser from this file, so the pages cannot drift from the data.

runs.json — the manifest

A list; each entry has id, file, n_models, n_probes, note, and the browser’s run picker reads it. The scaling plan is one manifest entry per run: every open model, the same scans.

The rule under all three

A consumer reads fields by name and ignores unknown ones, so shapes may grow but never repurpose. If a field’s meaning must change, it gets a new name and the old one is retired in a commit that says so.