orange tomato
Page 1/10

WHAT DID THE SCORE MEAN?

A companion notebook to the deep dive — readings, proofs, filings, and what they establish
THE CITED NUMBER
→
instrument
date
owner
scope
  • A score, a proof, a figure. Each arrives with the thing that made it — and each announcement quotes the figure while the instrument stays offstage.
  • The question: when an announcement cites a number or a proof, what does it actually establish?
55 → 46
One week. No model update. The test moved underneath the score.
orange tomato
Page 2/10

1. THE NUMBER AND ITS INSTRUMENT

  • Every cited number carries four things: the instrument that defined it, the date that bounds it, the owner who ran it, and the scope that limits what it settles.
  • The headline keeps the first and drops the other three. That is the whole subject of this notebook.
reading
what the test said
guarantee
what the proof covers
commitment
which direction it binds
claim
documented · asserted · inspectable
Ask for the instrument before citing the number: method and version, date read, who ran it, what it excludes.
orange tomato
Page 3/10

2. READINGS — THE SCORE UNDERNEATH THE SCORE

  • Kimi K3.1 Pro: 55.09 → 46.00 between two dated captures (Oct 26 → Nov 1). Eleven shared evaluation scores identical to the last digit. The arithmetic was rebuilt (index v5); the model did not move.
  • Field-wide: 79 models on both captures, median composite shift −10.12. 78 of 79 kept every shared measurement frozen. The exception was a preview still being evaluated.
  • Ranks need denominators: 4th of 215 → 21st of 58 → 9th in press → 1st of 26. Every one real; none the same field.
62 → 3
One model. Ten harnesses. The same 113 tasks. The system around the model co-determines the outcome.
Wilson 95% intervals overlap mid-field; a two-runs-per-task study measured ~4 points of rerun noise. The comparison is vendor-run and vendor-won; no independent replication as of Nov 8.
orange tomato
Page 4/10

3. GUARANTEES — WHAT A PROOF COVERS

  • FlashProve pins one exact weight set and proves, checkably on a CPU, that the committed weights, run on the prompt, produce exactly these tokens. Four properties: completeness, soundness, zero-knowledge, succinctness.
  • The narrowing: a yes does not say whose weights, which real model, or whether the answer is correct. Strict exact-match vs a looser policy that, loose, bars little. The proof is about the integer model; drift from the floating-point network is measured, not proved.
  • No independent audit; AI systems wrote nearly all the code and proofs. Frozen at 0.16.0 since Oct 30 — no reproduction, adoption, or incident since.
0.980 → 0.898
Top-bin confidence stated vs delivered. Every bin over-optimistic, on both index versions. An estimate, not a measured frequency.
The guide's only reliability advice: set your own thresholds on labeled examples. No calibration curve exists for these values. Probe (3,600 problems, 99% → 68%) bounds apply.
orange tomato
Page 5/10

4. THE FILING — WHAT $417 BILLION COMMITS

  • After Dec 31, 2025: ~$417B aggregate future minimum payment obligations (leases + cloud purchases, terms <1 yr to 20 yrs). The filing's own table — with footnotes.
  • Floors vs caps: Amazon ≥$110.0B regardless of use (take-or-pay, non-cancelable) vs SpaceXAI up to $120.5B (90-day outs after early 2027). Microsoft ≥$44.9B, Google ~$121.3B alongside.
  • Outside the figure: $125–150B of unfinished 2028–31 terms, and an AMD deal past $20B with no fixed dollar amount. About $69.8B falls due within twelve months of Sep 2026.
$ and $
The cover price range is still blank. Reported targets (week of Nov 16) are reported, not filed.
The filing warns of excess capacity underutilized and cash outflows without immediate revenue — and disclaims its own growth rates as a guide to the future.
orange tomato
Page 6/10

5. THE PLAYTEST CASE — DOCUMENTED, ASSERTED, INSPECTABLE

  • Documented: Oct 19 — the ReVamp playtest moves from Steam to a smaller Discord round; sharing a build with tens of thousands felt like too big a risk. The public round waits for lower risk. Outcome unconfirmed as of Nov 8.
  • Asserted: that a public build could be cloned into a competing game before launch. The feared event stays untested — the broad distribution never happened.
  • Inspectable: the clones on record started from a clip, not a build — seven-plus lookalikes within days. One creator's account is self-report; one code-copy dispute is a reporter's inference.
MicroDuck's mirror image: $399 and open policies you can inspect — against a single-sourced preorder count, a signed-not-closed acquisition, and two delivery dates on the vendor's own pages.
A second studio cancelled a demo the same week (two reasons, neither demonstrated). Mechanics have always traveled fast; speed is the new variable, not copying.
orange tomato
Page 7/10

6. THE PATTERN — READINGS, GUARANTEES, COMMITMENTS, CLAIMS

  • Readings establish what an instrument said, under its method of the day. When the method changes, the citation outlives its referent — and both readings stay honest.
  • Guarantees establish exactly their stated statement. Provenance certifies the label; an estimate reports a belief. Neither certifies the answer.
  • Commitments establish obligations in named directions. Floors bind down, caps bind up, excluded estimates bind nothing.
  • Claims exist in three states: documented, asserted, inspectable. Only the third is portable.
Where we land: the number is rarely the problem. The missing instrument is.
orange tomato
Page 8/10

7. WHAT YOU CAN INSPECT — THE READER'S CHECKLIST

  • Which instrument, which version? Name the method with the score; give ranks their denominator.
  • What does the guarantee exclude? Whose weights, which model, exact-match or limits? What evidence ties stated confidence to accuracy?
  • What does the figure count? API spend or total cost? A floor or a cap? In the table, or the estimate the table excludes?
  • Where does the decision live? Thresholds need owners, review queues, and drift checks. The endpoint supplies none of them.
  • When was it read? Two dated readings of a moving page are normal. Report both; never merge them.
  • Which evidential state? Documented, asserted, or inspectable — price them differently.
If any answer is "the announcement didn't say", don't treat the number as self-explanatory.
orange tomato
Page 9/10

8. WHAT CHANGES THE JUDGMENT

  • Index: v5.0.1 regraded one component for all models (Nov 3) — the 55→46 finding is scoped to Oct 26–Nov 1. Decision Index v2 moved its own scores; v1 arithmetic is now historical.
  • Proofs & confidence: FlashProve unchanged since Oct 30. No vendor-published calibration data for the Decisions API values.
  • Filing & IPO: price range still blank; reported target week of Nov 16. A filed range replaces every reported figure.
  • Cases: ReVamp outcome unconfirmed; MicroDuck pages disagree today. A demonstrated playtest clone — or the quiet end of the fear — moves the judgment.
The unglamorous list that matters most: an independent harness rerun · a documented Elo conversion · vendor calibration curves · a filed price range.
orange tomato
Page 10/10

9. SOURCES & METHOD

  • Method: desk research and source audit only — no models run, no proofs checked, no hardware bought, no games cloned. Reading pages is not running anything.
  • Dated readings (Nov 8, 2026): AA methodology/version history/changelog/model pages + two Wayback captures; AgentField docs + CodeAF repo (API); Anthropic provable-inference page + FlashProve repo (raw + API); OpenAI Decisions guide (browser) + announcement (extract + Wayback); both SEC S-1 filings (EDGAR) + submissions JSON; Steam/Bluesky public APIs; TechCrunch, PC Gamer, Kotaku pages.
  • Carried material from seven earlier pieces, each re-cited to its primary source; none of the retired articles is cited in the deep dive.
  • Attributed, not verified: IPO targets (WSJ/Reuters/IFR/CNBC), DOJ inquiry (Reuters), preorder count (single stakeholder tweet via TechCrunch), probe results (vendor with interest). Revenue/growth figures are the issuer's own arithmetic.
  • Open at compilation: Elo→index conversion; changelog anomaly cause; ReVamp outcome; MicroDuck delivery reconciliation; filing exhibits + Note 17 detail; FlashProve technical report.
Companion to "The Test Changed. What Did the Score Mean?" — the article carries the full source list with links at the point of each claim.