Forecast evidence, frozen at the decision
See which forecasters miss together.
Every forecast kept as issued, with its real clocks.
What is in the record · measured 2026-10-22
Headline numbers
- 13.5M
- forecasts
- 19,019
- markets with full price history
- 808
- forecasters in 82 families
- 12
- domains
- 431k
- weather training cases
- 38
- data defects published
ForecastBench panels: 13,523,509 rows, 6,311,674 with real answers
Of 19,030 verified temperature markets; 11 have no Polymarket history, 0 failed (v6.11)
756 AI models, 38 human groups, 6 baselines, 8 aggregates
Geopolitics & world events, Single-run weather, Ensemble weather, Prediction markets, Market traders, Crypto, Central-bank surveys, Economics, Energy, Health, Space weather, Hurricanes
205,746 single-run + 225,235 ensemble, 59 sites, 18 weeks
Two audits on 2026-10-20: all 11 critical fixed (invented or early clocks, answer-conditioned forecasts, stalled collectors); 12 lower-severity still open. Affected rows are excluded or flagged, never silently kept.
At a glance · 2026-10-22
What changed this week
Market prices
99.94%
19,019 of 19,030 markets priced. 11 have no history; none filled with zeros.
Forecasters with fixed IDs
- COVID-19 hub (legacy)129
- FluSight126
- FluSight (legacy)44
- COVID-19 hub29
- RSV hub20
- Weather ensembles5
- Crypto replay3
356 forecasters, 10 documented links. Without a link, independence is unknown.
Warning list
1,386 cases · 0 resolved so far
Every case is registered before the outcome, at most 50 alerts per run. Outcomes arrive as events resolve.
Releases
- v6.6 Variable catalogue, GFS, 2.96M station readings
- v6.7 Warning ledger, forecaster scores
- v6.8 Corrected market ranges
- v6.9 Price history begins (1,517)
- v6.10 ForecastBench dates, track records
- v6.11 All 19,030 markets priced
- v6.12 Market events with pre-deadline prices (18,466)
- v6.13 ECB labels, Numinous episodes, energy, river flow (current pin)
Each release is fingerprint-checked after download. Pin v6.13 (3c16e8fd) or its parent 8c5d3ad1; older revision IDs were removed on 23 Oct.
Latest frozen release
handoff-v6.13-20261024
commit 3c16e8fde1ac · +13 files · verified 2026-10-24All data we hold is cleared for sharing with Zeno and training. Only leakage, missing outcomes and Zeno's own test set are kept out. Every new file was re-downloaded with the repository token and matches its fingerprint.
The signature view
Which forecasters miss together?
Nine weather models, 7 marine sites, 21 frozen days. The live miss-together file covers 543 weather system pairs, each with its shared-outcome count.
- pair
- ecmwf aifs × ecmwf ifs
- shared outcomes
- 1,592
- miss rate A / B
- 16% / 16%
- missed together
- 7.5%
- if independent
- 2.4%
- ratio
- 3.12×
- SHARED OUTCOMES
- 1,592
- WORST PAIR
- 3.1× chance
- CONFIDENT AND WRONG
- 35,970 cases
More — coverage, method, paths, live record
Coverage
13 domains, one set of clocks
Bar length on a log scale. Muted bars are test-only: kept out of training because their publication times can't be proven.
- Geopolitics & world eventsForecastBench56,013answered question-horizons · Jul 2024 – Sep 2026
- Single-run weather7 model centres205,746cases · Jun – Oct 2026
- Ensemble weatherOpen-Meteo ensembles225,235cases · Sep – Oct 2026
- Prediction marketsManifold, Polymarket, Numinous170,096resolved questions with prices (97,788 Manifold, 71,195 Polymarket, 1,113 Numinous) · Aug – Oct 2026
- Market tradersPolymarket wallets457,874trades · 2026
- CryptoKalshi, Deribit, reference models255,807forecasts (mostly replays) + 51,459 venue quotes · Jul – Oct 2026
- Central-bank surveysECB SPF156,912probability distributions · 42 rounds
- EconomicsPhiladelphia Fed SPF1,979,271survey responses (2,250 first-release outcomes, RGDP/PGDP only) · 1968 – 2026
- EnergyNESO demand vs Elexon outturn137resolved half-hours (1,449 admitted forecasts at release) · Oct 2026
- HealthCDC FluSight, COVID-19 and RSV hubs567,189distinct dated outcomes (12,433 flu, 536,748 COVID, 18,008 RSV) · 2020 – 2023
- Space weatherKp observations103,623observations · Sep – Oct 2026
- HurricanesNOAA NHC ATCF forecast decks + best track152,707forecast rounds (one per method and cycle) + 1,711 best-track rounds · 2024–2025 seasons · collecting
- Market pricesPolymarket price history119,744pre-deadline price points on 19,019 temperature markets · 2026
Honest comparison
Not the biggest archive. The one built to show shared failure.
Single-domain archives
ECMWF, CDC forecast hubs
They have more: Decades of history in one field.
We add: Many domains joined, each forecaster tagged by family.
Market price feeds
Polymarket, Kalshi
They have more: Deeper live price history.
We add: Individual forecasters and outcomes dated by when they were actually known.
Forecasting benchmarks
ForecastBench, Metaculus, Good Judgment
They have more: Longer human track records in their own question sets.
We add: Who misses together, across benchmarks and the physical world, without future leakage.
Door A / Decision teams
Audit the forecasts you already buy
Analyst estimates, vendor feeds or internal models, assessed against observed outcomes and each other — like the marine audit above, on your data.
Door B / Zeno and ML teams
Train on decision-time episodes
Whole episodes as seen at the decision: every forecast available then, each forecaster's track record, the outcome kept apart. Explicit clocks, verified packs, no leakage.
Current record / Separate from the frozen example
Live evidence, as recorded
Evidence at the decision
The forecast and prior history are kept apart from what was known only afterward.
The frozen worked receipt is temporarily unreadable. Nothing is substituted for it.
Shared-case comparison
Signed errors on outcomes both models answered; not an explanation of shared failure.
Reading recorded comparisons…