ZenHodl vs Polymarket Consensus
A pre-committed, on-chain-anchored head-to-head calibration benchmark covering every NBA Conference Semifinal, Conference Final, and NBA Finals game from May 5 through June 25, 2026. Our model versus the live betting market. Same games, same instant, same metrics. Updated after every game.
Pre-commitment proof
Reproduce: curl -s /benchmarks/nba-playoffs-2026/manifest.json | sha256sum
Live leaderboard
Read the full manifest (the rules) ↓
{
"metrics": {
"auxiliary": [
"Brier score",
"Log loss",
"Accuracy"
],
"confidence_interval": "95% bootstrap CI on ECE with 1000 resamples, published alongside point estimate",
"ece_formula": "Sum over bins of |bin_avg_pred - bin_avg_outcome| weighted by bin sample fraction",
"headline": "Expected Calibration Error (ECE), 10 equal-width bins"
},
"model_versioning": {
"policy": "ZenHodl model weights as deployed at T-60 of each game are what counts. Each prediction row in raw.jsonl includes the model version ID so post-hoc retrains do not invalidate prior predictions.",
"retrains_during_window": "Permitted. Disclosed in the per-game row\u0027s model_version field."
},
"publication": {
"live_url": "https://zenhodl.net/benchmarks/nba-playoffs-2026",
"manifest_file": "https://zenhodl.net/benchmarks/nba-playoffs-2026/manifest.json",
"raw_data_jsonl": "https://zenhodl.net/benchmarks/nba-playoffs-2026/raw.jsonl",
"we_publish_when_we_lose": true
},
"published_at": "2026-05-04T20:00:00Z",
"rule_changes": "Once this manifest\u0027s SHA-256 hash is broadcast on Polygon, the rules above are frozen. If ZenHodl edits this file at any later point, the on-chain hash will not match the served file. Anyone can verify by hashing the served manifest.json and comparing to the on-chain transaction data field.",
"scope": {
"first_eligible_game_after": "2026-05-05T00:00:00Z",
"last_eligible_game_before": "2026-06-25T00:00:00Z",
"sport": "NBA",
"window": "2026 Conference Semifinals through Finals (inclusive)"
},
"snapshot": {
"matching": "Each NBA game matched to its Polymarket market by team names + game date. Matching script published in this repo so the join is auditable.",
"polymarket_source": "Polymarket NBA game-winner market YES-side mid price (best bid + best ask) / 2, fetched from clob.polymarket.com",
"tie_handling": "If either source is unavailable at T-60, the game is excluded from BOTH model\u0027s metrics. Recorded with status=\u0027polymarket_unavailable\u0027 or \u0027zenhodl_unavailable\u0027 in the public raw.jsonl.",
"timing": "Both predictions captured no later than T-60 minutes before official tip-off",
"zenhodl_source": "ZenHodl pregame win probability via internal SignalEngine.get_pregame_predictions(\u0027NBA\u0027)"
},
"title": "ZenHodl vs Polymarket Consensus \u2014 NBA Playoffs 2026 Calibration Benchmark",
"version": "1.1",
"why_polymarket": "Polymarket\u0027s market price represents the consensus probability of every smart-money trader actively wagering real capital. Beating it on calibration is the canonical hedge-fund-grade benchmark for any sports forecasting model, equivalent to closing-line value (CLV) in traditional sports analytics."
}
Why benchmark against the market?
Polymarket's mid-price is the consensus probability of every smart-money trader actively wagering real capital on the outcome. It's the toughest opponent we could pick — not because it's a smarter model, but because it aggregates every smart model.
A solo operator beating the market on calibration would be the canonical hedge-fund-grade demonstration that the underlying model has edge. We may not. The point is to find out, transparently, in front of you.
If we lose, the loss appears here, in the same row, with the same Brier score. The manifest commits us to publishing that outcome — there is no edit path that changes it without invalidating the on-chain hash.
7-day free trial. Same NBA pregame WP feed. Same calibration we're being judged on right here.