Model Validation: Calibration & Clustered CIs

HOW WE'RE JUDGED

Calibration & CLV, not win rate

-2.5c

mean P&L per trade

2,142 resolved fills over 2,058 games; 48.7% win rate

0/9

sports pass the significance screen

Game-clustered bootstrap, Holm-corrected across sports — per-sport panels show the CIs

Pinnacle parity

Pre-registered benchmark

Pregame fair value statistically at parity with Pinnacle & Polymarket in the published benchmark scope; frozen artifacts and hashes on that page

This is a fair-value reference, not a guaranteed profit machine; the full live record, losses included, is on the results page. Calibration, realized P&L, execution quality, and explicitly scoped market-relative diagnostics answer different questions; no single one is the verdict.

Calibration here is measured on executed fills — an adversely-selected population (the market chose which of our orders to fill), so it is a harder test than population calibration and will read worse than training-time ECE. That is disclosed, not hidden.

This snapshot covers 9 sports (ATP, CS2, LOL, MLB, NBA, NHL, SOCCER, WNBA, WTA) and 2,142 resolved live trades — the headline and the panels below share one population. Sports with fewer than 10 trades (NCAAMB: 3, NCAAWB: 4, TENNIS: 6) are excluded from both, not just hidden. 0 of 9 sports pass the Holm-corrected significance screen. For production outcomes, see live results.

Companion: closing-line value

The unconditioned CLV aggregate (share of fills beating our captured close, mean CLV per trade) lives on the scorecard. The "78-point gap" whitepaper that used to be linked here was retracted on 2026-07-21 — the split was a measurement artifact, and the retraction shows the control that proved it.

CLV scorecard → Read the retraction →

What does each page show?

/validation — resolved live trades, statistically summarized: clustered CIs, calibration on fills, per-sport uncertainty. Numbers shown above.
/results — filtered public ledger. Confirmed positions under the published loader contract, including losses and nonbinary outcomes; the page reports measured execution-identifier and direct-transaction-hash coverage.
/clv-evidence — retraction. Why the CLV-gap statistic we used to publish was a measurement artifact, and the control that proved it.

Performance Summary

Trades	301
Win Rate	40.2%
Avg c/Trade	-5.9c
Total P&L	$-17.72
Source	resolved live fills

Uncertainty (game-clustered bootstrap, 259 games)

Bootstrap Mean	-5.9c
95% CI	[-12.4c, +0.6c]
99% CI	[-14.3c, +2.6c]
Bootstrap ASL P(mean≤0)	0.9625
Holm screen (0.05)	does not pass
Reading	CI straddles zero — that IS the result

Risk Metrics

-5.9±46c

Per-trade mean ± std

-2504.4c

Max Drawdown

0.76

Profit Factor

9

Best Streak

22

Worst Streak

Equity Curve (Cumulative c)

Calibration on fills Brier 0.2546 · log loss 0.7224 · ECE 19.7%

Performance by Edge Size

Edge	Trades	Win Rate	Avg c/Trade
5-10c	43	41.9%	-6.7c
10-15c	52	30.8%	-17.7c
15-20c	95	42.1%	-0.3c
20+c	61	41.0%	-1.0c

Example Trades

Matchup	Side	Score	Market	Model Fair	Edge	Result	P&L
Internazionali BNL d'Italia: Roberto Bautista Agut vs Francesco Maestrelli	Roberto Bautista Agut	0-0 (6-3)	80.0c	95.4c	+15.4c	WIN	+20.0c
Internazionali BNL d'Italia: Thiago Agustin Tirante vs Gianluca Cadenasso	Thiago Agustin Tirante	0-0 (2-1)	65.0c	80.4c	+15.4c	WIN	+35.0c
Internazionali BNL d'Italia: Botic van de Zandschulp vs Aleksandar Kovacevic	Aleksandar Kovacevic	0-1 (0-1)	63.0c	78.4c	+15.4c	LOSS	-63.0c
Internazionali BNL d'Italia: Mattia Bellucci vs Martin Landaluce	Mattia Bellucci	0-0 (3-2)	38.0c	54.4c	+15.4c	LOSS	-38.0c
Geneva Open: Raul Brancaccio vs Stan Wawrinka	Raul Brancaccio	1-1 (3-3)	29.0c	44.4c	+15.4c	LOSS	-29.0c

Performance by Month

Month	Trades	Win Rate	Avg c/Trade	Total P&L
2026-04	97	47.4%	-0.4c	-40c
2026-05	199	36.2%	-8.7c	-1736c

Performance Summary

Trades	370
Win Rate	48.9%
Avg c/Trade	-3.9c
Total P&L	$-14.35
Source	resolved live fills

Uncertainty (game-clustered bootstrap, 363 games)

Bootstrap Mean	-3.9c
95% CI	[-8.9c, +1.1c]
99% CI	[-10.3c, +2.8c]
Bootstrap ASL P(mean≤0)	0.9382
Holm screen (0.05)	does not pass
Reading	CI straddles zero — that IS the result

Risk Metrics

-3.9±48c

Per-trade mean ± std

-1817.9c

Max Drawdown

0.84

Profit Factor

8

Best Streak

6

Worst Streak

Equity Curve (Cumulative c)

Calibration on fills Brier 0.2727 · log loss 0.7553 · ECE 18.4%

Performance by Edge Size

Edge	Trades	Win Rate	Avg c/Trade
5-10c	46	47.8%	-12.9c
10-15c	56	51.8%	-4.2c
15-20c	58	43.1%	-8.9c
20+c	121	42.1%	+0.8c

Example Trades

Matchup	Side	Score	Market	Model Fair	Edge	Result	P&L
Counter-Strike: Turma do Pagode vs Bounty Hunters Esports (BO3) - CCT South Amer	Bounty Hunters	0-1	70.0c	86.6c	+16.6c	LOSS	-70.0c
Counter-Strike: Keyd vs paiN Academy (BO3) - CCT South America Series #1 Group S	paiN Academy	1-0	8.0c	24.7c	+16.7c	LOSS	-8.0c
Counter-Strike: Ground Zero vs Rooster (BO3) - ESL Challenger League Oceania Cup	Ground Zero	0-0	39.0c	55.8c	+16.8c	LOSS	-39.0c
Counter-Strike: Isurus vs Fake do Biru (BO3) - CCT South America Series #1 Playo	Isurus	0-1	42.0c	59.0c	+17.0c	LOSS	-42.0c
Counter-Strike: Imperial Academy vs INFURITY Gaming (BO3) - United21 Group B	INFURITY Gaming	0-0	57.0c	72.1c	+17.1c	WIN	+43.0c

Performance by Month

Month	Trades	Win Rate	Avg c/Trade	Total P&L
2026-04	138	44.2%	-6.1c	-846c
2026-05	177	50.8%	-1.5c	-265c
2026-06	46	52.2%	-8.6c	-396c

Performance Summary

Trades	172
Win Rate	46.5%
Avg c/Trade	+0.4c
Total P&L	$0.68
Source	resolved live fills

Uncertainty (game-clustered bootstrap, 159 games)

Bootstrap Mean	+0.4c
95% CI	[-6.9c, +7.6c]
99% CI	[-9.0c, +10.1c]
Bootstrap ASL P(mean≤0)	0.4604
Holm screen (0.05)	does not pass
Reading	CI straddles zero — that IS the result

Risk Metrics

+0.4±46c

Per-trade mean ± std

-601.2c

Max Drawdown

1.02

Profit Factor

5

Best Streak

5

Worst Streak

Equity Curve (Cumulative c)

Calibration on fills Brier 0.2513 · log loss 0.6979 · ECE 14.6%

Performance by Edge Size

Edge	Trades	Win Rate	Avg c/Trade
5-10c	52	55.8%	+4.5c
10-15c	38	47.4%	-0.2c
15-20c	23	56.5%	+11.5c
20+c	47	27.7%	-12.2c

Example Trades

Matchup	Side	Score	Market	Model Fair	Edge	Result	P&L
LoL: Barcząca Esports vs BOMBA Team (BO3) - Rift Legends Regular Season	Barcząca Esports	1-1	29.0c	40.8c	+11.8c	WIN	+71.0c
LoL: Movistar KOI vs G2 Esports (BO3) - LEC Regular Season	G2 Esports	0-1	49.0c	58.8c	+11.8c	WIN	+51.0c
LoL: Forsaken vs Lodis (BO3) - Rift Legends Regular Season	Forsaken	0-1	64.0c	75.9c	+11.9c	WIN	+36.0c
LoL: Citadel Gaming vs Blue Otter (BO3) - North American Challengers League Regu	Blue Otter	0-0	38.0c	50.0c	+12.0c	LOSS	-38.0c
LoL: TLN Pirates vs Vitality.Bee (BO3) - LFL Playoffs	TLN Pirates	0-1	25.0c	37.3c	+12.3c	LOSS	-25.0c

Performance by Month

Month	Trades	Win Rate	Avg c/Trade	Total P&L
2026-04	79	44.3%	-4.0c	-316c
2026-05	78	44.9%	+1.2c	+91c

Performance Summary

Trades	559
Win Rate	53.8%
Avg c/Trade	-2.0c
Total P&L	$-11.05
Source	resolved live fills

Uncertainty (game-clustered bootstrap, 554 games)

Bootstrap Mean	-2.0c
95% CI	[-5.8c, +1.9c]
99% CI	[-7.0c, +3.1c]
Bootstrap ASL P(mean≤0)	0.8439
Holm screen (0.05)	does not pass
Reading	CI straddles zero — that IS the result

Risk Metrics

-2.0±46c

Per-trade mean ± std

-2117.1c

Max Drawdown

0.91

Profit Factor

9

Best Streak

8

Worst Streak

Equity Curve (Cumulative c)

Calibration on fills Brier 0.2171 · log loss 0.6254 · ECE 7.4%

Performance by Edge Size

Edge	Trades	Win Rate	Avg c/Trade
5-10c	121	66.9%	+0.6c
10-15c	102	57.8%	+0.3c
15-20c	35	48.6%	-4.6c
20+c	28	53.6%	+7.1c

Example Trades

Matchup	Side	Score	Market	Model Fair	Edge	Result	P&L
Cincinnati Reds vs. Chicago Cubs	cincinnatireds	0-3	65.0c	77.9c	+12.9c	LOSS	-65.0c
Minnesota Twins vs. Cleveland Guardians	minnesotatwins	0-1	40.0c	52.9c	+12.9c	WIN	+60.0c
Detroit Tigers vs. Kansas City Royals	detroittigers	2-3	45.0c	57.9c	+12.9c	WIN	+55.0c
Milwaukee Brewers vs. St. Louis Cardinals	stlouiscardinals	1-0	47.0c	60.0c	+13.0c	WIN	+53.0c
Houston Astros vs. Cincinnati Reds	houstonastros	0-2	57.0c	67.0c	+13.0c	WIN	+43.0c

Performance by Month

Month	Trades	Win Rate	Avg c/Trade	Total P&L
2026-04	130	61.5%	+0.5c	+68c
2026-05	188	48.9%	-2.1c	-395c
2026-06	211	51.7%	-4.6c	-973c
2026-07	26	61.5%	+0.2c	+5c

Performance Summary

Trades	43
Win Rate	39.5%
Avg c/Trade	-18.9c
Total P&L	$-8.11
Source	resolved live fills

Uncertainty (game-clustered bootstrap, 41 games)

Bootstrap Mean	-18.9c
95% CI	[-32.5c, -4.7c]
99% CI	[-36.4c, +0.1c]
Bootstrap ASL P(mean≤0)	0.9949
Holm screen (0.05)	does not pass
Reading	CI excludes zero (negative)

Risk Metrics

-18.9±46c

Per-trade mean ± std

-856.0c

Max Drawdown

0.41

Profit Factor

5

Best Streak

9

Worst Streak

Equity Curve (Cumulative c)

Calibration on fills Brier 0.3068 · log loss 0.8584 · ECE 31.4%

Performance by Edge Size

Edge	Trades	Win Rate	Avg c/Trade
5-10c	14	71.4%	+0.8c
10-15c	6	16.7%	-38.0c
15-20c	12	25.0%	-27.6c
20+c	8	37.5%	-14.8c

Example Trades

Matchup	Side	Score	Market	Model Fair	Edge	Result	P&L
Hornets vs. Timberwolves	timberwolves	40-36	43.0c	59.4c	+16.4c	LOSS	-43.0c
Nuggets vs. Spurs	nuggets	91-100	69.0c	85.2c	+17.2c	WIN	+31.0c
Spurs vs. Trail Blazers	trailblazers	58-43	68.0c	86.3c	+17.3c	LOSS	-68.0c
Grizzlies vs. Jazz	jazz	33-30	67.0c	81.4c	+17.4c	WIN	+33.0c
76ers vs. Knicks	76ers	66-69	34.0c	51.6c	+17.6c	LOSS	-34.0c

Performance by Month

Month	Trades	Win Rate	Avg c/Trade	Total P&L
2026-04	32	50.0%	-15.7c	-502c
2026-05	11	9.1%	-28.1c	-309c

Model Architecture

Architecture	Split-Phase XGBoost (early-game + clutch-time models)
Features	14 engineered features
Calibration	Isotonic regression
Training Data	5,285 games

Features: score_diff, time_fraction, home_elo, away_elo, elo_diff, home_court, pregame_wp, score_diff_x_tf, score_diff_sq, total_score, score_diff_x_elo, pace_diff, ortg_diff, drtg_diff

Performance Summary

Trades	120
Win Rate	60.8%
Avg c/Trade	+2.6c
Total P&L	$3.15
Source	resolved live fills

Uncertainty (game-clustered bootstrap, 113 games)

Bootstrap Mean	+2.6c
95% CI	[-6.9c, +11.6c]
99% CI	[-9.8c, +14.3c]
Bootstrap ASL P(mean≤0)	0.2865
Holm screen (0.05)	does not pass
Reading	CI straddles zero — that IS the result

Risk Metrics

+2.6±47c

Per-trade mean ± std

-392.0c

Max Drawdown

1.13

Profit Factor

9

Best Streak

4

Worst Streak

Equity Curve (Cumulative c)

Calibration on fills Brier 0.2372 · log loss 0.6796 · ECE 11.0%

Performance by Edge Size

Edge	Trades	Win Rate	Avg c/Trade
5-10c	32	50.0%	-7.4c
10-15c	33	63.6%	+2.5c
15-20c	20	85.0%	+22.4c
20+c	20	50.0%	-3.1c

Example Trades

Matchup	Side	Score	Market	Model Fair	Edge	Result	P&L
Devils vs. Red Wings	redwings	1-0	69.0c	81.1c	+12.1c	LOSS	-69.0c
Avalanche vs. Wild	avalanche	1-0	41.0c	53.1c	+12.1c	LOSS	-41.0c
Rangers vs. Lightning	rangers	0-1	57.0c	67.2c	+12.2c	WIN	+43.0c
Flames vs. Kraken	flames	0-1	55.0c	64.3c	+12.3c	LOSS	-55.0c
Hurricanes vs. Senators	hurricanes	0-1	65.0c	77.5c	+12.5c	LOSS	-65.0c

Performance by Month

Month	Trades	Win Rate	Avg c/Trade	Total P&L
2026-04	85	63.5%	+2.8c	+240c
2026-05	28	60.7%	+8.3c	+233c

Model Architecture

Architecture	XGBoost + Isotonic calibration
Features	12 engineered features
Calibration	Isotonic regression
Training Data	4,225 games

Features: score_diff, time_fraction, home_elo, away_elo, elo_diff, home_ice, pregame_wp, score_diff_x_tf, score_diff_sq, pace_diff, ortg_diff, drtg_diff

Performance Summary

Trades	85
Win Rate	41.2%
Avg c/Trade	-3.2c
Total P&L	$-2.69
Source	resolved live fills

Uncertainty (game-clustered bootstrap, 85 games)

Bootstrap Mean	-3.2c
95% CI	[-13.3c, +6.5c]
99% CI	[-16.5c, +9.7c]
Bootstrap ASL P(mean≤0)	0.7424
Holm screen (0.05)	does not pass
Reading	CI straddles zero — that IS the result

Risk Metrics

-3.2±47c

Per-trade mean ± std

-676.9c

Max Drawdown

0.87

Profit Factor

6

Best Streak

7

Worst Streak

Equity Curve (Cumulative c)

Calibration on fills Brier 0.2531 · log loss 0.7048 · ECE 17.4%

Performance by Edge Size

Edge	Trades	Win Rate	Avg c/Trade
5-10c	11	45.5%	-5.3c
10-15c	57	43.9%	-0.6c
15-20c	11	18.2%	-15.6c
20+c	5	40.0%	-9.0c

Example Trades

Matchup	Side	Score	Market	Model Fair	Edge	Result	P&L
Girona FC vs. Real Sociedad de Fútbol	draw	1-1	43.0c	56.0c	+13.0c	WIN	+57.0c
Sevilla FC vs. Real Madrid CF	away	0-0	48.0c	61.1c	+13.1c	WIN	+52.0c
FC Lorient vs. Le Havre AC	home	0-0	41.0c	54.2c	+13.2c	LOSS	-41.0c
RC Strasbourg Alsace vs. AS Monaco FC	away	0-1	51.0c	65.2c	+13.2c	LOSS	-51.0c
Portland Timbers vs. San Jose Earthquakes	away	0-2	57.0c	70.2c	+13.2c	WIN	+43.0c

Performance by Month

Month	Trades	Win Rate	Avg c/Trade	Total P&L
2026-04	34	32.4%	-11.0c	-373c
2026-05	51	47.1%	+2.0c	+104c

Performance Summary

Trades	119
Win Rate	38.7%
Avg c/Trade	+0.6c
Total P&L	$0.77
Source	resolved live fills

Uncertainty (game-clustered bootstrap, 119 games)

Bootstrap Mean	+0.7c
95% CI	[-7.8c, +9.1c]
99% CI	[-10.1c, +11.5c]
Bootstrap ASL P(mean≤0)	0.4433
Holm screen (0.05)	does not pass
Reading	CI straddles zero — that IS the result

Risk Metrics

+0.7±47c

Per-trade mean ± std

-419.0c

Max Drawdown

1.03

Profit Factor

6

Best Streak

6

Worst Streak

Equity Curve (Cumulative c)

Calibration on fills Brier 0.2289 · log loss 0.6504 · ECE 7.5%

Performance by Edge Size

Edge	Trades	Win Rate	Avg c/Trade
5-10c	84	40.5%	+1.6c
10-15c	19	31.6%	-0.3c
15-20c	2	0.0%	-35.5c
20+c	6	33.3%	-5.7c

Example Trades

Matchup	Side	Score	Market	Model Fair	Edge	Result	P&L
Seattle Storm vs. Toronto Tempo	seattlestorm	0-2	35.0c	44.6c	+10.6c	LOSS	-35.0c
PortlandFire vs. Chicago Sky	portlandfire	0-3	42.0c	51.6c	+10.6c	LOSS	-42.0c
Minnesota Lynx vs. Golden State Valkyries	minnesotalynx	2-0	56.0c	67.3c	+11.3c	WIN	+44.0c
Dallas Wings vs. Connecticut Sun	connecticutsun	0-3	25.0c	36.4c	+11.4c	LOSS	-25.0c
Phoenix Mercury vs. Golden State Valkyries	phoenixmercury	2-0	26.0c	37.8c	+11.8c	LOSS	-26.0c

Performance by Month

Month	Trades	Win Rate	Avg c/Trade	Total P&L
2026-05	23	52.2%	+11.7c	+268c
2026-06	57	36.8%	-1.5c	-85c
2026-07	39	33.3%	-2.7c	-107c

Performance Summary

Trades	373
Win Rate	50.7%
Avg c/Trade	-1.0c
Total P&L	$-3.86
Source	resolved live fills

Uncertainty (game-clustered bootstrap, 365 games)

Bootstrap Mean	-1.0c
95% CI	[-5.8c, +3.7c]
99% CI	[-7.6c, +5.2c]
Bootstrap ASL P(mean≤0)	0.6598
Holm screen (0.05)	does not pass
Reading	CI straddles zero — that IS the result

Risk Metrics

-1.0±46c

Per-trade mean ± std

-1329.5c

Max Drawdown

0.95

Profit Factor

8

Best Streak

8

Worst Streak

Equity Curve (Cumulative c)

Calibration on fills Brier 0.2341 · log loss 0.6667 · ECE 12.5%

Performance by Edge Size

Edge	Trades	Win Rate	Avg c/Trade
5-10c	72	56.9%	+1.2c
10-15c	54	53.7%	+3.6c
15-20c	54	50.0%	-0.5c
20+c	75	37.3%	-3.9c

Example Trades

Matchup	Side	Score	Market	Model Fair	Edge	Result	P&L
Internazionali BNL d'Italia, Qualification: Anastasia Pavlyuchenkova vs Veronika	Anastasia Pavlyuchenkova	0-1 (0-0)	25.0c	39.6c	+14.6c	LOSS	-25.0c
Grass Court Championships, Qualification: Lulu Sun vs Anhelina Kalinina	Lulu Sun	0-0 (1-1)	41.0c	52.7c	+14.7c	LOSS	-41.0c
Jiujiang: Viktoria Morvayova vs Yexin Ma	Viktoria Morvayova	0-0 (1-1)	41.0c	55.8c	+14.8c	WIN	+59.0c
Roland Garros, Qualification WTA: Leyre Romero Gormaz vs Sloane Stephens	Leyre Romero Gormaz	0-0 (2-1)	49.0c	63.9c	+14.9c	LOSS	-49.0c
Internationaux de Strasbourg: Daria Kasatkina vs Liudmila Samsonova	Liudmila Samsonova	0-0 (1-1)	47.0c	63.0c	+15.0c	LOSS	-47.0c

Performance by Month

Month	Trades	Win Rate	Avg c/Trade	Total P&L
2026-04	42	40.5%	-9.7c	-408c
2026-05	247	51.4%	-0.8c	-197c
2026-06	84	53.6%	+2.6c	+219c

Benchmark Comparisons

How our model performs vs naive strategies. A model that can't beat simple baselines isn't worth using.

ATP

Strategy	Win Rate	c/share
Our Model	40.2%	-5.9c
Random (50/50)	50.0%	-2.0c
Market-Efficient	46.2%	+0.0c

CS2

Strategy	Win Rate	c/share
Our Model	48.9%	-3.9c
Random (50/50)	50.0%	-2.0c
Market-Efficient	53.3%	+0.0c

LOL

Strategy	Win Rate	c/share
Our Model	46.5%	+0.4c
Random (50/50)	50.0%	-2.0c
Market-Efficient	45.5%	-0.0c

MLB

Strategy	Win Rate	c/share
Our Model	53.8%	-2.0c
Random (50/50)	50.0%	-2.0c
Market-Efficient	55.8%	+0.0c

NBA

Strategy	Win Rate	c/share
Our Model	39.5%	-18.9c
Random (50/50)	50.0%	-2.0c
Market-Efficient	58.4%	+0.0c

NCAAMB — excluded: fewer than 10 trades (n=3).

NCAAWB — excluded: fewer than 10 trades (n=4).

NHL

Strategy	Win Rate	c/share
Our Model	60.8%	+2.6c
Random (50/50)	50.0%	-2.0c
Market-Efficient	57.8%	+0.0c

SOCCER

Strategy	Win Rate	c/share
Our Model	41.2%	-3.2c
Random (50/50)	50.0%	-2.0c
Market-Efficient	44.0%	+0.0c

TENNIS — excluded: fewer than 10 trades (n=6).

WNBA

Strategy	Win Rate	c/share
Our Model	38.7%	+0.6c
Random (50/50)	50.0%	-2.0c
Market-Efficient	38.0%	-0.0c

WTA

Strategy	Win Rate	c/share
Our Model	50.7%	-1.0c
Random (50/50)	50.0%	-2.0c
Market-Efficient	51.7%	-0.0c

Academic Foundation

Our approach is grounded in peer-reviewed research on sports prediction markets and probabilistic forecasting.

Beating the bookies with their own numbers - and how the online sports betting market is rigged

Kaunitz, Zhong, Kreiner (2017)

CLV validation - demonstrates that a positive closing line value strategy yields positive long-term returns

Verification of forecasts expressed in terms of probability

Brier, Glenn W. (1950)

Foundation for calibration analysis - Brier score measures probabilistic prediction accuracy

Using random forests to estimate win probability before each play of an NFL game

Lock, Dennis; Nettleton, Dan (2014)

In-game WP modeling methodology - random forests on game state features for real-time prediction

Why are gambling markets organised so differently from financial markets?

Levitt, Steven D. (2004)

Market efficiency analysis - sports markets exhibit inefficiencies exploitable by informed bettors

Optimal betting odds against insider traders

Shin, Hyun Song (1991)

Theoretical foundation for bookmaker pricing models and adverse selection in betting markets

A Brownian motion model for the progress of sports scores

Stern, Hal (1994)

Score-diff as Brownian motion - theoretical underpinning for WP models based on score differential and time

Methodology & Anti-Overfitting Safeguards

Population — Resolved LIVE trades from the canonical public ledger: mode=live, confirmed fills, binary outcomes, no backfills, no shadow/simulated rows. Fills span the model versions deployed over the window, so this validates the system as actually traded — not a single frozen pkl.

Fill selection — Executed fills are an adversely-selected sample (the market chose which of our orders to fill), so calibration measured here is a strictly harder test than population calibration and will read worse than training-time ECE. We show it anyway.

Reproducibility — The artifact records sha256 hashes of the input ledger and the generator source (trades.jsonl: aa276e59c217fa97… · generate_validation_data.py: 24940c6307fe6270…). Missing model-probability fields abort generation rather than defaulting.

Bootstrap confidence intervals — 10,000 resamples, drawn by game cluster rather than individual trade (same-game entries are correlated; per-trade resampling understates uncertainty). The ASL is the fraction of resampled means ≤ 0 — an achieved significance level by CI duality, not a formal null-hypothesis p-value, and we label it accordingly. Per-sport ASLs are additionally screened with Holm-Bonferroni across sports at 0.05.

Calibration — Predictions are bucketed into 5%-wide bins (min 5 trades each). A well-calibrated model's dots land on the diagonal; points below the line indicate overconfidence.

Per-trade dispersion — We report the raw per-trade mean and standard deviation in cents. We no longer publish an annualized "Sharpe" here: per-trade cents are not daily portfolio returns, so sqrt(252) scaling was unit-invalid and we removed it (2026-07-21).

Profit factor — Gross wins / gross losses. Above 1.25 = profitable. Above 1.5 = strong. Above 2.0 = excellent.

Fee assumptions — Results are shown net of the execution-cost assumptions used when this validation export was generated. Live exchange fees and market microstructure can change over time.

Pregame filter — For NBA and NHL, the deployed strategy requires the pregame market price to agree with the model's bet side at ≥55c. The rule excludes model/market disagreements; whether it reduces adverse selection must be measured rather than inferred from the rule itself.