← Back to blog

How to Detect Capture Gaps in Historical Data: A Buyer's Guide

By the ZenHodl team — we run the trading bots this blog writes about, and the qualifying live-position record, including losses, is public with its admission rules at /results.

You bought an archive. The README says "complete." How do you know? You don't — until you derive coverage from the rows themselves. This guide walks through a four-step audit, using our own two-day Kalshi tryout as the worked example. Every number below comes from the data or from our published coverage notes; treat any archive without that kind of provenance as unverified.

Why READMEs fail

On 2026-09-06 our own shipped COVERAGE.md called itself "the complete known list." The data showed 233 hours of dark time — 31.8% of wall-clock since Aug 14 — that the list did not contain. It was maintained by hand, and hand-maintained lists drift. The fix is to derive every coverage number from the data at build time, never from a static sentence. We now do that, and this guide shows you how to check any seller, including us.

Step 1: Derive coverage from data, not claims

For each market family, count two things: (a) the share of traded markets that ever get a depth snapshot, and (b) the share of trade prints that have a book within ±60 seconds. If a README says "all markets," ask for the query that produced the count. Our Kalshi depth archive is a top-150-by-volume subset, not a census, and measured against the exchange's own trade prints it looks like this (football rows: Aug 24 – Sep 13; MLB moneyline and WNBA rows: Jul 10 – Sep 6):

Sport / market type Traded markets with any depth Prints with a book within ±60s
CFB moneyline 86% 90%
CFB spread 34% 81%
CFB total 36% 71%
NFL moneyline 96% 78%
NFL spread 58% 86%
NFL total 60% 82%
MLB moneyline 99.5% 96%
WNBA 100% 98%

The lesson: coverage varies by sport and market type. WNBA and MLB moneylines are close to complete; CFB spreads are covered for only about a third of traded markets. The print-weighted column always looks better than the market-count column, because volume-ranked capture follows volume. If a seller can't produce a table like this, treat their coverage claim as a hypothesis.

Step 2: Detect dark windows

Sort every poll timestamp and compute the gaps between neighbours. A healthy capture has a tight median and no gaps above your tolerance. In the two-day tryout there were 4,013 global polls, zero intervals over 120 seconds, and a median interval of 43.133 seconds. In the full tape, the same test is what exposed the 233-hour hole. One caveat matters more than any other: a missing observation never means an unchanged book. A quiet market and a dead recorder look identical in raw row counts; only the timestamp gaps tell them apart.

Step 3: Check cadence — measured vs claimed

A README's cadence tells you what the system was supposed to do; the timestamps tell you what it did. Our own documentation once claimed ~28 seconds; the measured global poll interval was ~31 seconds before Jul 12 and ~42–47 seconds after (43.133 seconds median in the tryout). Compute the median and p95 interval yourself for any archive, and record both.

Step 4: Check completeness per family

Count the ticker families actually present in the files, not the ones the seller lists. The tryout contains 40 depth families but only 3 trade-print families (MLB 596,392 prints; WNBA 150,097; NFL 401). Wide depth coverage with narrow print coverage means you can see order books but not who traded — fine for microstructure work, not enough for fill-price studies. Always ask: depth, trades, or both?

A spreadsheet you can reuse

Copy this layout for every archive you audit.

Column What to fill in Example (our Kalshi tape)
Sport / market type Family name CFB spread
Markets with any depth Share of traded markets that appear in the depth files 34%
Prints with a book within 60s Share of trade prints with a nearby snapshot 81%
Dark time (hours) Sum of gaps above your threshold 233
Dark time (% of wall-clock) Dark hours / total hours 31.8%
Cadence claimed From the README ~28s (old docs)
Cadence measured Median timestamp gap 43.133s
Notes Caveats top-150 subset; depth capped at 20 levels per side

Honest caveats

The worked example covers two UTC days (2026-07-18 and 2026-07-19) from a top-150-by-volume selection — not every sport or every hour. Depth is capped at 20 levels per side, so deeper book coverage can't be assessed. The game-state field on each print is recorded at retrieval time (median lag ~26 seconds, p99 ~53 minutes), so event-time analysis needs a separate score timeline. We disclose all of this because the alternative — another "complete known list" — costs exactly the trust a buyer is paying for. Derive, don't claim.

Try the audit yourself

Run these four steps on real capture data: the free samples show every schema, and the $9 two-day Kalshi tryout is the dataset used above. For the full coverage write-up, including every known capture gap, see how we built the Kalshi microstructure tape.

Related reading

Get ZenHodl Weekly

One weekly email with live results, one model insight, and product updates.

Tuesday mornings. No spam.

Want the data behind this post?

Historical sports prediction-market datasets with measured coverage, documented schemas, and disclosed gaps.

Join the community

Discuss strategies, share results, get help.

Join Discord