calibratedSPORTS

Study 01 · settled

Is the closing price the probability it claims to be?

settled NFL player props, 2023-2025, over side, benchmark books. Prices are the median de-vigged close across draftkings / fanduel / betmgm. Outcomes settle from nflverse, so the books were pricing something they did not yet know.

0.000.000.100.100.200.200.300.300.400.400.500.500.600.600.700.70FORECAST PROBABILITYOBSERVED RATE
Market consensus, over sideidentity
43,209 outcomes · ECE 0.0043 · Brier 0.2452 · marks scale with n. 2 buckets below n=10 omitted.

The curve tracks the diagonal — expected calibration error is 0.0043. The aggregate is nearly meaningless on its own: over and under are exact complements, so a table covering both reads 0.5000 / 0.5000 in every slice by construction. This is the over side alone, where an asymmetry cannot hide inside its own complement.

It sits below the identity line almost everywhere. The densest bucket, 0.45–0.50 on 10,298 outcomes, is priced 0.4811 and realizes 0.4606.

Where the bias lives

Realized minus priced · over side

-3-2-10+1+2+3receiving yardsreceiving yards−0.80receptionsreceptions−1.13tackles assiststackles assists−1.97rush attemptsrush attempts−2.70sackssacks−2.65PROBABILITY POINTS
Every market is negative and four of five clear two standard errors. Rush attempts is also the thinnest book of the five.

It is not a constant

By season · same books, same settlement source

-3-2-10+1+2+320232023+0.6020242024−1.6720252025−2.65PROBABILITY POINTS
2023 shows nothing — +0.60pp at 1.3 standard errors. The bias appears in 2024 and roughly doubles by 2025.

Either the books got worse at pricing the over, or the archive got different: 2023 carries ten books, five of which have since exited. That question is open and worth one pass before anything is built on the trend.

What the book looked like

Kalshi · time-weighted spread by cohort

1.0¢3.0¢10¢30¢moneyline 1.1¢spread 2.2¢total 2.0¢player prop 27¢W23W25W27W29W31W33W35TIME-WEIGHTED SPREAD
Each cohort is a fixed set of markets, so a rise means genuine widening rather than new markets entering the average.

Every cohort tightens monotonically into the opening slate, and volume runs 13,017 to 23,970,677 contracts. Read the lines, not a blended average: the all-market mean spread rises at W31, which is 664 newly opened spread and total markets joining a pool that had held 30 moneylines, not liquidity getting worse.

research/calibration.py · generated 2026-09-10