Last Updated: 2026-10-26
A poll showing 52% support and a contract priced at 70 cents do not necessarily disagree. The first estimates a share of respondents or voters; the second is a price for a specified outcome. Neither number, by itself, tells you which source forecasts better.
This guide replaces the earlier unsupported claim that markets generally beat polls. Its examples are synthetic. It does not claim a personal forecasting record, recommend a trade or rank current polling providers.
Define what each number measures
Scroll sideways to see every column. Keyboard: focus the table region and use the arrow keys.
| Observation | What it can describe | What it does not establish |
|---|---|---|
| 52% in a poll | Estimated support in the stated population, during fieldwork | A 52% probability of winning |
| 70-cent last trade | A transaction price for that exact contract | A calibrated 70% forecast or an available purchase price |
| A model's 70% win probability | An outcome estimate under its inputs and assumptions | Certainty, a 70% vote share or a profitable entry |
A national vote-share poll and an electoral-college contract have different targets. A party-win contract and a named-candidate contract can also settle differently. Match geography, eligible population, outcome definition and deadline before interpreting a gap.
A lead needs an uncertainty model
Suppose a fictional election forecast has a central two-point lead. In one deliberately simplified model, all three possible final margins are +1, +2 and +3 points, equally likely: victory occurs in all three. In another, the margins are −2, +2 and +6, equally likely: victory occurs in two of three. Both models have the same average two-point lead, but different win probabilities.
These are teaching distributions, not election models. Real forecasting must address turnout, undecided voters, changes after fieldwork, correlated errors and the electoral rules. There is no universal conversion from a four-point polling lead to a 75% or 85% chance of victory.
AAPOR's polling-accuracy guide discusses coverage, response and turnout challenges. Its election resources help locate methodology disclosures. Check sample recruitment, population, field dates, question wording, weighting and uncertainty. A method label such as “online” or “telephone” is not a sufficient quality verdict.
Freeze a fair comparison before the result
Choose the event set, forecast horizon and scoring rule first. For example, save each eligible forecast seven days before its defined resolution deadline. Preserve cancelled or unresolved events with an exclusion rule; do not quietly discard inconvenient losses. Do not compare one source's final update with another's month-old estimate.
For a market, save the contract rules, timestamp, last trade, bid, ask and available depth. A last trade may be stale; a midpoint may not be executable. Spreads, fees, capital constraints and participant selection can affect prices. Rapid updates and large volume do not prove accuracy, while a slow update may still reflect valuable evidence.
For a polling-based forecast, save the model version and published probability. A raw vote-share poll cannot be scored as a win probability without an explicit conversion model. Keep the poll's own share-estimation error as a separate measure.
Score probabilities, not just winners
For a binary outcome, the Brier error is (p − y)^2, where y is 1 if the event happens and 0 otherwise. Lower average error is better on the same defined cases. Metaculus explains the score and baseline comparison; that explanation is not evidence of market superiority in elections.
Scroll sideways to see every column. Keyboard: focus the table region and use the arrow keys.
| Synthetic case | Forecast p | Outcome y | Squared error |
|---|---|---|---|
| A | 0.70 | 1 | 0.09 |
| B | 0.60 | 0 | 0.36 |
| C | 0.20 | 0 | 0.04 |
| D | 0.40 | 1 | 0.36 |
| Average | — | — | 0.2125 |
Predicting 0.50 for every case scores 0.25 here. Four examples demonstrate arithmetic, not statistical superiority. Related elections are not independent observations. Report the number and mix of events, forecast timing, exclusions and uncertainty before drawing a general conclusion.
Calibration asks a different question: among many forecasts around 70%, did roughly 70% occur? Seven wins among ten such cases is compatible with calibration but is too little evidence to certify it. One 60% favorite winning does not vindicate a forecaster; a 60% favorite losing does not alone refute one.
Use disagreement to investigate
Write down whether the gap comes from a different target, stale data, turnout assumptions, contract wording or genuinely different evidence. If the explanation is unknown, record that. Do not average sources 70/30 merely because one has money behind it, and do not treat agreement as independent corroboration when a market may already incorporate the same polls.
You can keep this record without depositing money. Better forecasting scores do not establish tradable profit after prices and costs. See the paper research workflow for an evidence-first journal with an insufficient-information outcome.
Disclosure: AITR may earn commissions from marked referral links elsewhere. These methodology links are not endorsements of a funded strategy. Trading can lose capital; this is statistical education, not financial advice.