What exactly may happen?
Define the event.
Ambiguous wording makes later evaluation impossible.
Side 27
Forecasting &
Uncertainty
A study of making explicit bets about uncertain futures. Forecasting turns vague expectation into a probability, defines what would count as resolution and learns by comparing confidence with outcomes.
question→base rate→drivers→probability→resolution
05forecast stages
06uncertainty sources
05scoring habits
27Side
A forecast must be resolvable.
“I think this will probably happen” is not yet a forecast. A usable forecast defines the event, time horizon and probability before the outcome is known.
By when?
Fix the time boundary.
Probability depends heavily on the forecast window.
What similar events exist?
Find the base rate.
Outside-view frequency helps anchor judgment before case details dominate.
What could move the probability?
List mechanisms, not headlines.
Drivers should connect plausibly to the target event.
How will the forecast be scored?
Specify the evidence source.
A forecast should tell future-you exactly how to decide whether it resolved true or false.
Start outside the story.
Base rates prevent vivid details from making a case feel more unique than it is.
Which past cases are comparable?
Too broad and the comparison loses relevance; too narrow and the sample disappears.
What usually happens?
Begin with frequencies across similar cases before incorporating case-specific information.
What is special here?
Case details can justify movement away from the base rate when they are genuinely diagnostic.
Extremes often move toward typical values.
Exceptional recent performance may partly reflect noise that will not persist.
Visible cases may be selected.
Successful firms, projects or technologies are easier to study than failures that disappeared.
History may stop being representative.
Base rates should be down-weighted when the underlying process has genuinely changed.
Large questions become tractable when broken apart.
Decomposition turns an opaque forecast into smaller conditional questions whose assumptions can be inspected.
Chain
What must happen first?
Break the event into sequential prerequisites and estimate each link.
Drivers
Which variables move the outcome?
Estimate demand, capacity, regulation, financing or other mechanisms separately.
Branches
What alternative paths exist?
Scenario trees help when several mutually exclusive routes can lead to different outcomes.
Conditional
What if one event occurs?
Conditional forecasts reveal where uncertainty is concentrated.
Recombine
Does the math match the story?
Reconstruct the whole forecast carefully so dependencies are not double-counted.
Forecast habitopaque question → smaller mechanisms → explicit probabilities → recombined judgment
Confidence should mean something over time.
A forecaster is calibrated when events assigned a given probability occur at roughly that frequency across many forecasts.
Does 70% mean about 70%?
Calibration evaluates reliability of stated confidence across repeated forecasts.
Are probabilities decisive?
A forecaster who always says 50% may be calibrated but not very informative.
Score probabilistic accuracy.
Squared error penalizes confident mistakes more strongly than modest ones.
Do beliefs move?
Forecasts should change when new evidence changes the expected likelihood of the event.
Write forecasts before outcomes.
Timestamped records prevent hindsight from rewriting prior confidence.
Which methods improve performance?
Forecast journals make it possible to test whether particular rules actually improve accuracy.
Some uncertainty cannot be compressed into one forecast.
Scenario work is useful when several coherent future states would change strategy differently.
Critical uncertainties
Identify variables that are both highly consequential and genuinely uncertain.
Scenario logic
Construct distinct combinations rather than one optimistic, one neutral and one pessimistic version of the same story.
Internal consistency
Each scenario should contain mechanisms that fit together plausibly.
Indicators
Define observable signals that would suggest the world is moving toward one scenario.
Robust actions
Identify decisions that perform acceptably across multiple scenarios.
Contingent actions
Predefine moves that become attractive only if particular signals appear.
Forecasting improves only if forecasts are reviewed.
Outcome review should separate bad luck from bad reasoning and identify recurring sources of miscalibration.
You forecast 80% and the event did not happen.
One miss does not prove the forecast was poor. Review the reasoning, compare against similar 80% forecasts and ask whether the evidence justified such confidence.
Your forecasts are accurate but always near 50%.
You may be well calibrated but under-resolved. Look for cases where evidence could justify stronger movement away from the base rate.
A major surprise changed the environment.
Distinguish unforeseeable shock from ignored possibility. The review should ask whether the scenario deserved nontrivial probability beforehand, not whether its exact form was predictable.
You were right for the wrong reason.
Do not reward the reasoning merely because the outcome matched. Compare the predicted mechanism with what actually caused the event.
SuperforecastingPhilip Tetlock & Dan Gardner · probabilistic judgment
The Signal and the NoiseNate Silver · prediction across domains
Forecasting: Principles and PracticeHyndman & Athanasopoulos · statistical forecasting
Thinking in BetsAnnie Duke · uncertainty and outcome review