forecastingprinciples.com Reviews of Important Papers on Forecasting,
1985-1995 Reviews
Review of:

Hoch, Stephen J. (1985), "Counterfactual reasoning and accuracy in predicting personal events," Journal of Experimental Psychology: Memory and Cognition II, 719-731.


[Review written with J. Scott Armstrong]

Here is more evidence to support advice from Benjamin Franklin. Namely, you will be able to make better forecasts if you explicitly generate ideas for and against the forecast. This experiment is a useful extension of the Koriat, Lichtenstein, and Fischhoff (1980) study; in that earlier study, the listing of arguments why the prediction might be wrong was found to reduce overconfidence in "predicting" the answers to almanac questions. Fischhoff and MacGregor (1982) found additional, though mild support, in a study involving actual predictions.

Hoch asked 260 MBA students to forecast the outcomes of their job search efforts over a nine-month forecast horizon. The forecasts covered three different events (e.g., "What is the probability that you will receive more than XX job offers by the end of the school year?") The results showed that the key is to generate explicit arguments against a favorable outcome (or for an unfavorable outcome). This procedure lessened the optimistic bias, though it did not eliminate it. It also improved the accuracy. The description of explicit arguments for a favorable action neither improved nor diminished accuracy. Thus, if it is not clear what is favorable, use Franklin's advice and develop both pro and con arguments. Hoch found that the improvement in accuracy occurred for events with a low base rate, as expected. It also occurred for events with a high base rate; this latter result was a bit surprising as it was suspected that the generation of arguments against an event that occurs very frequently could lead to underconfidence. The study has a practical implication: prior to making judgmental forecasts, the forecasters should write about the reasons why a favorable forecast might be wrong. It is not known to what extent it would help to produce such reasons after the judgmental forecast has been made.

References:

Fischhoff, B. and D. MacGregor (1982), "Subjective confidence in forecasts," Journal of Forecasting, 1, 155-172.

Koriat, A., S. Lichtenstein, and B. Fischhoff (1980), "Reasons for confidence, Journal or Experimental Psychology: Human Learning and Memory 6, 107-118.