forecastingprinciples.com Reviews of Important Papers on Forecasting,
1985-1995 Reviews
Review of:

Karl Halvor Teigen (1990), "To be convincing or to be right: A question of preciseness," in: K.J. Gilhooly, M.T.G. Keane, R.H. Logie and G. Erdös (eds.), Lines of Thinking (Wiley, Chichester) 299-313.


This paper reports on a fascinating set of studies about what Teigen calls the "preciseness paradox." That is, under a wide variety of circumstances, the more precise the forecast, the more confident we are about the forecast. In fact, we should be less confident. The cause of the paradox is that when a forecaster provides detail, this is a cue that the forecaster has much expertise about the topic. "She must know what she is talking about!" Thus, this effect should be stronger for post-diction than prediction. It was. Consider one of Teigen’s studies. Subjects were asked how much confidence they would have among different informants when they visit Iceland and receive answers to the following question:

Owing to various price regulation measures, this year’s inflation rate was down to 5%. Was it higher last year?

Responses:

Olafur said "Yes, it was."
Larus said "Yes, it was about 7%."
Jon said "Yes, it was between 5 and 9%."

Which of these statements will you be most confident about? Rank order the alternatives and try to give some of your reasons.

Teigen says that Olafur’s statement is the most general, and Larus’s the most exact. If Larus is right, so are Olafur and Jon. On the other hand Olafur can be right, while Larus and Jon are wrong (if inflation were to be 14%, for example). However, most of the subjects (16) were most confident in Larus. Eight subjects were most confident in Jon. Only seven subjects were most confident on Olafur.

When the statements about inflation were converted from the past to be a forecast about next year’s inflation, the confidence in the most precise forecast (by Larus) decreases (to six of 34 subjects), but does not disappear. Teigen suggests that this occurs because people would not expect forecasters to be able to provide precise forecasts of inflation.

In situations where it is expected that experts can make good forecasts, added detail and preciseness are likely to lead people to have more confidence about the forecasts. This is often a problem when using scenarios to make forecasts. (I have argued elsewhere that scenarios should not be used to make forecasts; instead they should be used in the implementation phase (see Armstrong, 1985, pp. 40-45). The obvious solution is that the forecaster should provide quantitative estimates of the likelihood associated with forecast. If this is not acceptable within the organization, warming labels can be applied when scenarios are used. I suggest "Caution: It is difficult to assess uncertainty when using scenarios. The purpose of the scenario is to create a situation that seems plausible so that it can be used in contingency planning."

Reference

Armstrong, J.S., 1985, Long-Range Forecasting (Wiley, New York).