Reviews of Important Papers on Forecasting,
1985-1995 Reviews
Review of:

Phillip A. Braun and Ilan Yaniv (1992), ‘A case study of expert judgment: Economists probabilities versus base-rate model forecasts,’ Journal of Behavioral Decision Making, 5, 217 – 231.

At times, I thought that I needed my Sky King decoder ring for this paper. It was difficult for me to keep things straight such as that an increase on their Exhibit 2 meant that the economy was getting worse. Nevertheless, this paper does reward the reader’s effort.

The major conclusions from this study of 40 economic forecasters are that (1) they were no better than a model in forecasting turning points in real GNP for one quarter ahead, and (2) they were significantly worse for two, three, and four quarters ahead. The findings were based on forecasts of a total of 1612 one-quarter-ahead forecasts, 1637 two-aheads, 1641 three-aheads, and 1510 four-aheads over the period from the fourth quarter of 1968 to the end of 1989. They were collected as part of the survey of professional economic forecasters made by the National Bureau of Economic Research and the American Statistical Association. The survey was sent out in the second month of every quarter and completed in the third month. The model vas a simple base-rate model: it looked at the historical probability the n-ahead forecast would go down, conditioned by what happened in the most recent quarter. Thus, the model provided two updated probabilities, one for when the last quarter had trended down and one when it had trended up.

Previous research has typically shown that judgmental forecasts improve accuracy for short-range forecasts. So this study does contain news. In addition to making one wary of the judgmental forecasters, the paper also raises the hope hat the models might be improved. For example, does the judgmental forecast contain any information to add to the model? (This could be tested by taking a weighted average of the model and economists’ forecasts.) Could the model be improved by basing it on more than just the history of the most recent quarter? For example, might it help to know that the economy has been trending downward in the previous two quarters? Could it help to shrink the model estimates, perhaps towards some long-term unconditional average for upwardly or downwardly trending series, or perhaps a bit towards 0.5 (complete uncertainty)? Thus, the paper provides not only useful findings, but also hope. This study really does merit further research.