forecastingprinciples.com Reviews of Important Papers on Forecasting,
1985-1995 Reviews
Review of:

Alison Hubbard Ashton (1985), Does consensus imply accuracy in accounting studies of decision making?, Accounting Review 60, 173-85.


In many situations no objective criterion is available by which the forecasts and decisions can be judged. Instead, consensus is used as a proxy for accuracy. Is this valid? Little direct study of this issue has been done. Holtzman and Sells (1954) in a study on the success of aviators, found little relationship between consensus and actual outcome. Ogburn (1934) concluded that high agreement among sportswriters was not closely related to accuracy. However, a re-analysis of data from Walker (1970) by Armstrong (1985, p. 145) showed a relationship between consensus and accuracy.

Ashton’s study helps to fill this gap in the literature. She examined two different forecast situations: forecasts of annual advertising sales for Time magazine by 13 Time, Inc. executives given forecast horizons for 1, 2 and 3 quarters and covering 14 years; and forecasts by 25 auditors of ‘going concern’ problems, like bankruptcy, for 40 firms. Using two criteria, correlations and mean absolute differences, Ashton compared the actual degree of agreement (between forecasts by two different judges) against the accuracy of the judges. She also compared the agreement of judges (among all other judges) and related it to that judge’s accuracy. As she says, the results are good news. Agreement among judges does imply greater accuracy. The relationships were strong and statistically significant. This gives a bit of confidence for using consensus as a proxy for accuracy when formal assessment of accuracy is not feasible. One hopes additional research will suggest rules-of-thumb for forecasting in various situations because many studies lead one to question this relationship. Also, would it be possible to use measures of consensus to assess the uncertainty about a given forecast?

References

Armstrong, J. Scott, 1985, Long-range forecasting, second ed. (Wiley, New York).
Holtzman, W.H. and S.B. Sells, 1954, Prediction on flying success by clinical analysis of test protocols, Journal of Abnormal and Social Psychology 49, 485-490.
Ogburn, William F., 1934, Studies in prediction and the distortion of reality, Social Forces 13, 224-229.
Walker, H.E., 1970, The value of human judgement, Industrial Marketing Research Association Journal 6, 71-74.