forecastingprinciples.com Reviews of Important Papers on Forecasting,
1985-1995 Reviews
Review of:

Milton Friedman (1991), "A cautionary tale about multiple regression" (the appendix to "Alternative approaches to analyzing economic data" by Milton Friedman and Anna J. Schwartz), American Economic Review 81,   48 – 49.


Friedman reported on his use of multiple regression in 1945. As a statistical consultant, he was asked to analyze data on alloys used in turbine blades for engines. The task was to develop an alloy that would withstand high temperatures for long periods. Friedman proposed a single equation regression that sought to explain ‘time to fracture’ as a function of stress, temperature, and variables representing the composition of the alloy. To obtain estimates for this equation, along with associated test statistics, it would take a highly skilled operator about three months to calculate the equation. Fortunately, there was one large scale computer in the country that could perform the calculations. This computer, the Mark 1, was at Harvard. It was built from a number of IBM card-sorting machines and was housed in an enormous air-conditioned gymnasium. Not counting data input, it required 40 hours to calculate the regression. The size of the regression was such that it could be solved in a matter of seconds on today’s desktop computers.

Friedman was delighted with the results. It had a high R and performed well on all of the relevant statistics. This regression allowed Friedman to construct two new alloys. Using the regression model, he predicted that each alloy would survive for several hundred hours at very high temperatures. Because this was metallurgy, not economics, he was able to test the predictions in short order. A MIT lab carried out the tests. Friedman was sufficiently skeptical that he did not reveal his predictions. As it turned out, rather than surviving for hundreds of hours, each alloy ruptured in less than four hours. Friedman concluded from this that the statistical measures associated with regressions are of little va1ue for judging the ability of the model to predict with new data. This has been a common theme in much of the research over the past three decades (see Pant and Starbuck, 1990, for recent evidence on this issue). Nevertheless, the lesson does not seem to have spread widely. Journals are still filled with papers that cling to faith in their regression statistics. What saves most of these authors is that their predictions will never be put to the test. After all, if they were unable to test it themselves, why should they expect others to do so? Imagine how much space would be saved in economics journals if authors were required to test their predictions.

Reference

Pant, P. Narayan and William H. Starbuck, 1990, "Innocents in the forest: Forecasting and research methods," Journal of Management, 16, 433-460.