forecastingprinciples.com Reviews of Important Papers on Forecasting,
1985-1995 Reviews
Review of:

Terence Hancock, Julia Lane, Russ Ray, and Dennis Glennon (1992), “The Ombudsman: Factors Influencing Academic Research Productivity: A Survey of Management Scientists,” Interfaces, 22(5), 26-38.


[Review by: J. Scott Armstrong, The Wharton School, University of Pennsylvania, Interfaces, 22(5), 26-27.]

Some researchers are more prolific than others. What do you think explains the differences between those who publish a great deal and those who publish less?

Hancock, Lane, Ray, and Glennon surveyed researchers who published in Management Science and Operations Research . They compared those with high rates of publication to those with lower rates. They asked how these researchers spent their time, whether they enjoyed research, where they obtained support, what things interfered with their research, and how they selected research topics.

Some of the Hancock research examines whether there is a conflict between teaching and research. They found that prolific publishers spend 32 percent less time on teaching-related activities. A literature review by Marsh (1984) indicated little evidence of a relationship between the number of papers published by professors and students’ ratings of these professors. Consistent with this, the literature review by Abrami, Leventhal, and Perry (1982) found little relationship between lecture content and students’ ratings of professors. (Lecture content was related to student achievement, but student achievement almost never plays a role in faculty promotions and pay.) These findings imply that, given the current reward systems based on publications and student ratings, faculty members should spend less time on teaching-related activities.

After Hancock’s paper was reviewed and accepted for publication, I asked four academics to common on its findings. Mike Mahoney, who has studied the publication process, questions whether to be prolific is to be useful to the scientific community. He believes that published papers are not useful unless they are read and applied. Because of the numerous barriers to publication, Ray Hubbard suggests that citations may be a better measure of scholarly productivity than publication counts. Finally, Fred Collopy, adding the perspective of a recently minted Ph.D., concludes that the implications of the study are not encouraging to researchers who are curious about practical problems.

Raymond Hubbard, Fred Collopy, and I viewed the comments. I then sent the complete package to three outside referees, Gary Lilien, William Ross, and Frederic Murphy. They suggested changes in the original paper and in all of the comments.

The commentators expressed concern that research productivity would be equated with the quality of the research publications. Hancock’s study presents no evidence that the quality of papers is constant across the respondent groups. Quality could differ for many reasons: prolific researchers (1) are not as careful, (2) have more joint publications, (3) are more likely to use the LPU strategy (submit the least publication unit), (4) concentrate on small insignificant issues, (5) product high quality papers, or (6) do not have controversial findings to report (according to Armstrong and Hubbard 1991, papers with controversial findings are likely to be rejected for publication.) In any event, simple publications counts remain an important factor in salary, promotion, and tenure decisions at major universities.

Read the Hancock paper, which follows, to examine whether the characteristics of product researchers differ from your expectations.

References

Abrami, P.C., Leventhal, L., and Perry, R.P., 1982, “Educational seduction,” Review of Educational Research, Vol. 52, No. 3, pp. 446-464.

Armstrong, J.S. and Hubbard, R., 1991, “Does the need for agreement among reviewers inhibit the publication of controversial findings?” Behavioral and Brain Sciences, Vol. 14, No. 1, pp. 136-137.

Marsh, H. W., 1984, “Students’ evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility,” Journal of Educational Psychology, Vol. 76, No. 5, pp. 707-754.