forecastingprinciples.com Reviews of Important Papers on Forecasting,
1985-1995 Reviews
Review of:

Michael A. McDaniel, D. L. Whetzel, F. L. Schmidt and S.D. Maurer, 1994, The validity of employment interviews: A comprehensive review and meta-analysis, Journal of Applied Psychology, 79, 599-615.


Do employment interviews provide valid predictions of job performance? McDaniel et al. present the results from a meta-analysis. The paper is a model of how to properly conduct a meta-analysis. For example, it describes the rules for the literature search and the procedures for coding and analyzing the studies. Their literature search alone was a major undertaking that was conducted over a 8-year period. Their findings should be of interest not only for personnel

forecasters, but also to forecasters in general because of certain methodological findings. In addition to studying (1) whether the interview has predictive validity, they examined (2) the relative value of structured versus unstructured interviews, (3) group versus individual interviews, and (4) the effect of giving an interviewer access to relevant predictor scores prior to the interviews.

  1. With respect to overall validity, they summarized previous studies of correlations between interviews and job performance. These studies, containing 160 comparisons drawn from over 25000 subjects, indicated that the interview does have some predictive validity. The typical correlation was about 0.20. McDaniel et al. state that this finding conflicts somewhat with traditional beliefs. They state "In our experience, many personnel psychologists believe that the interview generally has low validity." Their experience conflicts with findings from Dakin and Armstrong’s (1989) survey of personnel consultants in New Zealand. These respondents placed the validity of. the interview second only to experience in a list of 11 leading procedures. A replication by Ahlburg (1992) found that the interview was among the more effective techniques as judged by US experts.
  2. Consistent with research in other areas of forecasting, structure aids judgment. In general, the correlation was higher for structured interviews, especially when the content of the interview was job related. Interestingly, however, this was not the case when the criterion was the candidates’ performance during training courses.
  3. Individual interviews were more valid than when a group of interviewers were used. Why might this be? Perhaps the stress on the interviewee will lead to effects other than job-related responses. (Note that this is not a test of combining because the members of the group are not acting independently.) This finding is especially useful because individual interviews are less expensive.
  4. When interviewers were provided with relevant information in advance (namely, cognitive test scores) their overall validity was reduced. I find this surprising. Perhaps the quantitative evidence, which represents only a small part of the story, is weighed too heavily.

While the study shows that the interview has some predictive validity, I have three reservations about how these findings might be used. First, the criterion was typically ‘rated performance’ and this might not be closely related to ‘actual performance.’ Bommer et al. (1995), in a review based on three studies, found a mean correlation of only 0.39 between subjective and objective measures of performance. Luthans (1988, Chapter 9), for example, found no relationship between managerial success (e.g. promotions) and actual effectiveness (e.g. quantity produced by the group). Assume that personal style counts heavily in one’s subjective performance ratings. It is likely also to help in the interview. Thus, one would expect to find a positive relationship between interviews and ratings even if there were no relationship to actual performance.

The second reservation is whether the interview is superior to alternative techniques. For example, realistic job samples, bootstrapping (expert systems), and assessment centers have been shown to be effective. Of course, it may be that the interview, properly used can add predictive value.

A third reservation, noted by McDaniel et al., is that the interview as used in practice probably tends to be even less structured than the unstructured interviews in their study.

In general, this paper provides a useful summary of all that is known about the personnel interview when used alone as a predictive tool.

References

Ahlburg, D.A., 1992, Predicting the job performance of managers: What do the experts know? International Journal of Forecasting, 7, 467-472.

Bommer, W.H., J.L. Johnson, G.A. Rich, P.M. Podsakoff and S.B. MacKenzie, 1995, On the interchangeability of objective and subjective measures of employee performance: A meta-analysis, Personnel Psychology, 48, 587-605.

Dakin, S. and J.S. Armstrong, 1989, Predicting job performance: A comparison of expert opinion and research findings, International Journal of Forecasting, 5, 187-194.

Luthans, F., 1988, Real Managers (Ballinger, Cambridge, MA).