forecastingprinciples.com Reviews of Important Papers on Forecasting,
1985-1995 Reviews
Review of:

Anthony T. Dalessio, 1994, ‘Predicting insurance agent turnover using a video-based situational judgment test,’ Journal of Business and Psychology, 9, 23 – 32.


Improvement in the ability to predict which employees or insurance agents will be successful is of interest because of the high costs involved with recruiting, training, and poor performance. Companies typically lose about 44% of their agents in the first year. Fortunately, research on personnel selection continues to produce effective prediction procedures. One reason that predictive procedures have been improving is that more recent research has tried to develop realistic job sample tests to determine how the candidate is likely to perform on the job. The basic assumption is that the closer the correspondence between the predictive test and the criterion, the better the predictive validity. Yes, this principle does look obvious, but it has been ignored in many fields.

Dalessio’s study examines predictive validity where recently contracted insurance agents responded to video-based situational items, each of which presented realistic but unresolved sales situations. In these, agents received objections from prospects when asking for appointments, closing sales, and requesting referrals. Following each situation, viewers were presented with a multiple choice question asking what they would do if they were the agent in the situation.

This study builds upon prior research that showed the value of video-based situational tests, but it is the first to use an objective performance measure as a criterion. The criterion predicted by the test was the amount of turnover in the first year.

The tests were administered to 677 agents from 14 insurance companies, and were completed primarily within the first 3 months on the job. Interestingly, this test was administered after the normal selection procedures had been used to hire the agents, and so was being tested as an additional hurdle in the selection process. Also, the overall retention rate for the companies in this sample was already about 10% higher than the industry average, so this was a difficult situation in which to show improvement.

The sample was randomly divided into three parts to test the validity of the video test. Each part was used as a hold-out validation sample with item scoring weights that were based on the rest of the sample. Then another part was used.

The video-based selection test proved to be highly predictive. For example, the average retention for those who scored in the top quarter of the test was 78%, while for the next three quarters the retention rates were 66%, 62% and 59%.

Improved selection tests such as the one shown here should provide a high return on investment to companies by lowering costs associated with agent turnover. Unfortunately, research on personnel selection is often ignored by companies (Dakin and Armstrong, 1989).

Reference

Dakin, S. and J.S. Armstrong, 1989, Predicting job performance: A comparison of expert opinion and research findings, International Journal of Forecasting, 5, 187-194.