Structured Interviews

In the 1950s, the nascent Israel Defense Forces needed a way to evaluate young soldiers applying to become officers. They had a squad of interviewers and a task that they considered physically and mentally taxing. From the interviews and the climbing activity, the IDF identified which soldiers would make good officers. The problem was that their rankings didn’t line up with reality. Their score on the interview and task didn’t predict their performance in officer training.

The only psychologist available for improving their procedures was twenty-one-year-old Daniel Kahneman, at the very beginning of his Nobel Prize-winning career. He came to realize that the interviewers were extremely confident in their abilities to judge candidates, but the numbers didn’t back this up. He devised a new system for structuring and rating their interviews that received a lot of pushback from the interviewers. Despite the resistance, however, his system proved to be much more successful and is still being used to this day (Lewis, 2017, pp. 77-83).

What can we learn from the experience of the IDF and the current research on interviews? The job interview is one of the most basic elements of any hiring process. Anyone who is in charge of hiring at a school has experienced an interview (at the very least, their own) and it seems like a pretty straightforward process. Meet the candidates, get to know them, learn about their work experience, hear about their educational philosophy. With a general impression from the interview and a model lesson, you’re good to go.

The problem is that when an interview is run poorly, we’re not actually very good at predicting people’s job performance. The issue is not with interviews, per se, but is with the way that the interview is administered and rated. Whereas structured interviews are a useful tool for identifying talent, unstructured interviews don’t provide any more useful information than an applicant’s general intelligence and conscientiousness score (Cortina et al., 2000). This means that employers could get more information by giving an intelligence and personality test than by running an unstructured
interview. In fact, when given objective data like intelligence tests and personality measures, people making hiring decisions made worse predictions about future performance if they also had access to a rating from an unstructured interview. Unstructured interviews led to worse predictions of school performance in the lab (Dana, Dawes, & Peterson, 2013) and work performance with experienced hiring managers (Kausel, Culbertson, & Madrid, 2016). How can we avoid the pitfalls of an unstructured interview?

HERE ARE THREE IMPORTANT GUIDELINES FOR PREPARING AND RUNNING AN INTERVIEW:

1. Identify the traits that you need to measure. The first step in preparing for an interview is identifying what traits an applicant needs in order to succeed. There are many which apply to all schools and others which individual schools might look for. For example, conscientiousness is associated with success across many professions and roles (Sackett & Lievens, 2008). Alternatively, a school that values collaboration and which expects teachers to work closely should look for team-related skills, such as conflict resolution and collaborative problem solving. Identify the traits and behaviors that set successful teachers apart from the rest and write questions that target those areas. Unstructured interviews tend to leave out knowledge, skills, and abilities necessary to succeed at a job (Schmidt & Zimmerman, 2004).

When the IDF tried to identify future officers, the general impression that interviewers picked up was not tied into the specific skills and abilities that actually predicted success. Daniel Kahneman, through contact with experts in the field, compiled a list of traits that interviews should target such as “pride.”

Rooting the interview in specific factors allowed those in charge to recognize applicants who exhibited the skills and abilities that lead to success.

2. Ask the same questions to every candidate. It might feel natural to have a few questions lined up for an interview and then let the rest flow from there. This gives the interview a comfortable, conversational quality but it limits the usefulness of the interview. An interview should help employers differentiate between applicants and asking unique questions to each individual will make it more difficult to compare people. Furthermore, since the questions should be rooted in the specific traits of interest, leaving out questions or changing the questions would make them less informative. This is why the core of the interview should be the same set of questions, asked in the same order, using the same wording, to each potential employee. This does not preclude follow up questions, since that extreme level of rigidity does not seem to make interviews significantly more predictive (Dipboye & Johnson, 2013).

In the 50s, Daniel Kahneman standardized the set of questions which were included in the officer interview, based on his research into which dimensions were important for a military officer. This is one more element of structure that the interviewers resisted, feeling that they were losing more control over the process, but one which allowed the system to become more consistent.

3. Create a rating system for evaluating answers. Walking away from an interview with a general impression of the applicant is too vague to be useful. After an unstructured interview, hiring managers are left with nothing but an ambiguous metric to compare candidates with. Giving a score based on this overall feeling will not lead to better decision making. Instead, the questions should be scored using a pre-established rubric, like rating each answer on a numbered scale. The higher scores can be anchored in behaviors that demonstrate the trait or that high performers exhibit, while the lower end would refer to behaviors that don’t demonstrate it or to generic answers that don’t really address the issue (see Appendices 1 and 2 in Hamdani, Valcea, & Buckley (2014) for two examples). When the administrative team discusses in advance what a good answer looks and sounds like (as well as an inadequate answer), there is likely to be greater consistency in interviewer perceptions and it is more likely to align with the principles guiding the school.

The final piece of Daniel Kahneman’s changes to the interview was creating a 5-point rating scale. Interviewers gave a score to each section before moving on to the next, so that there was a score given to each dimension being assessed.

It should be clear that an interview is only one piece of a larger human resource strategy. To do it properly requires an analysis of what traits are exhibited by successful teachers in the school, along with what traits the school is looking to foster. Planning the interviews and creating a feedback loop to know if the hiring process is working require a solid system for performance evaluation, to identify who the top performers are. Structuring interviews is not a magic solution but it is a key component in making interviews useful.

One final note on the importance of structured interviews: It might seem that improving the interview process is important only for a school with a large applicant pool, since it needs to distinguish between the high potential employees and the others. A school which has few candidates for any open position doesn’t seem to have much choice anyway, no matter what the interview looks like. In fact, this is a mistake. Any time there is more than one option, including skills important for the job, increasing consistency, and designing a rating system will help hiring managers differentiate between applicants. Even if there is only one applicant, it will be beneficial. The structure of the interview can reveal areas of potential weakness so that the school administration can proactively plan a support system. Understanding the knowledge, skills, and abilities of new teachers will help their supervisors set them up for success.

An unstructured interview can be detrimental to your hiring process, but a well-run structured interview can be vital. Unstructured interviews might not lead to better decisions, but they do lead to greater confidence in those decisions (Kausel, Culbertson, & Madrid, 2016). Schools need to make sure that their hiring decisions are not being impaired by intuition.

Using these three guidelines for structuring interviews, schools can improve their hiring process, identify future talent, and ultimately create better learning environments.

Rabbi Yoni Gold is an educator in Skokie, IL. With degrees in Jewish Education and Industrial/Organizational Psychology, he brings the insights of psychology into the classroom and to working with school administrations. You can reach Yoni at rabbiyonigold@gmail.com.

References

Cortina, J. M., Goldstein, N. B., Payne, S. C., Davison, H. K., & Gilliland, S. W. (2000). Te incremental validity of interview scores over and above cognitive ability and conscientiousness scores. Personnel Psychology, 53, 325-351.

Dana, J., Dawes, R., & Peterson, N. (2013). Belief in the unstructured interview: Te persistence of an illusion. Judgement and Decision Making, 8(5), 512-520.

Dipboye, R. L., & Johnson, S. K. (2013). Understanding and improving employee selection interviews. In K. F. Geisinger (Editor-in-Chief), APA Handbook of Testing and Assessment in Psychology: Vol. 1. Test Theory and Testing and Assessment in Industrial and Organizational Psychology (pp. 479-499), Washington, DC: American Psychological Association.

Hamdani, M. R., Valcea, S., & Buckley, M. R. (2014). The relentless pursuit of construct validity in the design of employment interviews. Human Resources Management Review, 24, 160-176.

Kausel, E. E., Culbertson, S. S., & Madrid, H. P. (2016). Overconfdence in personnel selection: When and why unstructured interview information can hurt hiring decisions. Organizational Behavior and Human Decision Processes, 137, 27-44.

Lewis, M. (2017). Te Undoing Project: A Friendship That Changed Our Minds. New York: W.W. Norton & Company.

Sackett, P. R., & Lievens, F (2008). Personnel selection. Annual Review of Psychology, 59, 419-450.

Schmidt, F. L., & Zimmerman, R. D. (2004). A counterintuitive hypothesis about employment interview validity and some supporting evidence. Journal of Applied Psychology, 89(3), 553-561.

• Print
• Email