An Evidence Based Guide to Hiring Interviews

Without question, interviews are the most widely used means of selecting employees. To help industry professionals, we have designed an evidence based guide to hiring interviews. One popular misconception is that asking all applicants the same set of questions is sufficient for a structured interview. Structured interviews, on the other hand, are far more than that.

WHAT’S THE DIFFERENCE BETWEEN A STRUCTURED VERSUS AN UNSTRUCTURED INTERVIEW?

Structured interviews are based on a job analysis. This means assessment questions are job related. Structured interviews tap in to job knowledge, job skills, applied mental skills, and job related interpersonal skills. In contrast, unstructured interviews concentrate on general intelligence, education, training and other personal attributes which may not apply for a particular role.

Structured interviews involve a standardised scoring procedure. For that reason, interviewers use a scoring key to evaluate responses before making a decision regarding candidates. As a result, most responses are scored using one of three methods: right or wrong, typical-answer approach, and the key-issues approach.

STRUCTURED INTERVIEWS ARE A LOT OF WORK. WHY BOTHER?

Structured interviews are more valid than unstructured interviews. In other words, they can better predict job performance of candidates. Using a standardised interview format increases your chances of making a more informed decision on a candidate.

They also add predictive power to the use of cognitive ability tests. Cognitive ability is a predictor of performance on the majority of jobs. So, using them in addition to a ability tests increases the level of prediction beyond what the test has already shown. 

A cognitive ability test combined with a standardised interview will result in a more reliable and thorough assessment. Besides, structured interviews have a significantly lower negative impact on bias and prejudice and can help you avoid legal issues.

More can be found in our evidence based guide to hiring interviews which you can download and use to optimise your selection process.

WHY DON’T UNSTRUCTURED INTERVIEWS PREDICT JOB PERFORMANCE? (7 FACTORS)

1. Poor Intuitive Ability – Interviewers often make hiring decisions based on “gut reactions” or intuition. People, on the other hand, are not very good at using instinct to predict behaviour, whether it be potential job success or deceptive acts during the interview. Contrary to what many HR professionals think, there is no evidence indicating that some interviewers are able to predict behaviour, whereas others are not.
2. Lack of Job Relatedness – What do you see yourself doing five years from now? or How would you describe yourself? As you can see, they’re not related to any particular job. Not to mention, many of them are also illegal (e.g., Are you married?). Since proper responses to these questions have yet to be determined, evaluating them and making informed hiring decisions is impossible. Nonetheless, certain answers will be preferred by managers over others. Even so, preference alone doesn’t mean that they will predict future job success. It follows that, only information that is job related has that predictive power.
3. Primacy Effects – or “first impressions” in the interview refer to the higher impact that information presented prior to or early in the interview tends to have in comparison to the information presented later in the interview. Their occurrence may lead to a bad hiring decision. To reduce potential primacy effects make repeated judgments throughout the interview. For instance, try and rate responses after each question rather than waiting until the end of the interview to make a single, overall judgment of the candidate’s suitability for the job.
4. Contrast Effects – They occur when a candidate’s interview performance affects the interview score given to the next applicant. In other words, we may experience a contrast effect when we judge a candidate’s performance in relation to previous applicants’ performance. For example, if a poor applicant precedes an average one, the interview score of the latter will be higher. That is if no applicant or a very qualified one preceded them. However, in all fairness, it’s more important to train interviewers to be aware of possible contrast effects to reduce their occurrence.
5. Negative Information Bias – Negative information weighs more heavily than positive information. We’ve shown this in an article we wrote about the science behind irrationality. 

However, this effect is more likely to occur when interviewers are unaware of job requirements. Therefore, applicants tend to be afraid of being honest for fear that one negative response will cost them their job opportunities. This lack of honesty can be particularly noticeable during the interview. In this case, the face-to-face aspect of the meeting raises the probability that an applicant will respond in a way that makes them seem more attractive to the interviewer.

To reduce the potential effect of the negative-information bias, it’s important to reduce social pressure on the candidate. One way to do this is to use written or computerized interviews. But if you’re thinking about including AI into your evaluation process, read Andreea's article about what you should keep in mind about recruitment and technology.

6. Applicant Appearance – Physically attractive candidates seem to have an advantage in interviews over less attractive ones. As a result, those who dress professionally tend to receive higher interview scores than those who wear casual clothes. Unfortunately, this bias extends to weight as well, with studies showing that overweight applicants are more likely to earn lower scores than leaner peers. Moreover, this is common for both men and women.

7. Nonverbal Cues – The use of appropriate nonverbal cues such as smiling and making appropriate eye contact is highly correlated with interview scores. Other cues that have been shown to affect interview performance are: tone, pitch, speech rate, and pauses. Candidates displaying such behaviours tend to be perceived as more competent. To reduce the impact of nonverbal cues on hiring decisions, consider using structured interviews.

The first step is to conduct a job analysis and collect information about the job in terms of tasks performed, the conditions under which they are performed, and the knowledge, skills, abilities, and other characteristics (KSAOs) needed to perform the tasks. Other characteristics are important to the extent they impact job performance, and may include personality traits, interests, values, etc.

Secondly, determine the best way to measure an applicant’s ability to perform the tasks identified in the job analysis. While some of the KSAOs can be appropriately measured in an interview, others are better assessed through such methods as psychological tests, work samples, assessment centres, references, background checks, or job knowledge tests. In short, not every KSAO can or should be assessed in an interview. Ideally, you should decide on a set of 3-5 key job-related attributes that you’ll focus on during the interview.

The third step is to create questions that tap into the selected KSAOs to find out the extent to which applicants’ attributes match them.

WHAT TYPES OF QUESTIONS CAN YOU USE IN A STRUCTURED INTERVIEW?

CLARIFIERS – Allow the interviewer to clarify information in the CV, cover letter and application. Also, fill in gaps and obtain other necessary information. 

I noticed a 1-year gap between two of your jobs. Could you tell me more about that?

DISQUALIFIERS – Allow the interviewer to ask questions that must be answered in a particular way. Otherwise the applicant is disqualified based on job requirements.

Do you have a valid driver’s license?

SKILL-LEVEL DETERMINERS – Allow the interviewer to ask questions that tap into an interviewee’s level of expertise.

What is your level of proficiency in Java programming?

BEHAVIOURAL QUESTIONS

Behavioural questions focus on previous behaviour rather than future intended behaviour.

Also, applicants are asked to provide specific examples of how they have demonstrated job-related skills in the past.

In general, scores on behavioural questions are better predictors of performance. This is especially true in the case of higher-level positions.

SITUATIONAL QUESTIONS

Situational questions ask an applicant what they would do in a particular situation.

You can use the critical incidents technique to come up with job-related scenarios that future job incumbents may face and rewrite them into questions.

Above all, remember to design them so that candidates can answer them with their current level of knowledge and skills.

How do you assess candidate's answers?

Once you have the questions, the next step is to create a key to score applicants’ answers. The majority of interview responses are scored using one of three methods: 

• Right/Wrong
• Typical-Answer
• Key-Issues Approach. 

For instance, these are detailed further in our evidence based guide to hiring interviews, which you can download for free on The Inner View's LinkedIn page as well.

THE RIGHT/WRONG APPROACH

Some questions, especially disqualifiers and skill-level determiners, can be scored simply on the basis of whether the answer given was correct or incorrect. For example, consider this question as a skill-level determiner for a junior software developer:

What is your level of proficiency in Java programming?

However, if the candidate outlines the following skills for writing code – writing (e.g., correctly implementing a function), refactoring (e.g., adapting and changing code based on detailed instructions), embedding in a larger system (e.g., using I/O channels to input and print simple text or numbers) – they’ll get 1 point for being a basic user.

Consider this question as a disqualifier: Do you have a valid driver’s license? Asking this implies that driving is a job requirement. As such, answering no is the wrong answer here. This will disqualify the candidate from further consideration.

THE TYPICAL ANSWER APPROACH

How to use it? Create a list of all possible answers to each question. Have subject-matter experts (SMEs) rate the relevance of each response on a 5-point scale, with 5 being the most relevant and 1 being the least relevant. Rate the candidate during the interview based on the benchmarks mentioned. Then total their final score at the end.

Though some scoring keys have only one benchmark answer for each point on the scale, research indicates that increasing the number of benchmark answers will greatly increase the scoring reliability. In other words, the consistency of results from one use to another. 

Let’s consider the following behavioural question for assessing analytical thinking:

Thinking about your experience with analysing large data sets, what types of errors have you found most commonly?

A possible scoring key using the typical answer approach could range from 1 (does not identify errors) to 5 (identifies misunderstood relationships between data, misunderstood causes, influences and associations, misinterpreted statistics, misinterpreted observations, biased assertions and coincidences).

THE KEY ISSUES APPROACH

The typical-answer approach offers many possible answers to a question. As a result, chances are applicants will provide answers that could only fit parts of your identified benchmarks. This problem can be corrected with the key-issues approach.

How do we use this method? Firstly, have subject matter experts create a list of key issues (i.e., behaviours) they think should be included in the perfect answer. If needed, assign weights so that the most important issues get more points than the less important issues. Otherwise allocate 1 point. Secondly, rate the candidate based on the issues they mention during the interview. Lastly, sum up their total score at the end.

Let’s consider the following situational question for assessing active listening to clarify information:

You are in charge of a team of 15 call centre operators. Since some clients have expressed dissatisfaction with one of your direct reports, you’ve scheduled a meeting with them to discuss why this has occurred. What is the best way to clarify the situation?

Here’s a possible scoring key using the key issues approach:

• 0 pts - Ask questions to confirm your expectations about their poor results.
• 0 pts - Review their results since the last performance review.
• 2 pts - Analyse the clients’ feedback in relation to their KPIs.
• 5 pts - Ask questions about their interaction with the clients, such as “what”, ”how” etc.

WHAT’S NEXT? CONDUCTING A STRUCTURED INTERVIEW AND CONDUCTING A VIRTUAL INTERVIEW

Research suggests that structured interviews will predict performance when the same trained interviewers assess all applicants for a given position. Also, many organisations were forced to move from face-to-face to virtual interviews (VI) during the pandemic. In short, there’s limited scientific evidence when it comes to the effectiveness and the implications of synchronous virtual interviews.

To find out how to conduct a structured interview and what candidates say about virtual versus face-to-face interviews, we’ve put together key findings in our free to download evidence based guide. 

About The Inner View

Andreea - Cognitive Behavioural Coach and Organisational Consultant - with international experience in coaching and learning and development projects. My purpose is to facilitate transformational experiences for people, teams and organisations in order to reach their goals by using an evidence-based approach.

Cosmin - I/O Psychologist and HR Consultant - with international experience in psychometrics, hr consulting, and organisational diagnosis projects. My purpose is to help companies and individuals alike be aware of their inner workings and balance long-term performance, health and well-being.