Candidate Assessments: The Good, The Bad, The 1%-ers?

"We only hire the top 1% of talent..." the posting states. The first step in their selection process: complete an online assessment and receive a ridiculously high score. These types of 1% companies claim their approach leads to an organization of "rockstars" that drive exceptional business results. Could this actually be a means of discrimination that cultivates a non-diverse, group think culture?

Before we explore that question, let's discuss the various types of assessments commonly used in hiring:

Assessment Types

• Cognitive Assessments (tests that measure sustained attention, reading comprehension, problem-solving, numerical reasoning, verbal reasoning, spatial ability, logical reasoning, learning agility, perceptual speed and/or accuracy) - These are the most commonly used pre-employment assessment because research has correlated "general cognitive ability" with success across job types, levels and industries.

• Emotional Intelligence Assessments (unlike free self-report EQ tests you can find online, EI ability tests used in hiring are typically skill-based and measure self-awareness, self-management, social awareness and relationship management).
• Personality Assessments (tests that measure character traits across settings, such as Myers-Briggs Type Indicator, Caliper Profile, 16 Personality Factor Questionnaire, SHL Occupational Personality Questionnaire, etc.)
• Integrity Assessments (many personality assessments measure integrity but there are also overt integrity assessments that explore absenteeism, theft, impulsivity, honesty and past deviant behavior.)
• Skills Tests (measurements of specific job related skills, including typing, language fluency, clerical skills, technology skills and more).
• Physical Ability Tests (tests that evaluate physical strength and capability to perform essential physical job tasks, these include tests of stamina, dynamic strength, static strength, trunk strength, explosive strength, extent flexibility and more).
• Project-Based Evaluations (on-the-job type assessments that include work samples, presentations, group exercises, role playing, case studies, inbox exercises and more.)

There are also pre-employment screens such as drug tests, background checks, education verification and reference checks that we will leave beyond the scope of this discussion.

Assessment Prevalence

According to Harvard Business Review, about 76% of organizations with 100 or more employees use assessments in their hiring process.

Talent Board's Candidate Experience Research report shared that 82% of companies are using automated assessments.

CompTIA reported that 80% of the corporate officials they polled said they expect Artificial Intelligence (AI) to have a significant impact on HR and hiring this year.

With technology and testing controlling access to many opportunities, it's critical to understand whether these selection methods are fair, valid and necessary.

Assessment Validity

How does an assessment get validated?

You may be surprised to learn that companies don't have to conduct a formal validation study to prove that their tool is predictive of job success. According to Hire Success, "employers are not required, even when defending standardized or objective tests, to introduce formal validation studies showing that particular criteria predict actual on-the-job-performance." As Criteria Corp explains, a test has construct validity if it measures what it is supposed to, content validity if the subject matter relates to the capabilities and skills needed in a job and criterion validity if test scores correlate with desired business metrics. Cognitive aptitude tests, for example, have been determined to be valid across a wide range of job types. This leaves a lot of grey area for employers to play around in and can enable discrimination.

One example of this, I took an assessment for a Career Coach job that was supposed to measure cognitive ability. While the test contained many questions that would clearly correlate with job performance in this field, a significant amount of the questions involved doing math by-hand. As a person who has worked as a Career Coach for nearly 7 years now, I can assure you that it has never been necessary to do long division or complex multiplication by-hand in this job. It isn't even advisable to do so, as it would risk providing an incorrect calculation. This company is able to continue using a tool that doesn't align well with job performance under the umbrella of "cognitive ability".

Another interesting consideration of test validity is that candidates who practice an assessment can significantly increase their score. After taking a practice assessment, I was able to increase my cognitive ability score by about 12%. This could make the difference between a candidate being screened in or out. Sites like JobTestPrep offer practice versions of company-specific assessments (Korn Ferry's Leadership Assessment, Kenexa's Prove It Accounting Tests, Cubiks Logiks Numerical Test, etc.) for a fee ($79-$99 in most cases).

My question is, if you can pay to practice and increase your score on these assessments, are they really measuring skill and potential job performance, or are they measuring your job search budget, ability to access the Internet and awareness of these options?

Cognitive ability tests aren't the only ones to question. EI researchers Matthews, Roberts & Zeidner concluded that EI may not be able to be measured at all and that neither performance-based or self-report measures of EI meet the criteria for "construct validity".

Forbes shared that "many [personality] assessments on the market can't be shown to have either sufficient reliability or validity".

If a large percentage of assessments used in hiring today do not meet the threshold for validity, are they at least fair?

Assessment Fairness

As we raise awareness and passion for diversity, equity and inclusion in the workplace, how well are companies extending this effort to their hiring process? Consider the following:

• A 2020 study published by the NIH, found that 25% to 40% of all U.S. students suffer from test anxiety. A second study found that test anxiety negatively impacts scores as can overall stress levels (which could be caused by community violence, poverty, family instability, etc.).
• Integrity assessments that ask about past criminal behavior are intrusive, can potentially be violating federal anti-discrimination laws (since incarceration rates are higher among protected groups) and certainly are overtly discriminating against ex-convicts (who may be living and behaving very differently today).
• Physical ability tests are inherently discriminatory against candidates with physical disabilities. Could there be a reasonable accommodation that makes physical tasks feasible for a physically disabled person? If so, should physical ability tests even be used?
• Many cognitive ability assessments have been found to have an adverse impact on protected groups. Non-minority groups often score one standard deviation above minority groups and male candidates often score higher than female candidates in certain sections (like mathematical skills).

Obviously, there can be valid concerns about fairness when assessments are used.

Assessment Necessity

The reason so many companies choose to incorporate assessments into their hiring process is that data shows a stronger correlation for predicting performance with assessments in the mix (see graphics).

Interviews are evaluations conducted by humans, who often have implicit and/or overt biases that are influencing their selection decisions. Tools like assessments and AI can provide the opportunity to reduce bias, so long as they are developed appropriately.

I asked a few hiring experts to weigh-in on the necessity of assessments.

Shelley Piedmont, an experienced Career Coach and former HR executive, shares:

"Assessments provide another data point the hiring team can use to determine whether the candidate is a fit for a job... they may also help job seekers making a career pivot, or who have less experience but otherwise have the skills, abilities, or personality that indicate better success in the role."

Kelli Hrivnak, the President and Principal Recruiter for Knak Digital, also points out:

"Neurodiverse candidates may find social, face-to-face interviews stressful. An assessment or test eliminates the social interaction and allows them to focus on their output."

Dan Roth, a Technical Recruiter for a FAANG company and a global Recruitment Consultant, says:

"I think assessments play a role in reducing the number of interviews and to an extent they are necessary, especially when there are high volume, high competition roles... The biggest issue I see is that a lot of assessments are standardized and not specific to divisions... [which] negates a lot of the value."

Additional Thoughts

• Live Assessments - Erica Reckamp, an Executive Resume Writer and Job Search Consultant, addresses the benefits of choosing a LIVE assessment, when given the option, in this post. She mentions the ability to showcase your problem-solving process, collaborative nature and to earn "partial credit" for activities where you almost arrived at the right answer. While this option may scare a lot of folks, there are potential advantages to consider.
• Assessment Compensation - Job search can feel like a full-time job and completing assessments takes time. Employers may require work samples or presentations that demand hours of your time. This has elicited questions around whether candidates should be compensated for these activities. I've heard of companies reimbursing candidates for time spent traveling to an interview and/or preparing a requested project. Unfortunately, I've also had clients share with me that they prepared a 30/60/90 day presentation that was later implemented by the company without any recognition or compensation. For this reason, I advise job seekers to conserve their time and efforts for only their best-aligned opportunities.
• Accessibility Allies - My hope is that we will leverage the benefits of assessments in a manner that removes bias and improves equality. One awesome example of this is Amazon's "Applicant-Candidate Accommodation Team (ACAT)". The company has established a group dedicated to helping applicants with reasonable accommodations and a hotline (1-888-470-1688) that is staffed weekdays 6am to 4pm PT. These folks will speak with you to understand your needs and deliver disability and accommodation services throughout the hiring process. Let's hope this idea spreads like wildfire across corporate America!

Takeaways

Assessments can be valid, necessary and fair. However, many are not. Determining or proving assessment discrimination isn't an easy task for a candidate and suggestions of discrimination are likely to burn a bridge.

Remember those elitist (1%'ers) I mentioned in the beginning? I researched how the "rockstars" at a couple of these companies felt about the culture. The average employee review score was 3.2/5 across sites and consistent complaints included "bad culture", "workaholism encouraged", "limited growth opportunities" and "lack of trust". I would also guess that these organizations lack diversity of thought (among other forms of diversity), since their workforce is likely comprised of those with the most natural privilege who have been educated at similar institutions.

So, while there are potential benefits to using assessments in hiring, having one exclusively hold the access keys to your organization may be ill-advised.