Imagine this: Two candidates, A and B, both with impressive skills and the “right” experience, aim to find a consultant role at different companies. Yet, time and time again candidate B is consistently overlooked—not because they are less qualified than candidate A, but because of a hidden flaw in the recruitment tools: adverse impact.
This isn’t a one-off incident; it’s a larger issue that many organisations unknowingly face. Could your company’s recruitment practices be unintentionally shutting out top talent and limiting diversity?
In this article, we’ll explore:
- What adverse impact is and how it affects your organisation.
- A case study highlighting this risk in cognitive ability and personality tests.
- How to identify adverse impact in your hiring process.
- 2 actionable ways to reduce adverse impact and build a fairer hiring system.
A glimpse into the adverse impact of a recruitment tool
Have you ever considered that your hiring process might produce different results for different groups of people?
This could be silently working against your goals for diversity. Adverse impact happens when hiring tools—like assessments or interviews — disproportionately affect certain groups, whether based on race, gender, or ethnicity. These barriers can result in fewer opportunities to progress through the hiring stages and, ultimately, fewer job offers (Biddle, 2017).
For instance, women and people with disabilities often face extra challenges with physical ability tests, while written tests may unfairly disadvantage Black and Hispanic candidates more frequently (Biddle, 2017; Sackett et al., 2001; Neisser et al., 1996). This isn’t just about bias—these issues are often built into the tools themselves. And fixing them? Not as easy as it sounds.
Case study: cognitive ability test and personality test
The problem with (traditional) cognitive ability tests
(Traditional) cognitive ability tests are commonly used in recruitment but are notorious for producing significant differences in scores between racial groups, which can limit workforce diversity (Ng & Sears, 2010).
Research shows that these tests often lead to a 1 standard deviation (SD) difference in general mental ability (GMA) scores between Caucasian Americans and African Americans (Bobko et al., 1999; Schmidt, 2002; Bosco et al., 2015). However, the actual difference in job performance between these groups is much smaller—around 0.50 SD or less (Schmidt, 2002; Roth et al., 2003).
For example, a traditional cognitive test might result in one group scoring an average of 85 out of 100 on a test, while another group scores an average of 100. But when it comes to real job performance, the gap is much smaller—more like 90 versus 95. This shows that while test scores may highlight bigger differences, they don’t always reflect how well people perform in real work situations.
What about personality tests?
Personality tests are often seen as less biased than cognitive tests , but they’re not entirely free from issues (Hough, 2001). Cultural differences can affect how candidates express certain traits.
For example, people from collectivist cultures may downplay personal achievements to fit into the group, while those from individualistic cultures may emphasise their independence. If these cultural differences aren’t accounted for, personality tests could misinterpret candidates’ traits, leading to inaccurate assessments.
Beyond cultural concerns, another important factor to consider is how you select and rank candidates. A top-down approach—where only the highest-ranking candidates move forward—can potentially create adverse impact, especially when only a small number of people would be selected for the role (Hausdorf & Risavy, 2010; Risavy & Hausdorf, 2011 ).
Let’s say you’re hiring 1 person for your engineering department and use a personality test as the first step. If you only move the top 3 candidates to the next round, what about the 4th candidate? There may be little difference between their personality traits and those of the top three, yet they could be unfairly eliminated.
This scenario highlights the importance of carefully considering whether personality tests are suitable for your hiring process, especially when factoring in selection rates and ranking systems.
The stakes for your company
Why does addressing adverse impact matter so much? Using flawed recruitment tools can lead to severe consequences. In the U.S., over 27,000 cases of race discrimination are filed with the Equal Employment Opportunity Commission (EEOC) annually, costing companies more than $145.7 million in settlements in 2023 alone (EEOC, 2023).
Failing to address adverse impact doesn’t just expose your company to legal risks—it limits your ability to attract diverse talent, which is crucial for fostering innovation and staying competitive in a global market. If your recruitment tools unintentionally favour one group over another, you could miss out on highly qualified candidates who would bring diverse perspectives and fresh ideas to your team.
How to detect adverse impact during recruitment?
Detecting adverse impact in your hiring process can be tricky, but there are tools to help. One common method is the four-fifths rule (EEOC, 1978). This rule compares the hiring rates of different groups. If one group’s selection rate is less than 80% of the group with the highest rate, adverse impact might be present (Biddle, 2017; Newman & Lyon, 2009).
To make this clearer, let’s look at an example:
In Case A, out of 100 male applicants, 4 were hired, giving a selection rate of 4%. For females, 5 were hired out of 150 applicants, resulting in a selection rate of 3.33%. The female selection rate is 83.33% of the male selection rate, meaning no adverse impact is present since it’s above 80%.
In Case B, 10 males were hired out of 100 applicants (a selection rate of 10%), while 10 females were hired from 150 applicants (a selection rate of 6.67%). Here, the female selection rate is only 66.7% of the male selection rate, which falls below the 80% threshold, indicating adverse impact against female applicants.
Other statistical tools, like chi-square tests, logistic regression, and differential item functioning analysis, can also help detect these hidden patterns in your hiring process.
3 actionable strategies to reduce adverse impact.
1. Understand your vacancy and use only the necessary predictors
It’s crucial to understand which competencies are needed for the role before deciding on the assessment tools. Conducting a job analysis helps you identify which skills are essential and which are “nice to have” (Van Iddekinge et al., 2023; Ployhart & Holtz, 2008).
This approach can help you avoid using overly broad tests, like those that measure general intelligence, which carry a higher risk of adverse impact. Instead, focus on tests that target the specific competencies required for the role, such as flexibility or problem-solving skills .
On top of that, using more specific tests can reduce the adverse impact inherent in traditional cognitive ability tests. Since these tests rely less on hard knowledge and verbal components, they tend to show lower score differences between racial groups compared to traditional cognitive ability tests (Naglieri & Otero, 2018; Bosco et al., 2015; Naglieri et al., 2015; Burgoyne et al., 2021). This suggests they could offer a more equitable way to assess cognitive abilities across diverse populations.
2. Combine different types of assessments
Another strategy of reducing adverse impact is to try combining different types of assessments. Focusing on one type of test may increase the risk of adverse impact, but combining assessments—such as cognitive and behavioural tests—can provide candidates with more ways to demonstrate their strengths (De Corte et al., 2010; De Corte et al., 2007).
For example, while a cognitive ability test could show a candidate’s potential on problem-solving, a situational judgement test could reveal their ability to utilise their skills in real-life challenges. By using a variety of assessment methods, you can build a more balanced picture of each candidate’s capabilities and reduce the likelihood of unfairly disadvantaging any group.
3. Adjust your selection ratio and utilise a multistage selection system.
Selection ratio plays a critical role in adverse impact (Risavy & Hausdorf, 2011). For example, selecting 1 candidate out of 100 applicants is more likely to lead to adverse impact than selecting 3 candidates from the same pool.
Consider using a funnel approach in your selection process by adding more stages in your recruitment process (De Corte et al., 2006; Finch et al., 2009). By setting a higher selection rate in the early stages of the process—starting with the most critical criteria—then narrowing down with other predictors in later stages, you can reduce the risk of missing out on diverse talent.
Final thoughts
Adverse impact in recruitment reveals how even well-intentioned hiring processes can inadvertently disadvantage certain groups. Without realising it, organisations may limit diversity and miss out on top talent. By becoming aware of these hidden biases and refining selection processes—such as adjusting assessment methods and selection ratios—companies can reduce adverse impact, ensuring fairer hiring and fostering a more inclusive, innovative workforce. Happy Hiring!
References
Biddle, D. (2017). Adverse Impact and Test Validation. Taylor & Francis. https://doi.org/10.4324/9781315263298
Bobko, P., Roth, P. L., & Potosky, D. (1999). Derivation and implications of a meta‐analytic matrix incorporating cognitive ability, alternative predictors, and job performance. Personnel psychology, 52(3), 561-589. https://doi.org/10.1111/j.1744-6570.1999.tb00172.x
Bosco, F., Allen, D. G., & Singh, K. (2015). Executive attention: An alternative perspective on general mental ability, performance, and subgroup differences. Personnel Psychology, 68(4), 859-898. https://doi.org/10.1111/peps.12099
Burgoyne, A. P., Mashburn, C. A., & Engle, R. W. (2021). Reducing adverse impact in high-stakes testing. Intelligence, 87, 101561. https://doi.org/10.1016/j.intell.2021.101561
De Corte, W., Sackett, P., & Lievens, F. (2010). Selecting predictor subsets: Considering validity and adverse impact. International Journal of Selection and Assessment, 18(3), 260-270. https://doi.org/10.1111/j.1468-2389.2010.00509.x
De Corte, W., Lievens, F., & Sackett, P. R. (2007). Combining predictors to achieve optimal trade-offs between selection quality and adverse impact. Journal of Applied Psychology, 92(5), 1380. https://doi.org/10.1037/0021-9010.92.5.1380
De Corte, W., Lievens, F., & Sackett, P. R. (2006). Predicting adverse impact and mean criterion performance in multistage selection. Journal of Applied Psychology, 91(3), 523. https://doi.org/10.1037/0021-9010.91.3.523
Enforcement and litigation statistics. (n.d.). US EEOC. Retrieved October 7, 2024, from https://www.eeoc.gov/data/enforcement-and-litigation-statistics-
Finch, D. M., Edwards, B. D., & Wallace, J. C. (2009). Multistage selection strategies: Simulating the effects on adverse impact and expected performance for various predictor combinations. Journal of applied psychology, 94(2), 318. https://doi.org/10.1037/a0013775
Hausdorf, P. A., & Risavy, S. D. (2010). Decision making using personality assessment: Implications for adverse impact and hiring rates. Applied HRM Research, 12(1), 100.
Hough, L. M., Oswald, F. L., & Ployhart, R. E. (2001). Determinants, detection and amelioration of adverse impact in personnel selection procedures: Issues, evidence and lessons learned. International Journal of Selection and Assessment, 9, 152-194. https://doi.org/10.1111/1468-2389.00171
Naglieri, J. A., & Otero, T. M. (2018). Redefining intelligence with the planning, attention, simultaneous, and successive theory of neurocognitive processes. In D. P. Flanagan & E. M. McDonough (Eds.), Contemporary intellectual assessment: Theories, tests, and issues (4th ed., pp. 195–218). The Guilford Press.
Naglieri, J. A. (2015). Hundred years of intelligence testing: Moving from traditional IQ to second-generation intelligence tests. Handbook of intelligence: Evolutionary theory, historical perspective, and current concepts, 295-316. https://doi.org/10.1007/978-1-4939-1562-0_20
Neisser, U., Boodoo, G., Bouchard Jr, T. J., Boykin, A. W., Brody, N., Ceci, S. J., … & Urbina, S. (1996). Intelligence: knowns and unknowns. American psychologist, 51, 77. https://doi.org/10.1037/0003-066X.51.2.77
Newman, D. A., & Lyon, J. S. (2009). Recruitment efforts to reduce adverse impact: Targeted recruiting for personality, cognitive ability, and diversity. Journal of applied psychology, 94(2), 298. https://doi.org/10.1037/a0013472
Ng, E. S., & Sears, G. J. (2010). The effect of adverse impact in selection practices on organizational diversity: A field study. The International Journal of Human Resource Management, 21(9), 1454-1471. https://doi.org/10.1080/09585192.2010.488448
Ployhart, R. E., & Holtz, B. C. (2008). The diversity–validity dilemma: Strategies for reducing racioethnic and sex subgroup differences and adverse impact in selection. Personnel Psychology, 61(1), 153-172. https://doi.org/10.1111/j.1744-6570.2008.00109.x
Risavy, S. D., & Hausdorf, P. A. (2011). Personality testing in personnel selection: Adverse impact and differential hiring rates. International Journal of Selection and Assessment, 19(1), 18-30. https://doi.org/10.1111/j.1468-2389.2011.00531.x
Roth, P. L., Huffcutt, A. I., & Bobko, P. (2003). Ethnic group differences in measures of job performance: A new meta-analysis. Journal of Applied Psychology, 88(4), 694. https://doi.org/10.1037/0021-9010.88.4.694
Sackett, P. R., Schmitt, N., Ellingson, J. E., & Kabin, M. B. (2001). High-stakes testing in employment, credentialing, and higher education: Prospects in a post-affirmative-action world. American Psychologist, 56, 302. https://doi.org/10.1037/0003-066X.56.4.302
Schmidt, F. L. (2002). The role of general cognitive ability and job performance: Why there cannot be a debate. Human Performance, 15(1-2), 187–211. https://doi.org/10.1207/S15327043HUP1501&02_12
U.S. Equal Employment Opportunity Commission, Civil Service Commission, Department of Labor, and Department of Justice. (1978). Uniform guidelines on employee selection procedures. Federal Register, 43, 38290-38315.
Van Iddekinge, C. H., Lievens, F., & Sackett, P. R. (2023). Personnel selection: A review of ways to maximize validity, diversity, and the applicant experience. Personnel psychology, 76(2), 651-686. https://doi.org/10.1111/peps.12578
