Amani Carter L’22 develops new study on Unmasking Coded Bias
The mission of the Lab is to identify a new generation of biases and “stereotype threat” in AI and help provide context and nuance to the conversation to mitigate those biases. Unmasking Coded Bias highlights AI and stereotype threat in a new generation of Black students and professionals. It questions how AI can reify and reconstruct bias based on our gender, age, race, and class and creates a new toolbox to be shared with industry leaders and policymakers.
“What we find in Amani’s field work is that even when the respondents may not yet have direct experience of bias in AI, the threat itself or what the social psychologist Claude Steele has named “stereotype threat” can profoundly affect our use of AI, threatening to undermine performance and participation, causing both emotional and intellectual reactions affecting our career choices,” said Professor Rangita de Silva de Alwis. “Amani’s work helps us recognize these algorithmic threats shared among a new generation of professionals and points us in the direction of new mitigation tools needed to address these threats.”
To assess Black professionals’ and students’ perceptions, Carter surveyed eighty-seven Black professionals and students coupled with an analysis of three-hundred and sixty online professional profiles with the goal of understanding how AI-powered platforms reflect, recreate, and reinforce anti-Black bias.
As part of some of Carter’s findings, she reported in an analysis of job board recommendations of those surveyed, 40% of respondents noted that they had experienced recommendations based upon their identities, rather than their qualifications. Moreover, 30% also noted that the job alerts they had received were below their current skill level.
The report also shows almost two-thirds (63%) of respondents noted that academic recommendations made by the platforms were lower than their current academic achievements.
Unmasking Coded Bias recommends the kind of inquiry that hiring platforms, employers, programmers, and designers of AI should be undertaking and challenges them to broaden the conversation to include the marginalized communities impacted by this technology. Carter concludes to meaningfully transform our world, AI is unfinished until it works for all and must be built in the context of equity and inclusion.
The AI and Bias Lab, and the “Unmasking Coded Bias” report is featured in a Thompson Reuters article on AI and Bias. Bianca Nachmani L’22 designed the Elephant in AI report.