When I get push-back that bias in AI is just "the same as human bias", thus nothing worse than we have now, one of the arguments I have made is that while people have different biases, an AI tool presents a singular bias. This argument is based on the work of Virginia Eubanks, who claims this pattern can lead to "automating inequality."
Looks like this problem is playing out with a lawsuit against Workday. Derek Mobley is suing Workday for AI bias. He claims he has been turned down for over 100 jobs that use Workday's AI screening tools. If he were being screened by real people, the screeners would have different biases, providing more possibility for acceptance somewhere. However, the Workday AI has the same bias in every case, making getting past the screener incredibly difficult for some and not others.
Although there are many different AI tools used in education, most are built on ChatGPT, thus bias in ChatGPT is likely present in many of these applications.
Furthermore, these models are necessarily based on past discourse (data); they have no ability to reflect and revise. The biased results will be part of the data future models are trained on, and bias may become, in a sense, "frozen in time," with little possibility for moving beyond and addressing current inequities.
Perhaps this is a slippery slope argument. But we need to be thinking about potential unintended consequences of the technologies we use--conducting pre-mortems on what could happen so we can address possibilities that may become reality.
https://lnkd.in/gbYquV7h
https://lnkd.in/giDBQmxS
