When humans teach computers how to behave, the machines have no choice but to learn from us.
That much is clear in a newstudy published Thursday in the journal Science that found artificial intelligence replicates the same gender and racial stereotypes we already struggle to control.
That finding, while not entirely surprising, suggests that AI might accidentally perpetuate bias instead of simply streamlining data analysis and work tasks.
To reveal this troubling dynamic, the researchers used off-the-shelf AI and developed an algorithm to determine how it associated pairs of words. The AI generated by machine learning was based on a recent large-scale crawl of the web that captured the complexity of the English language.
Then the researchers turned to what’s known as an Implicit Association Test, a scientific measure of the unconscious connections people rapidly make between, say, a person’s gender and their career, or a person’s name, race, and likability. No matter how much we insist we’re not racist, sexist or homophobic, years of research using the IAT show that we hold biases, often without realizing it.
In order to see whether the AI associated neutral words with biases, the researchers first used an IAT about whether flowers and insects were pleasant or unpleasant. The AI responded how most people would: flowers were likable, insects not so much.
Then they moved on to IATs related to stereotypes we have of certain groups of people. A previous experiment using resumes of the same quality but featuring either European-American names and African-American names found that people in the former group were twice as likely to get called for an interview. When the researchers conducting this study tried to replicate those results with the same database of names and tested for an association with pleasantness or unpleasantness, the European-American names were viewed more favorably by the AI.
“It was a disturbing finding to see just by names we are able to replicate the stereotypes.”
“It was a disturbing finding to see just by names we are able to replicate the stereotypes,” says Aylin Caliskan, the study’s lead author and a postdoctoral researcher at Princeton University’s Center for Information Technology Policy.
A different study from 2002 found that female names were more associated with family than career words, but that wasn’t the case for male names.
You can probably see where this is going.
The AI once again replicated those results, among others, showing that female words like “woman” and “girl” are more associated than male words with the arts versus mathematics or the sciences.
The findings shed light on a maddening chicken or egg problem: Do humans put their biases into language or do we learn them through language? Caliskan can’t conclusively answer this question yet.
“We are suggesting that instead of trying to remove bias from the machine, [we should] put a human in the loop to help the machine make the right decision,” she says.
That, of course, requires a human who is aware of his or her own tendency to stereotype.
Kate Ratliff, executive director of Project Implicit and an assistant professor in the department of psychology at the University of Florida, says it’s currently unrealistic to try to eradicate biases because there’s no empirical evidence that it’s possible. After all, our language, culture, entertainment, and politics are rife with stereotypes that keep reinforcing the associations we’re trying to reject.
“Maybe you could train people to spot these biases and override them,” says Ratliff, who was not involved in the Science study.
Indeed, that’s what many companies, including Facebook, are attempting to do through employee trainings. And that’s exactly the kind of skill and self-awareness you’d need in a human charged with preventing a computer from stereotyping a stranger.
Those human-machine matches will no doubt make quite the pair.