Sexist AI: Amazon ditches recruitment tool that turned out to be anti-women
It was supposed to make finding the right person for the job easier. However, an AI tool developed by Amazon to sift through potential hires has been dropped by the firm after developers found it was biased against picking women.
From pricing items to warehouse coordination, automation has been a key part of Amazon’s rise to e-commerce domination. And since 2014, its developers have been creating hiring programs aimed at making the selection of top talent as easy and as automated as possible.
“Everyone wanted this holy grail,” one of the anonymous sources told Reuters about the ambitions for the software.
“They literally wanted it to be an engine where I’m going to give you 100 resumes, it will spit out the top five, and we’ll hire those.”
However, a leak by several of those familiar with the program give an insight into some of the mishaps in the AI-based hiring software’s development, and how it taught itself to penalize women… for being women.
It was in 2015 that human recruiters first noticed discrepancies with the tool, when it seemingly marked down female candidates for roles in the male-dominated spheres of software development and other technical roles at the firm.
When the engine came across words like “women’s” on a resume, or if a candidate graduated from an all-women’s college, it unfairly penalized female candidates from selection, the sources said.
Investigations into the cause of the gender imbalance found that the data which fed the algorithm was based on ten years of resumes sent to the company. The vast majority of which were submitted by men.
The algorithm in turn learned to dismiss female candidates as a negative leading to its sexist scoring system.
Edits were made by programmers to make the engine neutral to these particular terms, however, there was no certainty that it wouldn't develop other ways to discriminate in future.
READ MORE: Racist & sexist AI bots could deny you job, insurance & loans – tech experts
Dejected executives eventually scrapped the team in 2017 after losing hope in the project. An Amazon spokesperson told RT that the project never made it out of the trial phase. In addition to its apparent bias, the software "never returned storng candidates for the roles." Now, a “much-watered down version” is instead used for minor HR tasks such as sorting out duplicate applicants from its databases.
Amazon’s sexist algorithms isn’t the first time AI has landed tech firms in hot water. Last month Facebook got flack after it was discovered that women users were prevented from seeing job advertisements in traditionally male-dominated industries.
In May 2016, a report found that a US court that used automated software to provide risk assessments was biased against black prisoners, recording them as twice as likely to reoffend as their white counterparts.
Think your friends would be interested? Share this story!
