Last week, Reuters revealed that Amazon had abandoned an AI based, automated recruitment system because it was sexist. After pouring millions of dollars over three years into its development, the company finally conceded that they could not prevent it from reinforcing gender bias.
Just read that again.
The world’s second-largest technology company could not prevent its own creation from being sexist, despite the best efforts of a
Amazon’s system was designed to take in the vast number of applicants for jobs and rate them with between one and five stars. Human recruiters could then focus their time looking at the applicants with more stars.
This is a classic example of augmentation – AI augmenting human work. Exponents of AI tell assure us that AI won’t replace human beings, merely augment them – be helpful and take the drudge work away.
Computers still cannot think. They can’t anticipate, reason or form an opinion based on truths and values. But they can suck in huge existing data sets, look for patterns and then make predictions from those existing datasets. That’s what AI is doing.
Training AI on our past data is the problem
The problem occurs with our data sets – the vast troves of data that we feed into AI engines to train them. AI systems are only as good as the data sets they learn from.
Feed them biased data, and they learn bias. The old computer term GIGO still applies – Garbage In, Garbage out.
Amazon fed ten years of recruitment and people performance data into the system that they designed. But that was ten years of data where most of the applicants had been men, women were disadvantaged by the inherent bias in Amazon’s previous human based recruitment system. So the Amazon AI system learned that bias and repeated it.
Women, black men, black women, people of colour, people outside the majority all know they face unconscious bias in recruitment, promotion and every step of life. They know they need to be twice as good and work twice as hard to get to the same place.
First generation AI could make that significantly worse, not better. We run a real risk of taking the existing biases in our system and the amplifying them on an industrial scale, codifying them into AI software that even the creators don’t fully understand and, like Amazon, can’t fully control.
If Amazon can’t get this right, what chance do we have?
Amazon built their system in-house and had the skills and expertise to see it wasn’t working and switch it off. But what will happen when we the masses are buying systems off the shelf, and we don’t know how they work or indeed what they do?
Hirevue is one of those systems, and the company counts companies like Unilever and Goldman Sachs amongst its clients. They’ve been investing in AI recruitment for four years and are a leader in the field. Their current advertising promises “Better hiring with AI-driven predictions”.
Their product uses voice and facial recognition to try and find candidates that match existing high-performance staff. Their algorithm then sorts the applicant list in order of rank with the candidates matching high performing team coming top of the list.
“We’re just trying to be more efficient”
When asked whether this will enforce existing biases – companies with a white, male workforce simply matching to more white men, Hirevue is quick to say their job is not to replace recruiters but to make them more efficient.
They state that their system is only suggesting a ranking order and that recruiters should still look at every candidate. But it’s fantasy to think that busy recruiters faced with a long list of applicants will look at any of them other than the ones at the top. And who in their right mind wants to tell their boss they just hired the guy who only got one star?
I asked social recruiting expert, Katrina Collier what she thought :
“The whole recruitment process is biased. It’s a job of rejection, it can mean starting with 100 applications to whittle down to 1, forever looking for reasons to reject someone. Unfortunately, we all have biases, this is what makes us human, so without training to recognise it within ourselves, with or without the tech, there will always be bias.
I feel the biases could be amplified because the human creating the algorithm is biased and lazy recruiters will be too reliant on the technology. Good recruiters quiz, question, delve and, apologies to all the left- brain thinkers, trust their guts!”
We need to take care before this tech gets stuck inside our organisations
I’ve been working in B2B tech for long enough to know that Technology doesn’t have to be great to get embedded, to get stuck in an organisation, it just has to seem to work.
Most organisations run hundreds, maybe thousands of systems, and not all of them are aggressively monitored or checked.
Only a few weeks ago at Reward Gateway, I noticed we were asking for college degree results on roles that previously didn’t need them. When I asked if we’d had a policy change, I was told it was a mistake – accidentally brought in from the default template in the recruiting system. Easy mistakes to make maybe, but
A recruitment system that makes life easier for the recruiter by halving the candidate list can be in place for years before anyone notices it throws out perfectly able candidates with a particular genetic or racial trait. That’s if it gets noticed at all.
Recruitment AI 1.0 is going to be awful – just like any technology 1.0 is. But this time early adopters could hurt people. I really hope for all of us, the adoption is low, and we rapidly accelerate to something advanced and fair.
I’m not holding my breath though.
- In hiring, racial bias is still a problem. But not always for reasons you think – Fortune Magazine, November 2014
- Identifying Discrimination at
Work :The Use of Field Experiments :Devah Pager, Princeton University and Bruce Western, Harvard University. Published in Journal of Social Issues, Vol 68, No 2, 2012 pp 221-237
- How blind auditions help orchestras to eliminate gender bias – The Guardian, Oct 2013
- Google Machine Learning Crash Course adds
lessonon ensuring AI fairness
- The Pros And Cons Of Algorithms In Recruitment
Alternative and opposing viewpoints
As I’m researching an article, I often find opposing viewpoints. It seems wrong to hide them so here they are and you can make your own mind up.
- Top 3 Benefits of AI in Recruitment
- How A.I. Is About To Disrupt Corporate Recruiting
- How AI Is Changing Your Job Hunt
About Glenn Elliott
Glenn Elliott is a technology entrepreneur, investor and advisor, MBA drop-out and recovering CEO with 20 years of experience. His bestselling book Build it: The Rebel Playbook for Employee Engagement is published by Wiley. He writes about people, culture, leadership, technology and the future of work weekly at www.glennelliott.me.
If you liked this article you can make his day by commenting below, sharing it on LinkedIn, Twitter or joining his VIP list by subscribing here.