Applying for a job, looking for low-interest mortgage, in need of insurance, passing through an airport security check? Chances are that if you are a white male you’ll be called in for an interview, you’ll secure a great rate for your loan, you’ll be offered an interesting insurance package and you’ll breeze through to the departure gate. Chances are that if you are a dark-skinned female you will have completely different results.
With the rapid advancement of new technologies and computer science, artificial intelligence – the ability of machines to perform tasks normally associated with humans – has become commonplace. Based on complex algorithms, artificial intelligence is meant to make life easier, to facilitate everyday situations and help us save time and energy by automating intricate tasks with step-by-step instructions to a computer. As more and more businesses and institutions rely on artificial intelligence to influence decisions and facilitate behaviours, the need to build fairer and more accurate algorithms is also emerging. Obviously, the humans creating the programmes driving artificial intelligence are not perfect. So, what happens when human imperfections seep into the algorithms to reflect or uncover social issues?
A recent ground-breaking study has revealed that machine algorithms deployed in facial recognition software contain biased data resulting in discrimination against women – especially dark-skinned women. Joy Buolamwini, a young MIT student and founder of the Algorithmic Justice League and advocate against ‘Algorithmic Bias’, was working on a project regarding automated facial detection when she noticed that the computer did not recognize her face. Yet, there was no problem when her lighter-skinned friend tested the programme. Then, when Buolamwini put on a white mask, the computer finally detected a face. After further testing multiple different algorithms with other pictures of herself, the results followed her initial assumption of computer bias. Indeed, two of the facial analysis demos didn’t detect her face at all. “And the other two misgendered me!” she exclaims in a video that explains her “Gender Shades” research project which evaluates and highlights the serious issue of racial and gender discrimination in automated facial analysis and datasets.
After undertaking more in-depth research, tests and analyses, “Gender Shades” reveals disturbing flaws that point to gender and racial discrimination in facial recognition algorithms. The project, testing 1270 photographs of people from varied European and African countries that were then grouped by gender and skin type, used facial recognition programmes from three different well-known companies. Overall, the programmes systematically performed better detecting males over females, even more so for lighter-skinned subjects. Cross-evaluating gender and skin found that black women were the group of people with least successful results: not one of the three programmes was able to detect 80% black female faces. That means that two out of five black women risk being inaccurately classified by artificial intelligence programmes.
In practical terms this can translate into some dubious consequences. What if a company uses inaccurate results to select targets for marketing products or to screen job applications or to apply for a student loan? In America, California Representative Jimmy Gomez, who is member of the Congressional Hispanic Caucus and serves on the House Committee on Oversight and Reform, backed a ban to temporarily stop the use of facial recognition by law enforcement officials. If the software provides inaccurate results, “that could mean even more trouble for black men and boys, who are already more than twice as likely to die during an encounter with police than their white counterparts,” he said. As points out Cathy O’Neil, mathematician, data expert and author of the best-seller Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, “these algorithms are reinforcing discrimination. Models are propping up the lucky and punishing the downtrodden, creating a ‘toxic cocktail for democracy.’”
Common thinking is that human emotions and flaws would not be present in objective machine learning. But as artificial intelligent systems depend on heavily complex datasets that run through programmed algorithms, then, logically, who arranges those datasets and how they are arranged can influence the results. Today, developers of artificial intelligence are largely white men and, whether intentional or not, they will reflect their own values, stereotypes and thinking – the coded gaze as Buolamwini calls it – which are then embedded in these algorithms. And when existing bias is entrenched into code we will have potentially damaging outcomes. O’Neil asserts that everyone should question their fairness, not just computer scientists and coders. “Contrary to popular opinion that algorithms are purely objective, models are opinions embedded in mathematics.”
The risk is that having bias creep into seemingly objective and broadly used decision-making systems can seriously undermine the great strides we have made in empowering women and overcoming racial discrimination in recent decades. “Gender balance in machine learning is crucial to prevent algorithms from perpetuating gender ideologies that disadvantage women,” says Dr Susan Leavy, specialist in data analytics and researcher at the University of Dublin.
As the ‘Gender Shades’ report concludes, inclusive product testing and reporting are necessary if the industry is to create systems that work well for all of humanity. An outcome that has fuelled doubt on the assumed neutrality of Artificial Intelligence and automation as a whole. Microsoft and IBM, both used in the project, responded positively by committing to undertake changes in their facial recognition programmes. To deal with possible sources of bias, IBM for example, has several ongoing projects to address dataset bias in facial analysis – including not only gender and skin type, but also bias related to age groups, different ethnicities, and factors such as pose, illumination, resolution, expression, and decoration. Microsoft, for its part, recognizes that artificial technologies are critical for the industry and are taking action to ensure to improve the accuracy of their facial recognition technology to recognize, understand and remove bias. These are all right needed steps in the right direction if we are to use data responsibly and wisely and not have artificial intelligence work against us.