Hardwired for Disparity: The Hidden Biases in Artificial Intelligence Systems

Artificial intelligence (AI) systems are being rapidly adopted across many industries, from finance and healthcare to criminal justice and recruitment. However, concerns have emerged about hidden biases within these systems that can lead to unfair and harmful outcomes. This comprehensive guide examines the sources of bias in AI, the real-world impacts, and what can be done to create more ethical, fair and inclusive AI.

Introduction

AI systems are designed to automate tasks and make decisions by learning from data. However, the data used to train these systems often reflects and amplifies existing societal biases. As a result, AI can discriminate against certain groups, such as women and minorities, in ways that are difficult to detect.

Left unaddressed, biased AI can have severe consequences, from denying loans and healthcare to justifying over-policing in minority communities. There have already been multiple high-profile cases of real-world harm caused by biased algorithms.

It’s clear that while AI promises many benefits, there are also risks that must be carefully managed. Understanding the sources of bias and developing strategies to mitigate it is crucial for creating AI that lives up to ideals of fairness, accountability and transparency.

This in-depth guide examines where bias in AI comes from, how it leads to unjust outcomes, and what developers, companies and policymakers can do to ensure AI prioritizes inclusion and equity. Read on to learn how we can prevent AI from further amplifying discrimination.

Where Does Bias in AI Come From?

AI systems rely on data to learn. Unfortunately, the data used to train AI often contains human prejudices and structural inequalities. Here are three key sources of bias in AI systems:

1. Biased Training Data

One major source of bias is the training data itself. AI learns from patterns in data, so if the data reflects existing societal biases, those get picked up and amplified. For example, a facial recognition system mainly trained on white male faces will be less accurate at identifying women and minorities.

Humans are inherently biased by their background and experiences. If developers are not careful, their own unconscious biases can creep into the data selection and labeling process, skewing the training data.

Historical discrimination and exclusion also means the available data is disproportionately male and white. Relying solely on this imbalanced data bakes unfair bias into AI systems.

2. Poorly Selected Algorithms

The choice of algorithms used in AI can also introduce bias. Algorithms are designed to maximize accuracy based on the patterns observed in data.

However, maximizing accuracy does not guarantee fairness or prevent discrimination. Algorithm design choices that ignore impacts on different demographic groups often lead to biased outcomes.

For example, natural language processing algorithms trained only on “standard” English tend to perform worse for minorities. Predictive policing algorithms trained on biased crime data can lead to over-policing of minority neighborhoods.

3. Lack of Diversity in AI Development

The lack of diversity among AI developers and leaders is another key factor. In 2018, only 22% of AI researchers were women. The field is also dominated by white and Asian people, with black and Hispanic minorities severely underrepresented.

Homogenous teams build homogenous AI. Without diverse voices involved in the design process, many blindspots occur that lead to biased systems. Having more representative teams helps reduce unfair impacts on marginalized groups.

In summary, bias in AI stems from imbalanced training data, algorithm design choices that ignore fairness, and lack of diversity among developers. Next, let’s look at how these biases translate into real-world discrimination.

Real-World Impacts of Biased AI

Biased AI systems are already perpetuating injustice and creating problems across many domains. Here are some examples of real-world discrimination caused by flawed algorithms:

Discriminatory Hiring Tools

In 2018, Amazon scrapped an AI recruiting tool after discovering it disadvantaged women candidates. The algorithm learned to downgrade resumes containing words like “women’s” that revealed the applicant’s gender.

Other hiring algorithms have been found to filter out minorities due to biased training data. The “ideal candidate” determined by these AIs reflects historical imbalances in tech.

Racial Discrimination in Healthcare

A 2019 study found an AI system widely used in US hospitals to guide healthcare decisions was less likely to refer black patients to extra care programs. This could lead to blacks receiving inferior treatment.

Other research revealed image recognition AIs are less accurate at classifying skin lesions on minorities, limiting diagnostic ability. The impacts on patient outcomes could be severe.

Gender and Racial Bias in Facial Analysis

Studies of major facial analysis systems found they had error rates of up to 35% higher for dark-skinned females compared to light-skinned males. Such biases could enable racial and gender profiling in law enforcement applications.

Discriminatory Credit Decisions

Algorithms used to determine credit eligibility have often been found to discriminate unfairly. In one case, an algorithm denied credit to applicants from minority zip codes at twice the rate of white zip codes, even when income was equivalent.

Biased Predictive Policing Systems

PredPol and similar data-driven policing tools have been found to disproportionately target minority neighborhoods. Critics argue they reinforce existing police bias rather than accurately predicting crime.

In summary, without fair design, AI can amplify discrimination, deny opportunities, and cause real harm – particularly for marginalized groups. The impacts are difficult to audit making it hard to prove algorithms are at fault. Next we’ll look at solutions.

How to Reduce Bias and Increase Fairness in AI

While the challenges seem daunting, there are concrete steps AI developers and companies can take to mitigate unfair bias and increase transparency. Here are six evidence-based strategies:

1. Ensure More Representative Training Data

Address imbalanced training data by sourcing higher quality, more diverse datasets that better represent minorities. This improves accuracy across all demographics.

Actively label more data for underrepresented groups. Techniques like data augmentation can also synthetically expand minority samples.

2. Perform Rigorous Bias Auditing

Test algorithms extensively during development using bias detection techniques and auditing procedures. This identifies unfair impacts that can then be addressed.

Evaluate performance across different demographic groups – e.g. gender, ethnicity, age. Achieve statistically equal rates of false positives/negatives.

3. Utilize Techniques to Mitigate Bias

Apply bias mitigation methods to algorithms, such as techniques that ensure equalized odds for groups or remove information that enables discrimination (e.g. name, gender).

Adjust model hyperparameters and utilize techniques like adversarial debiasing to optimize fairness criteria while maintaining accuracy.

4. Increase Diversity in AI Teams

Prioritize recruiting women, minorities and people from disadvantaged backgrounds for AI roles. Varied perspectives lead to greater awareness of potential biases.

Enable staff from less represented groups to participate meaningfully in the development and auditing of AI systems.

5. Implement Third Party Audits

Have external auditors and technical committees review algorithms before deployment to identify bias risks independently. Continuously monitor for unfair impacts.

6. Make Ethical AI a Leadership Priority

Foster an ethical AI culture. Leaders should make fairness and accountability core values embedded at every stage of design and deployment.

Involve legal/compliance teams and ethics advisers in the development process. Make inclusivity a key performance indicator.

6 Key Questions about Bias in AI

FAQ

There are still many questions surrounding the topic of unfair bias in artificial intelligence systems. Here we answer 6 key questions about AI and algorithmic bias:

What are some examples of biased algorithms causing problems in the real world?

Some examples include:

• Recruitment algorithms that discriminate against women applicants
• Facial analysis AIs that are less accurate for minorities
• Credit approval algorithms with higher denial rates for minority groups
• Healthcare AIs that provide inferior medical recommendations for black patients compared to whites
• Predictive policing tools that disproportionately target minority neighborhoods

How can bias get baked into AI algorithms during the machine learning process?

Bias can enter algorithms in several ways:

• Training data reflects existing societal biases and lack of representation
• Data labeling and selection done by biased humans
• Choice of algorithms that maximize overall accuracy but allow bias against subgroups
• Homogenous AI developer teams overlooking potential harms to disadvantaged groups

Together these can result in unfair discrimination being embedded into the algorithm.

What are the main causes of biased data being used to train AI systems?

Biased training data usually stems from:

• Lack of diversity – data consists mostly of majority groups
• Historical discrimination that underrepresents minorities
• Data collected reflecting existing structural inequalities
• Humans labeling data based on their unconscious biases
• Insufficient labeling and poor sampling of minority groups

What is algorithmic fairness and how can it be improved in AI systems?

Algorithmic fairness involves ensuring AI systems do not discriminate or have unintended impacts on disadvantaged groups. Some ways to improve fairness are:

• Testing algorithms for equivalent accuracy across different demographics
• Techniques that balance accuracy vs fairness, such as equalized odds
• Removing information enabling potentially discriminatory decisions
• Inclusive data collection and augmentation of minority samples
• Ongoing bias testing and third-party auditing

How does having more diversity in AI development teams help reduce bias?

More diverse teams help catch biases and unfair impacts earlier in development because:

• Women and minorities draw from lived experiences regarding potential for discrimination
• Broader range of viewpoints to provide perspective
• Increased awareness of challenges faced by disadvantaged groups
• Reduced homogeneity and groupthink

Diverse teams are key for building AI that is fair for all groups.

What steps are technology companies taking to address algorithmic bias?

Many tech firms now recognize the reputational and legal risks of biased algorithms. Some positive steps being taken include:

• Investing more in bias testing e.g. Microsoft’s Fairlearn toolkit
• Releasing tools for detecting bias e.g. IBM’s AI Fairness 360
• Forming independent committees to review algorithms
• Making ethical AI principles, including fairness, a priority
• Additional staff training and auditing around recognizing bias
• Changes in recruitment and procurement to get more diverse data

However, more work remains to fully address the issue across the industry.

The Bottom Line

Algorithmic bias is an increasingly critical issue as AI systems are deployed more widely. Biases within these AIs can cause very real harm through discrimination, lost opportunities and widening inequality.

However, with greater awareness and the right strategies, we can prevent AI from unjustly amplifying existing societal prejudices and structural disadvantages. By taking proactive steps to detect, measure and mitigate bias, we can get closer to realizing the many benefits of AI while also upholding ideals of fairness, accountability and transparency.

The path forward requires acknowledging how bias creeps in, making diversity a priority, extensive auditing and utilizing bias-mitigation techniques. There is also an urgent need for clearer regulation and accountability around biased algorithms.

Creating unbiased AI won’t be easy – but with diligence and the right vision, the machine learning technologies powering our future can work for everyone, not just the privileged few. The decisions we make today will determine whether AI removes barriers or erects new ones. The choice is ours.