Algorithmic Biases: When AI Inherits Our Prejudices

Algorithmic biases occur when artificial intelligence systems trained on historical data reproduce and amplify patterns of discrimination present in that data. The algorithm doesn’t “invent” prejudice: it detects it as a statistical pattern in the training data and mathematically encodes it as a decision rule.

If a hiring database shows that, historically, 85% of the engineers hired were white men, the algorithm learns that “white man” is a positive predictor of “good engineer.” Not because it is true, but because the data reflects an unequal social structure. AI inherits discrimination as if it were objective information.

In 2016, the news organization ProPublica published an investigation that shook up the debate about AI and justice. They analyzed COMPAS (Correctional Offender Management Profiling for Alternative Sanctions), an algorithm used in U.S. courts to predict the likelihood that defendants would reoffend. Judges used the result to make decisions about bail, sentencing, and parole.

The investigation revealed that COMPAS classified Black defendants as "high risk" almost twice as often as white defendants, even when controlling for objective criminal factors (type of crime, history, age). False positives (classifying someone as high risk who did not reoffend) were disproportionately higher for Black defendants: 44.9% compared with 23.5% for white defendants.

In 2018, Joy Buolamwini and Timnit Gebru published the study “Gender Shades” through the MIT Media Lab. They tested three commercial facial recognition systems (Microsoft, IBM, Face++) and found dramatic disparities: the error rate for dark-skinned women reached 34.7%, while for light-skinned men it was below 1%.

The cause was straightforward: the datasets used to train these systems were composed mostly of white male faces. The algorithm had simply seen few Black women's faces during training, so it didn't know how to recognize them. It wasn't malicious: it was structural negligence in data curation.

In 2018, Reuters revealed that Amazon had developed and then scrapped an AI tool for résumé screening. Trained on the previous 10 years of hiring data, the system learned to systematically penalize résumés with indicators of female gender: the name of a women’s college, participation in women’s clubs, or even the word “women’s” on the résumé.

The logic was simple and devastating: since the vast majority of engineers hired by Amazon historically were men, the algorithm learned that "being a man" was a positive predictor of hiring. Not because men were better engineers, but because the hiring pipeline had historically favored men. The AI optimized the existing bias.

The naive solution seems obvious: just remove protected variables (race, gender, religion) from the training data. This approach, called “fairness through unawareness,” systematically fails. The reason is that the entire social structure is correlated: ZIP code correlates with race (because of residential segregation), school type correlates with social class, and language patterns correlate with ethnicity.

These correlated variables function as proxies: they replace the protected variable without the developer realizing it. A credit algorithm that uses ZIP code as a variable is, in practice, using race as a predictor, because decades of redlining (denying mortgage credit to Black neighborhoods) created an almost perfect correspondence between ZIP code and racial composition.

The field of fairness in machine learning proposes multiple mathematical definitions of "algorithmic fairness." The three main ones are demographic parity (the same positive decision rate for all groups), equal opportunity (the same true positive rate), and calibration (the same meaning of the score for all groups).

In 2017, Alexandra Chouldechova demonstrated mathematically that when the base rate of an event differs between groups (e.g., different recidivism rates between racial groups for socioeconomic reasons), it is impossible to satisfy calibration and equal error rates at the same time. This isn’t a technical limitation: it’s an impossibility theorem. Every choice of fairness metric is an ethical choice, not just a technical one.