Lessons in Bias for Machine Learning
Algorithms are only as good as their human instructors
Over the past century “bias” expressed a widening array of social and psychological conditions and now factors into every aspect of our daily lives. Nothing is off limit, including artificial intelligence.
In his groundbreaking 1976 book Computer Power and Human Reason, AI pioneer Joseph Weizenbaum highlighted how bias can stem from both the data used in programs and the way those programs are coded.
Software consists of rules crafted by humans, embodying specific ways to tackle problems. The programmer’s assumptions and biases are built in, while the data fed into machines mirrors human decision-making processes—further shaping outcomes.
Put differently, technology is not neutral—it reflects the biases and quirks of its creators.
With the rise of large language models (LLMs), concerns about AI as biased, harmful, or even dangerous have grown louder. Erroneous algorithms can reinforce discrimination and lead to faulty decisions. Further, unequal access to AI threatens to worsen existing inequalities, expanding the gap between those who benefit from advanced technology and those who don’t.
Still, AI also has the potential to reduce unfairness. As technology subsumes a larger part of our lives, the task is to design systems to promote justice rather than reinforce bias. To realize this vision though, it may take no less than reaching agreement on the meaning of ‘fair’—and that’s the biggest challenge of all.
As awareness of AI’s inner workings and possibility for bias grows, the number of high-profile instances of algorithmic bias is increasing.
A 2023 Bloomberg article titled “Humans Are Biased. Generative AI is Even Worse” describes a test that generated than 5,000 AI images. The findings were problematic:
“The world according to Stable Diffusion (a deep learning, text-to-image model) is run by white male CEOs. Women are rarely doctors, lawyers, or judges. Men with dark skin commit crimes, while women with dark skin flip burgers.”
Image generation is only one of many examples of bias in AI in a wide range of settings. Here are a few more of the biggest offenders:
Healthcare—Underrepresented data of women or minority groups can skew predictive AI algorithms.
Applicant tracking systems— Issues with natural language processing algorithms can produce biased results within applicant tracking systems.
Online advertising—Search engine ad algorithms can reinforce job role gender bias and produce biased results within applicant tracking systems.
Predictive policing tools—AI-powered predictive policing tools used by some organizations in the criminal justice system often display biases when identifying areas where crime is likely to occur.
One notable situation is the lawsuit by The U.S. Department of Justice against RealPage in August. The DOJ alleges the company facilitated collusion among landlords to inflate rents across millions of units, violating antitrust laws. RealPage’s algorithmic pricing system analyzes vast amounts of rental market data to recommend rent increases for lease renewals or newly vacated apartments.
A DOJ victory could bolster private lawsuits, allowing plaintiffs to benefit from the government’s findings. While this case could set a precedent for applying antitrust laws to tech-driven pricing strategies, proving a conspiracy is never simple.
Psychologist Rhoda Au, PhD, a professor of anatomy and neurobiology, nicely encapsulates the complexity and nuances in play wherever algorithmic bias is a potential factor, explaining that:
“There’s a lot of pushback against AI because it can promote bias, but humans have been promoting biases for a really long time. We can’t just be dismissive and say: ‘AI is good’ or ‘AI is bad.’ We need to embrace its complexity and understand that it’s going to be both.”
Where do biases in AI systems come from? Researchers identify the key sources, each impacting the fairness of machine learning in distinct ways:
Measurement bias occurs when facial recognition systems are trained on skewed datasets, leading to higher misidentification rates for underrepresented groups.
Representation bias surfaces in datasets that lack diversity, such as job recommendation algorithms that perpetuate historical gender imbalances in hiring.
Sampling bias distorts results when models, like those for creditworthiness, rely on data from only urban populations, failing to accurately assess rural individuals.
Aggregation bias oversimplifies demographic needs, for instance, medical data that overlooks health issues specific to the elderly.
Linking bias introduces errors by incorporating irrelevant data, like social media activity, into models such as credit scoring.
As cited earlier, misdirected algorithms can have real consequences, especially when applied to critical areas such as healthcare, criminal justice, and personal finance.
Faced with these circumstances, what can we do to reduce the negative effects from algorithmic biases?
The ‘Goldilocks Principle’
In his new book “Why Machines Learn: The Elegant Math Behind Modern AI,” Anil Ananthaswamy explains why the question of LLM’s ability to reason is unresolved:
“Some [researchers] argue that this is still nothing more than sophisticated pattern matching. Others see glimmers of an ability to reason and even model the outside world. Who is right? We don’t know, and theorists are straining to make mathematical sense of all this.”
Ananthaswarmy then describes the two competing forces at work in LLMs, which helps us to understand what’s happening under the hood:
“One is called bias: The simpler the model, the greater the bias. The other is called variance: The more complex the model, the greater the variance.
High bias (i.e., the simpler models) leads to underfitting, a higher risk of training error, and a higher risk of test error, whereas high variance (i.e., more complex models) leads to overfitting, a lower risk of training error, and a higher risk of test error.
The job of an ML engineer is to find the sweet spot.”
In machine learning, the number of tunable knobs (i.e., parameters) serves as a proxy for a model's complexity or capacity. Ananthaswarmy recommends that a model should strike a balance—with an optimal number of parameters:
Too few, and the model is overly simplistic, suffering from high bias, which prevents it from grasping the intricate patterns within the training data.
Too many parameters, and the model becomes too complex and learns the patterns in the data with such fine granularity that it fails to generalize to unseen data.
ML expert Mikhail Belkin aptly refers to this philosophy as the Goldilocks principle: “Not too hot, not too cold.”
The truth is machine learning is only as good as the people behind it. We depend on experts to choose the right data, tweak the algorithms, and make sense of the results, ensuring the model works accurately and reliably. However, without competent guidance even the smartest AI can miss the mark. That’s why skilled human oversight is so vital for success.
And there’s another important thing to consider: the challenge of minimizing algorithmic bias is only part of the equation for achieving fairness; even more ground demands to be excavated. According to Harvard researchers:
“Beneath the surface “we [still] need to understand how AI shapes the supply and demand for goods and services in ways that perpetuate and even create inequality…
To usher in an era where the adoption of AI acts as a lifting and equalizing force, collaboration between stakeholders will be key.”
Now, that’s fertile territory for future research projects, and affirms what Silicon Valley elites like to say: Yes, we are only in the early innings.
Octopus Brain
The Cheetos Effect
In chaos theory, the Butterfly Effect describes how even tiny changes in a system can lead to big differences down the road. It’s the idea that a small event—like a butterfly flapping its wings—is a part of the initial conditions of an interconnected complex web. One set of conditions leads to a tornado, while the other set of conditions doesn't.
This fascinating concept is linked to the work of mathematician and meteorologist Edward Norton Lorenz. He noticed that minor disturbances could dramatically alter the outcome of complex systems, such as weather patterns. Lorenz introduced this idea in his 1993 book “The Essence of Chaos”, defining the Butterfly Effect as:
“The phenomenon that a small alteration in the state of a dynamical system will cause subsequent states to differ greatly from the states that would have followed without the alteration.”
While humans are too often unaware of the consequences stemming from incidental actions, a recent event illustrates the value of dialing up the cosmic consciousness a bit more in our daily lives.
This news story on a discarded bag of Cheetos left in a cave in the Carlsbad Caverns in New Mexico captures our attention. According to the US National Park Service:
“The processed corn, softened by the humidity of the cave, formed the perfect environment to host microbial life and fungi. Cave crickets, mites, spiders, and flies soon organize into a temporary food web, dispersing the nutrients to the surrounding cave and formations. Molds spread higher up the nearby surfaces, fruit die and stink. And the cycle continues…
Some members of this fleeting ecosystem are cave-dwellers, but many of the microbial life and molds are not. At the scale of human perspective, a spilled snack bag may seem trivial, but to the life of the cave it can be world changing. Great or small we all leave an impact wherever we go.”
One snack bag can be harmful, but there are plenty of actions that lead to disasters on a vastly larger scale.
We find that 85% of wildfires in the United States are caused by human activity. Careless campfires, unsupervised debris burning, and discarded cigarettes are all frequent culprits, while arson adds a sinister edge to the tally. This places enormous strain on fire management resources, and with wildfires becoming ever more destructive, it’s urgent to address human negligence.
The situation in our oceans is even worse, with 5.25 trillion pieces of plastic debris invading marine life. Of that mass, 269,000 tons float on the surface, while some four billion plastic microfibers per square kilometer litter the deep sea. Every year more scientific studies show that microplastics cause wide-ranging negative effects on sea and land animals.
The Cheetos Effect also applies to the massive consumption of junk food. Representing only a tiny fraction of the $50 billion-per year revenues in the U.S. snack food market, Cheetos (and over-processed stuff in general) has a detrimental effect on the human species, too. So choose your calories wisely.
Disclaimer: Content provided in this newsletter is intended to be used for informational purposes only and does not constitute investment advice or any other kind of advice.