Originally published at Forbes
Twenty years ago, a U.K. comedy sketch from Little Britain featured a woman dismissing every reasonable customer request with a blunt “Computer says no.” I suspect what was humorous then will soon seem eerily prophetic.
There’s a well-documented problem inherent in artificial intelligence (AI)—biases are inherited from their data, developers and users. Training data may be skewed, amplifying societal inequalities. Programmers and users might unintentionally embed biases into algorithms simply because they’re human. And unchecked, this leads to real-world discrimination:
- In the U.S., mortgage refusals are highest among Black people (80% more likely to be rejected than similar white applicants), Native Americans (70%) and Latinos (40%).
- Amazon found its recruitment algorithms favored men due to language usage differences.
- Healthcare models trained predominantly on Caucasian patients resulted in higher misdiagnosis rates for other ethnicities.
While these issues are distressing, a much larger problem looms: the proliferation of anti-competitive, discriminatory and manipulative AI behavior that will interfere with all of our lives.
And why now? Suddenly, the ease of deploying autonomous AI agents is transforming enterprise AI at breakneck speed. Businesses can now implement AI models with specific actions and data within hours. Finance agents assess risk and compliance; banking agents manage client requests and convert insights into sales; health and sales agents can turn questions into diagnoses and sales. The list is virtually endless.
The Hidden Threat
Beyond unintended biases, we now face a new threat—intended bias. Everyone knows businesses routinely segment customers to target the most profitable and use advertising to influence behavior. But AI-driven insights could accelerate and deepen these practices, permanently altering the consumer landscape.
Profiling customers will become more sophisticated by integrating data from partners and social media. It may soon be effortless to score individuals on creditworthiness, honesty, political stance, social standing, sexual orientation, beliefs and financial status. Sound familiar? It should—this de facto social scoring resembles China’s pilot schemes monitoring and regulating citizen and corporate behavior, which they are looking to roll out nationwide.
Now, imagine such a system operated by companies buying and selling your information, potentially acting on it—pricing products based on your ability to pay, denying finance due to algorithmic risk scores linked to your postcode, race or financial standing and manipulating your behavior using subconscious cues targeting your vulnerabilities. Consumers might face issues ranging from AI service denial (AISD)—”computer says no”—to subconscious AI manipulation.
Stopping The Juggernaut
This step change would affect all of us. The real question is how to respond. Globally, we’ve seen two distinct approaches: controlling the process or policing the outcomes.
The European Approach
Europe has opted for top-down control. The European AI Act represents these concerns with a stringent regulatory regime that began on August 1, with a phased rollout over three years. Over 150 companies, including Siemens, Heineken, Renault and Airbus, expressed concerns about potential regulatory overkill. The regulation is indeed strict: Violations can result in fines up to 35 million euros or 7% of turnover, whichever is greater.
The most interesting part is the practices in the EU AI Act in Article 5, which are completely outlawed:
- Subliminal techniques: Methods that distort a person’s ability to make informed decisions.
- Exploitation of vulnerabilities: Targeting individuals or groups based on age, disability, social/economic situation or associations.
- Social evaluation: Creating inferred personality characteristics or social scores.
- Facial recognition: Unauthorized use of facial recognition technologies.
- Biometric categorization: Deducing race, political opinions, trade union membership, religious or philosophical beliefs, sex life and orientation through biometric data.
The US And UK Approach
In contrast, the U.S. and U.K. focus on outcomes. For example, the U.K.’s regulation is minimal—termed a “principles-based,” sector-focused approach. Essentially, the government delegated AI governance to existing regulators, issued guidelines and stepped back, hoping each regulator would manage effectively.
The argument is that strict regulation might disadvantage countries by limiting AI’s benefits, potentially allowing less-regulated nations like China and the U.S. to gain the upper hand. The EU, however, feels differently. Dragoș Tudorache, a member of the European Parliament, said of the legislation that the EU AI Act “safeguards … our citizens and our democracies against any abuses of technology by public authorities.”
That means every one of these practices outlawed in the EU is currently fair game in the U.S. and U.K. It’s not so much light-touch regulation as nearly invisible, and perhaps the best path lies between these two extremes.
Regulatory Challenges
And there’s one further challenge. Even if some of these practices are banned under existing laws in the U.S. and U.K., how will regulators know these practices are occurring? The U.K.’s Financial Conduct Authority (FCA), responsible for overseeing financial services and markets, has issued AI guidelines stating that existing laws cover discriminatory practices, and they will enforce them as necessary. However, AI models often conceal underlying algorithms, making it nearly impossible to determine intention, let alone liability.
There are no easy answers, and no magic solution will equally satisfy businesses and consumers. Perhaps a middle ground is appropriate, given that none of us truly know how this will unfold.
A Real-World Example
Consider the numerous stories (examples 1, 2, 3 and 4) about bias in image generation models—gender bias, race bias and cultural stereotypes. Image generation serves as a useful benchmark to monitor AI progress, both in image quality and in understanding how algorithms interpret the world.
Testing an updated model released last month, I requested an image of a diverse group of protesters outside the U.K. House of Parliament holding a sign saying “Stop AISD now!” It looked impressively realistic, but what came back was best described as a sea of white Europeans.
It seems the computer still says no.
Comments are closed.