AI bias fight reveals the limits of UK protections for gig workers

Uber settled with Uber Eats courier Pa Edrissa Manjang after facial recognition checks blocked his access to the app. The case shows how difficult it can be for workers to challenge AI systems, even when equality law and U.K. GDPR rights are available.

WTF Index TERMINATOR
◄ Terminator 3 Idiocracy 1 ►

The story centers on facial recognition controlling a worker's access to income and exposing weak safeguards against biased automated decisions.

AI bias fight reveals the limits of UK protections for gig workers

Uber Eats courier Pa Edrissa Manjang has settled a legal claim against Uber after facial recognition checks stopped him from accessing the app he used for delivery work. The case, reported by the BBC, puts a practical spotlight on a bigger problem: when AI tools affect someone’s income, the route to answers can be slow, technical and expensive.

Manjang, who is Black, had used Uber’s platform since November 2019 to pick up food delivery jobs. After Uber introduced its Real Time ID Check system in the U.K. in April 2020, his account was later suspended and terminated following failed identity checks and what Uber described as “continued mismatches.”

How the Uber Eats ID dispute began

Uber’s Real Time ID Check system requires an account holder to submit a live selfie. That selfie is checked against a photo already held on file to verify identity. The system was based on Microsoft’s facial recognition technology.

According to Manjang’s complaint, Uber suspended and then terminated his account after a failed ID check and a subsequent automated process. He filed legal claims against Uber in October 2021, with support from the Equality and Human Rights Commission, the App Drivers & Couriers Union and Worker Info Exchange.

The dispute did not move quickly. Uber failed in attempts to have Manjang’s claim struck out or to have a deposit ordered for continuing with the case. The EHRC described the matter as still in “preliminary stages” in fall 2023 and said it showed “the complexity of a claim dealing with AI technology.”

A final hearing had been scheduled for 17 days in November 2024. That hearing will not happen because Uber offered, and Manjang accepted, a payment to settle. The financial terms have not been disclosed.

What remains unclear after the settlement

The settlement means the public will not get a full hearing into what happened inside Uber’s systems and review process. Uber did not provide details when asked by TechCrunch, and Microsoft declined comment.

Uber has not publicly accepted that its systems or processes were at fault. The company said courier accounts are not terminated as a result of AI assessments alone and pointed to human review as a safeguard.

“Our Real Time ID check is designed to help keep everyone who uses our app safe, and includes robust human review to make sure that we’re not making decisions about someone’s livelihood in a vacuum, without oversight,” the company said in a statement. “Automated facial verification was not the reason for Mr Manjang’s temporary loss of access to his courier account.”

Yet the available facts still show a serious failure around access to work. Worker Info Exchange obtained all of Manjang’s selfies from Uber through a Subject Access Request under U.K. data protection law. It was then able to show that the photos submitted to the facial recognition check were photos of Manjang himself.

Worker Info Exchange also said Manjang sent many messages to Uber asking for the issue to be fixed and specifically asking for a human to review his submissions. According to WIE, he was repeatedly told: “we were not able to confirm that the provided photos were actually of you and because of continued mismatches, we have made the final decision on ending our partnership with you”.

Why the case matters for AI bias and work

The case is important because it shows how AI bias can become a workplace issue long before it becomes a courtroom issue. A verification tool that looks administrative can still affect whether a worker can earn through a platform.

Manjang’s claim relied on equality law, specifically a discrimination claim under the U.K.’s Equality Act 2006, which lists race as a protected characteristic. Baroness Kishwer Falkner, chairwoman of the EHRC, criticized the fact that he had to bring a legal claim “in order to understand the opaque processes that affected his work.”

She also warned that AI creates challenges for employers, lawyers and regulators, and said the EHRC was particularly concerned that Manjang was not made aware his account was in the process of deactivation or given a clear and effective way to challenge the technology.

That point is central. If an AI system or AI-assisted process blocks access to work, the affected person needs more than a generic support message. They need to know what is happening, how the decision is being made and how to challenge it in a meaningful way.

The role of U.K. data protection law

U.K. data protection law also played a key part in the case. The selfie evidence was obtained using data access rights contained in the U.K. GDPR. Without that evidence, it may have been harder to show that the identity checks had gone wrong.

Those access rights matter because proprietary AI systems are difficult for individuals to challenge from the outside. Platforms hold the records, the workflows and the technical context. A worker who cannot see the data behind the decision starts the dispute at a major disadvantage.

The U.K. GDPR is also supposed to provide safeguards around automated decisions with legal or similarly significant effects. It requires a lawful basis for processing personal data and encourages data protection impact assessments that can examine potential harms before systems are deployed.

But legal rights depend on enforcement. TechCrunch reported that the Information Commissioner’s Office did not step in and investigate complaints against Uber, despite complaints about its ID checks dating back to 2021.

Jon Baines, a senior data protection specialist at Mishcon de Reya, told TechCrunch that “a lack of proper enforcement” by the ICO had undermined legal protections. He said the ICO would have jurisdiction to consider whether the processing was lawful under the U.K. GDPR, including fairness, lawful basis, Article 9 conditions and whether there was a solid Data Protection Impact Assessment before implementation.

Regulation is still catching up

The Manjang case lands during a wider debate over how the U.K. should regulate AI. The government is in the process of diluting data protection law through a post-Brexit data reform bill. It has also confirmed it will not introduce dedicated AI safety legislation at this time.

Instead, the government has said it will rely on existing laws and regulators to cover AI risks in their areas. It also announced £10 million in extra funding for regulators, which could be used to research AI risks and develop tools to examine AI systems.

No timeline was provided for disbursing that funding. Multiple regulators are involved, and TechCrunch noted that if the money were split equally between bodies such as the ICO, the EHRC and the Medicines and Healthcare products Regulatory Agency, among 13 regulators and departments asked to publish an update on their “strategic approach to AI,” each could receive less than £1 million.

For workers facing AI-driven decisions, the lesson is stark. Existing law can help, but only after a long fight. Transparency, human review, access rights and active enforcement all matter because without them, an automated identity check can become a barrier to work with few immediate answers.