United Kingdom: Delivery Driver Awarded Settlement for Faulty Facial Recognition
The facial recognition tool in the Uber Eats driver app in the UK repeatedly failed to recognize a black driver, resulting in his being locked out of his account. Now he will receive a financial settlement.
The driver, Pa Edrissa Manjang, sued Uber Eats in 2021, accusing the company of racial discrimination for its use of facial recognition in the app. The lawsuit has now been settled out of court, as the British Equality and Human Rights Commission (EHRC) reported last month. The EHRC, an independent public body, provided funding for Manjang’s suit alongside the App Drivers and Couriers Union.
The amount of the settlement was not disclosed.
Identity Verification in the App
Pa Edrissa Manjang started working as a driver for the food delivery service in Oxfordshire in November 2019. As the BBC reported, he initially had no problems with the app, which assigns deliveries to individual drivers. Over time however it increasingly demanded that he verify his identity when registering for a shift – and to do so by taking a photo of himself. Sometimes he had to take several selfies a day.
Again and again, the app’s facial recognition tool failed to recognize him. Finally, in 2021, he was automatically blocked as a driver after a failed verification. Uber Eats informed him that there were “continued mismatches” in the photos he had sent, the EHRC reported.
The commission criticized the use of so-called artificial intelligence (AI) and automated processes in the case, “particularly how it could be used to permanently suspend a driver’s access to the app, depriving them of an income.” Uber Eats neither informed Mr. Manjang as to why his account had been suspended, nor did the company provide “an effective route to challenge this decision.”
In a statement, Mr. Manjang said: “My case shines a spotlight on the potential problems with the use of AI, particularly for low paid workers in the gig economy who want to understand how decisions are being taken which affect their livelihoods.”
In a separate statement, Mr. Manjang’s lawyer Hannah Wright commented: “This has been a very important case. It is among the first to consider AI and automated decision making in the context of work and the potential for unfairness and discrimination.” The use of AI in the workplace “carries substantial risks,” Wright warned, and “current protections are inadequate.”
According to the EHRC, Mr. Manjang continues to work for Uber Eats. After his initial suspension from the app, his access was later restored – though as with the suspension, “the process that led to his access being restored was never fully explained to him.”
As the Times reported, Uber Eats initially attempted to have the case dismissed, claiming that it “had no reasonable prospects of success.” This motion was rejected by a judge. The case was set to go to trial in November 2024, but this has been averted by the out-of-court settlement.
Kishwer Falkner, chairwoman of the EHRC, said in a statement: “It is important to understand that as AI usage increases, the technology can lead to discrimination and human rights abuses.”
Problems with Facial Recognition
To date facial recognition technology has proven to be error-prone and unreliable. Past studies have established that the technology is less successful at recognizing people with darker skin color. In the United States there have been several cases of innocent black citizens being arrested after facial recognition software falsely identified them as criminals.
The App Drivers and Couriers Union has criticized the use of the technology by the ridesharing company Uber before. In London in 2021 there were several instances in which drivers were dismissed and had their taxi licenses revoked by Transport for London, the local transportation authority. Uber stated at the time that if problems arose with the automated system, verification was carried out by human reviewers.
There have been similar reports in India. There too drivers have been locked out of their accounts after the software failed to recognize them. (js)