A legal challenge in France is putting a long-running welfare algorithm under direct scrutiny. Human rights groups say the system, used by the French welfare agency, the CNAF, turns personal data into risk scores that can expose benefit recipients to investigations, suspended payments, and discriminatory treatment.
The case focuses on how governments use automated tools to police welfare systems. It also arrives as welfare algorithms across Europe are becoming an early test for how new AI rules may apply to public services.
A challenge to France's welfare algorithm
A coalition of 15 groups, including La Quadrature du Net, Amnesty International, and Collectif Changer de Cap, has launched legal action against the French government over the algorithm. The groups argue that the system violates European privacy rules and French anti-discrimination laws.
The algorithm has been used since the 2010s to detect welfare payments that may have been made in error, whether because of fraud or mistake. According to the litigation filed to France's top administrative court on October 15, the CNAF analyzes personal data connected to more than 30 million people. That includes people claiming government support, along with the people they live with and their family members.
The system gives each person a score between 0 and 1. That score estimates how likely the person is to be receiving payments they are not entitled to. A higher score can place someone closer to an investigation, and welfare recipients across Europe have described those investigations as stressful and intrusive.
Valérie Pras of Collectif Changer de Cap described the French case as a first for the country. “This is the first time that a public algorithm has been the subject of a legal challenge in France,” she said. She added that she wants these types of algorithms to be banned, arguing that other social organizations in France also use scoring systems to target poor people.
Why rights groups say the scoring is discriminatory
The dispute is not only about automation. It is about which personal details are used to mark someone as risky, and what happens when those details are tied to poverty, disability, family status, or instability.
The CNAF has not publicly shared the source code for the current model it uses to detect welfare payments made in error. But La Quadrature du Net has analyzed older versions of the algorithm, suspected to have been in use until 2020. Based on that analysis, the group claims the system assigns higher risk to marginalized groups.
Bastien Le Querrec, a legal expert at La Quadrature du Net, said people receiving a disability allowance are directly affected by the model. “People receiving a social allowance reserved for people with disabilities [the Allocation Adulte Handicapé, or AAH] are directly targeted by a variable in the algorithm,” he said. “The risk score for people receiving AAH and who are working is increased.”
The groups also argue that the system indirectly discriminates against single mothers. Their reasoning is that single-parent families receive higher scores than two-parent families, and single mothers are statistically more likely to be sole-care givers. Le Querrec pointed to one older version of the system: “In the criteria for the 2014 version of the algorithm, the score for beneficiaries who have been divorced for less than 18 months is higher,” he said.
Collectif Changer de Cap says it has been approached by both single mothers and disabled people after they were subject to investigation.
Privacy concerns go beyond individual cases
The legal challenge frames the algorithm as a privacy issue as well as a discrimination issue. The groups argue that the system applies broad data processing to people who depend on state support, including people close to them.
The legal documents say: “The processing, implemented by the CNAF, constitutes massive surveillance and a disproportionate attack on the right to privacy.” They also state: “The effects of this algorithmic processing particularly affects the most precarious people.”
The CNAF distributes financial aid including housing, disability, and child benefits. It did not immediately respond to a request for comment or to WIRED's question about whether the algorithm currently in use had significantly changed since the 2014 version.
The concern raised by the case is that a welfare fraud-detection tool can become more than a technical filter. When the output affects investigations or payment suspensions, a risk score can shape the experience of people already relying on public support.
A wider European fight over welfare algorithms
France is not alone in using algorithms to search for error or fraud in welfare systems. A three-part investigation by WIRED with Lighthouse Reports examined welfare fraud-detection algorithms in the Netherlands, Denmark, and Serbia.
Rights groups in other European countries have made similar arguments: that these systems subject low-income people to intense surveillance and can have serious consequences. In the Netherlands, tens of thousands of people, many of them from the country's Ghanaian community, were falsely accused of defrauding the child benefits system. They were ordered to repay money the algorithm said they allegedly stole, and many said they were left with spiraling debt and destroyed credit ratings.
Soizic Pénicaud, a lecturer in AI policy at Sciences Po Paris who previously worked for the French government on transparency of public sector algorithms, argues that the issue is the use of algorithms in welfare itself. “Using algorithms in the context of social policy comes with way more risks than it comes with benefits,” she said. “I haven't seen any example in Europe or in the world in which these systems have been used with positive results.”
Why the case could matter beyond France
The French case may have consequences outside the country because welfare algorithms are expected to be an early test of how the EU's new AI rules will be enforced. Those rules are set to take effect in February 2025.
From then, “social scoring” will be banned across the bloc. The source describes social scoring as the use of AI systems to evaluate people's behavior and then subject some of them to detrimental treatment.
Matthias Spielkamp, cofounder of the nonprofit Algorithm Watch, said many welfare fraud-detection systems may fit that category in practice. “Many of these welfare systems that do this fraud detection may, in my opinion, be social scoring in practice,” he said.
Public sector representatives are likely to disagree with that interpretation. Spielkamp said the question of how to define these systems is likely to end up in court, adding: “I think this is a very hard question.”
For welfare agencies, the stated goal is to find payments made in error. For the groups bringing the French case, the central issue is whether that goal can justify a scoring system that processes personal data at scale and places vulnerable people under deeper scrutiny. The court fight will test not only one French algorithm, but the broader assumption that automated risk scoring belongs in welfare administration.