The United Kingdom is preparing a new step in AI safety: checking models before they reach the public when there is a risk they could be used to create child sexual abuse material (CSAM). The plan would give authorized organizations a clearer role in testing systems early, before harmful uses become harder to contain.
The proposal sits inside a wider debate about how governments should respond when powerful AI tools can be misused for illegal imagery, non-consensual content, or violence-related material. In this case, the government’s stated aim is direct: build safety into AI systems before release, rather than trying to respond only after abuse appears.
What The UK Proposal Would Allow
According to a BBC report, the government wants to expand the "Crime and Policing Bill" so that targeted evaluations of AI systems can take place before models go public. The tests would focus on whether a model can be misused to generate CSAM.
The proposal would authorize certain testers, including technology firms and child protection organizations, to assess AI systems for those risks. The point is not simply to identify harmful outputs after deployment. It is to examine whether the model has dangerous capabilities before broad release.
Tech Minister Liz Kendall described the goal as making AI systems safe "at the source". That framing matters because it places responsibility at the development and release stage. If the risk is present inside the system, the government wants it addressed before the model becomes widely available.
Safeguarding Minister Jess Phillips also emphasized the concern that tools which appear harmless can be turned toward abusive content. The proposal is designed around that problem: AI systems may be general-purpose, but misuse can still be specific, severe, and illegal.
Why Child Protection Groups Are Pressing For Action
The Internet Watch Foundation (IWF) supports the initiative. The organization is one of the few groups authorized to proactively search for CSAM, which gives its warnings particular relevance in this debate.
According to the IWF, reports involving AI-generated abuse imagery have increased sharply. Between January and October 2025, the group removed 426 AI-related CSAM items. In the same period in 2024, it removed 199.
Those numbers are central to the case for earlier testing. They suggest that AI-generated material is not a theoretical concern for child protection organizations. It is already appearing in the systems and workflows used to identify and remove abusive content.
IWF CEO Kerry Smith warned that AI enables abusers to revictimize survivors. The source article also notes that the IWF had already reported a sharp rise in 2023, and that this kind of material can complicate investigations into real child abuse cases.
That investigative challenge is a major part of the issue. When synthetic abuse imagery becomes more realistic, investigators may face more difficulty distinguishing real material from artificially created content. That can create additional burdens in cases where speed and accuracy matter.
The Debate Over Voluntary Or Mandatory Checks
The NSPCC also backs the government’s approach, but it wants the testing requirement to go further. The organization argues that the checks should be mandatory, not voluntary.
NSPCC policy manager Rani Govender said child safety must be built into AI development from the start, rather than added later. That position reflects a broader concern in AI safety: once a model has been released, controlling its misuse can become much harder.
The distinction between voluntary and mandatory testing is important. A voluntary framework may encourage responsible companies and safety groups to act before release. A mandatory framework would make that expectation a requirement for covered systems.
The source article does not describe the full scope of how such requirements would operate. But the policy disagreement is clear: child protection groups support pre-release evaluation, while the NSPCC wants stronger obligations to make sure the checks happen consistently.
Beyond CSAM: Wider Abuse Risks
The proposed changes would not be limited to CSAM. They would also cover other forms of non-consensual or violence-related content.
Experts cited in the source warn that large AI models trained on unfiltered internet data can be exploited to generate realistic synthetic depictions of abuse or assault. The concern is that these systems may reproduce or create harmful imagery in ways that are difficult to police after release.
This makes pre-release testing a practical safety measure as well as a legal one. If a model can produce illegal or abusive material, discovering that before launch gives developers, companies, and authorized safety groups a chance to address the risk earlier.
The Home Office has already announced that the UK will become the first country to criminalize the possession, development, and use of AI systems designed to create CSAM. Violations could be punished with up to five years in prison.
Taken together, the testing proposal and the announced criminal penalties point toward a more source-focused approach to AI abuse. The government is not only targeting people who use AI systems for illegal imagery. It is also looking at the systems themselves, and at whether dangerous capabilities can be found before release.
What This Means For AI Developers
For AI companies, the message is that child safety is moving closer to the model development process. The proposal would make pre-release scrutiny part of the discussion around whether a system is ready for public use.
For child protection organizations, the plan could create a more formal role in evaluating AI risks before harm spreads. For the public, it signals that AI safety policy is becoming more concrete in areas where misuse can cause direct harm.
The central question now is how strong the final rules will be. The government is pursuing targeted evaluations through the "Crime and Policing Bill". Child protection groups support that direction, but the NSPCC is pushing for mandatory testing so that safety is not left to later corrections.