AI-generated child sexual abuse material is creating new challenges for organizations that work to identify and remove it. The Internet Watch Foundation (IWF) says its analysts found 20,254 AI-generated images on a single dark web forum during one month. More than half were considered potentially criminal and examined by specialists.
Thousands of images required specialist review
Of the images found, 11,108 were assessed as potentially criminal. A team of 12 IWF specialists spent a total of 87.5 hours analyzing them.
The IWF classified 2,562 images as criminal pseudo-images and 416 as criminal prohibited images. The material was assessed under the Protection of Children Act 1978 or the Coroners and Justice Act 2009.
The scale of the review illustrates the work involved in distinguishing and assessing AI-generated material. The IWF says such content remains a small part of its overall work, but warns that it could grow quickly.
Open tools can produce increasingly realistic images
According to the report, unrestricted open-source text-to-image systems can generate realistic images from written descriptions. The results can be virtually indistinguishable from real photographs, making detection more difficult for the IWF and law enforcement.
The concern is not limited to the volume of new images. The report says AI-generated material may contribute to the re-victimization of known abuse victims and celebrity children. It can also create another way for offenders to profit from child abuse.
As generated images become more realistic, investigators may face greater difficulty determining what they are examining and how to respond. The IWF says this trend threatens its mission to remove child sexual abuse material from the internet, and warns that advances in the technology could increase child abuse.
Recommendations span governments and technology companies
The IWF is calling for action by governments, law enforcement and the technology sector. Its recommendations focus on coordination, enforcement, training and oversight:
- Promote international coordination on how this content is handled.
- Review laws governing the removal of online content.
- Update police training to cover AI-generated child sexual abuse material.
- Provide regulatory oversight of AI models.
- Require companies developing and deploying generative AI and large language models (LLMs) to prohibit generating this material in their terms of service.
These proposals address different points in the process: how content is classified and removed, how investigations are conducted, and what rules apply to the systems that can generate images. The report presents them as measures for a problem that may expand beyond the IWF's current caseload.
Reports point to wider investigative challenges
The source article also cites a late-June report from the National Center for Missing and Exploited Children in the US, which described a “sharp increase” in AI-generated abuse images. The organization said the images could complicate investigations and hinder victim identification.
Users on abuse forums have shared instructions for generating images with open-source models. Child advocates and U.S. justice officials say these images are punishable by law, but the source reports that there have been no court rulings on their classification or sentencing.
Taken together, the findings describe a fast-moving challenge: AI tools can generate realistic material, while reviewers and investigators must assess it and determine what action is possible. The IWF's recommendations call for responses from public authorities and technology companies as well as the organizations handling reports.