How OpenAI aims to ease the burden on child abuse tip lines

OpenAI says its classifier can identify abusive imagery, including AI-generated material, while sending fewer false positives for review. The company is also offering vetted organizations free access and adding warnings, account pauses and support resources for ChatGPT users who try to generate child sexual abuse material.

WTF Index TERMINATOR
◄ Terminator 2 Idiocracy 1 ►

The story describes AI-generated child abuse imagery and its impact on investigators, though the classifier is intended to help identify harm and reduce review burdens.

How OpenAI aims to ease the burden on child abuse tip lines

As AI-generated sexual images of children add to the volume of reports investigators must assess, OpenAI is introducing a classifier intended to help identify abusive material and sort reports for review. The company says the tool flags fewer false positives while making the images it identifies more likely to contain abuse.

A growing volume of reports

The National Center for Missing and Exploited Children (NCMEC) operates the CyberTipline, which collects and triages reports of child sexual abuse material (CSAM). The source article says the tipline received 400,000 generative AI-related reports in 2025, up from 67,000 in 2024. Researchers have warned that the influx could make the workload “unmanageable.”

That increase creates a difficult sorting problem. Investigators need to identify reports that point to real children experiencing harm, while also dealing with synthetic images that may still be abusive. Researchers say a flood of fake imagery can make it harder to tell when a child is actively being harmed.

North Carolina Attorney General Jeff Jackson, who cochairs the AG Alliance’s AI Task Force, called the increase “explosive” at OpenAI’s briefing. He warned that if reports keep growing at that pace, “it is going to mean that we fail in our mission.”

What OpenAI says the classifier can do

OpenAI announced the classifier at a Monday press briefing in Washington, DC. The company says it can detect abusive imagery of children, including images generated by AI, and distinguish material that may depict a fictional child from imagery involving a real child.

OpenAI says the classifier flagged 48 percent fewer images for review, while the images it did flag were twice as likely to contain abusive material. It is designed to identify images made with different tools, including open-source models and nudifying apps. The company described it as the first broadly available CSAM classifier with a specific flag for AI-generated material.

Reducing false positives could help investigators spend less time reviewing reports that do not contain abusive material. A specific flag for synthetic imagery could also give organizations another way to triage cases. The tool’s stated purpose is to support review and reporting flows, rather than to replace the work of investigators.

OpenAI says it will offer free access to vetted organizations, provide developers with more guidance on detecting this content, and help train law enforcement on using AI in investigations. The source describes the classifier’s results as claims from OpenAI; it does not provide independent evaluation details.

Warnings and pauses for ChatGPT users

OpenAI says ChatGPT’s image generator is designed to refuse requests for CSAM. The company is adding a warning, developed with expert guidance, for users who attempt to create it. The warning is intended to encourage them to seek help.

A first offense will trigger a 24-hour pause in access to ChatGPT. The user will also be told that their account could be permanently closed and offered support resources. OpenAI says it reports “all instances of CSAM” to NCMEC.

John Buckley, OpenAI’s head of child safety public policy, said the pause draws on expert advice and “international evidence on what works to prevent and respond to CSAM seeking vs CSAM generation.” He described the goal as interrupting harmful behavior and directing people to specialist help.

Prevention alongside enforcement

The announcement comes amid pressure on AI companies to take stronger steps to protect children. The article says the US government has stated that both AI-generated and non-AI-generated CSAM is illegal, and that 48 states have laws addressing sexually explicit AI deepfakes. It also describes policy proposals and legal action concerning potential harms to children.

The Take It Down Act requires social media platforms to remove real or AI-generated nonconsensual intimate images within 48 hours. Separately, eight states sued OpenAI last month over alleged harms to children, including claims that ChatGPT encouraged illegal substance use and suicide.

Jackson said that working with a technology company to help prevent synthetic CSAM is a relatively new approach. He described the effort as an experiment in preventing harm before it happens, rather than relying only on accountability after harm occurs. Whether the classifier and user warnings can ease pressure on reporting systems will depend on how these measures work in practice.