9 Sources
[1]
OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM
OpenAI today announced new tools to combat child sexual abuse material (CSAM) amid growing safety concerns around the use of AI tools. These features include a new image classifier to detect CSAM, warnings and temporary blocks to prevent users from creating or accessing content that sexualizes
[2]
OpenAI wants to stop deepfakes from overwhelming child abuse tiplines
OpenAI is announcing a tool that it says will help child sexual abuse investigators cut through the noise to find harmful material they can act on quickly. At a Monday press briefing in Washington, DC, OpenAI announced a classifier it says can better detect abusive imagery of kids, including
[3]
New OpenAI tool aims to detect child sexual abuse
AI-generated CSAM is skyrocketing. OpenAI's new tool may help detect it. Credit: SchulteProductions via iStock / Getty Images Plus OpenAI announced Monday the release of a new tool to detect child sexual abuse material (CSAM), including content generated by artificial intelligence. The tool is a
[4]
OpenAI introduces stricter ChatGPT interventions to prevent the misuse of AI for child exploitation
OpenAI's new AI tools aim to make online platforms safer for children OpenAI is expanding its efforts to combat online child sexual exploitation with a new set of AI-powered detection tools, stricter interventions in ChatGPT, and additional support for developers and law enforcement agencies. The
[5]
OpenAI unveils tool to better detect child sexual abuse material
OpenAI released a new tool Monday aimed at detecting possible child sexual abuse material (CSAM), including content made with artificial intelligence. The image classifier, which will be available for vetted organizations and law enforcement investigators focused on online safety, was trained and
[6]
OpenAI rolls out CSAM detection tool and new child safety measures
OpenAI has announced new child safety measures, including an OpenAI CSAM detection tool to identify potential child sexual abuse material (CSAM), new warnings in ChatGPT to interrupt harmful behavior, practical guidance for developers, and AI tools and training for law enforcement
[7]
OpenAI releases new classifier to detect CSAM, AI-generated images
OpenAI has released an image classifier to detect child sexual abuse material (CSAM), including images that have been digitally manipulated or generated using AI. The company developed the model with assistance from the US-based National Center for Missing and Exploited Children (NCMEC) and is
[8]
OpenAI Rolls Out Classifier; Aims to Detect AI-Generated CSAM
OpenAI has launched an image classifier to detect suspected child sexual abuse material, including AI-generated images, alongside ChatGPT warnings, temporary suspensions, developer guidance and support for law-enforcement investigations. OpenAI introduced a new image classifier that flags
[9]
OpenAI brings new tools and warnings for ChatGPT to crack down child abuse content
Repeated violations could result in longer account restrictions, permanent suspension or reports to authorities. AI deployment company OpenAI is widely popular for its viral chat model, ChatGPT. In a recent update, the company is now introducing a new tool for detecting child sexual abuse material
Share
Copy Link
OpenAI released a free classifier that spots AI-generated abuse images. Its own model card shows precision falling from 99% in the lab to around 30% on live traffic.
OpenAI announced a package of child-safety measures on November 16, built around an image classifier that detects child sexual abuse material, or CSAM, child sexual abuse material,, child sexual abuse material,, including images made or altered with AI. The release also brings warning screens and 24-hour suspensions to ChatGPT, new guidance for developers, and investigative tools for police
1
5
2
.
Source: Digit
Reports of suspected AI-generated abuse material to the National Center for Missing and Exploited Children's CyberTipline rose from 67,000 in 2024 to about 400,000 in 2025. Researchers have warned the workload could become unmanageable
2
3
. Hash-matching, the existing standard, works by matching previously identified material, so new or AI-generated images slip past it8
7
.OpenAI says companies have a responsibility to stop their technology being used to harm children. That means working with industry, child-safety groups and law enforcement
3
.The classifier scores images for signs of abuse so human reviewers can prioritize their workload. It produces two separate scores: one for suspected abuse material and one for suspected AI-generated material. OpenAI says it is the first broadly available classifier to carry a specific signal for AI-generated material
5
2
7
. NCMEC helped develop it, and it is free through the OpenAI API to vetted organizations, shared via the Tech Coalition and ROOST, an open-source safety coalition1
4
.In early testing it flagged an estimated 48% fewer images while catching roughly the same number of abusive images
1
2
. Flagged images were roughly twice as likely to include abuse1
2
. Early partners reported their own results, self-submitted: fal's reviewers verified 72% of its high-confidence flags as new material4
6
.OpenAI's own model card carries a caveat. Per that card, offline precision is 99.1% for the likely-CSAM score and 100% for the AI-generated score. Against real ChatGPT traffic, those figures fell to 29.8% and 34%
7
. OpenAI attributes the drop to how rare this material is in ordinary traffic. Even so, a system flagging many images that turn out to be clean adds reviewer work rather than saving it7
. OpenAI says the score alone does not confirm abuse or establish anything about legal status, so human review stays in the loop5
7
.Camille François, president of ROOST, said "OpenAI is setting an important precedent for transparency in child safety." She considers it the first model card in the child-safety detection field
3
.Related Stories
Users who try to find or create content that sexualizes children will get a warning, lose the flagged conversation, and have their account paused for 24 hours. Repeat violations can bring longer restrictions, closure, or reports to authorities
1
2
. ChatGPT already blocked such requests and refused to generate this material; the new warnings add an intervention layer for the users who try anyway1
6
. OpenAI says the point is to interrupt the behavior and steer people toward specialist help. John Buckley, who leads child-safety public policy at OpenAI, said the 24-hour pause follows expert advice and international evidence on what disrupts people searching for this material5
2
.
Source: TechCrunch
Developer guidance on detecting and reporting abuse material was written with the Tech Coalition, Thorn, NCMEC and the Internet Watch Foundation. It was commissioned jointly with Anthropic and Google
1
4
. Police get AI tools that review chat logs and build timelines, plus training and a field guide. Buckley said police teams faced capacity problems during development. OpenAI wants its tooling to help them triage and reach high-risk cases faster5
.The announcement lands amid pressure: eight states sued OpenAI last month over alleged harms to children, and Common Sense Media has called its technology an unacceptable risk for teens
1
2
.North Carolina Attorney General Jeff Jackson, cochair of the AG Alliance's AI Task Force, spoke at OpenAI's event and described the influx of AI-generated abuse reports as explosive
2
. "Most of what AGs do is accountability work... But with this, we wanted to see if it was possible to develop a model of working together to prevent that harm. And I would say it's still basically an experiment."
Source: Analytics Insight
Summarized by
Navi
[4]
08 Apr 2026•Policy and Regulation

19 May 2026•Technology

24 Mar 2026•Technology
