OpenAI announced a package of child-safety measures this week, the most concrete of which is a free image classifier developed with the US National Center for Missing and Exploited Children (NCMEC) that detects child sexual abuse material, including AI-generated images that traditional hash matching misses. The classifier is offered at no cost through an API to vetted organizations regardless of team size.
The efficiency claim is the notable part. In OpenAI's internal evaluation, the classifier flagged 48 percent fewer images than the existing tooling while finding roughly the same number of abusive images, and images it did flag were about twice as likely to actually contain abuse. Those figures are company-reported and have not been independently audited, but the design goal — cutting reviewer workload without losing recall — addresses the core bottleneck of human moderation pipelines.
Early partners supplied real-world numbers that OpenAI passed along with their own caveats. In fal's human-reviewed testing, 72 percent of the classifier's high-confidence flags were confirmed as novel CSAM. SafetyKit reported a 90 percent detection rate on drawings and animations that sexualize children, a category that has grown sharply with generative tools and that hash databases cover poorly. Because the classifier returns scores rather than verdicts, it is positioned as a triage layer that helps small safety teams decide what to review first, with humans making the final call.
Inside ChatGPT, the intervention moves from after-the-fact review to the moment of the attempt — the product change English-language users will actually notice. When someone searches for child sexual exploitation material, a warning now interrupts the attempt on the spot, explains the harm of sexualizing minors including when material is AI-generated, and the flagged conversation is removed while access is paused for 24 hours. Repeated violations can bring longer restrictions, account closure, or reports to authorities.
The announcement also included practical guidance for developers building with OpenAI's models, and investigative tools plus training for law-enforcement units working exploitation cases. The division of labor is deliberate: detection tooling goes to institutions, intervention lives inside the product, and investigative support goes to police.
The release lands in the same week OpenAI began embedding external assessors — Apollo Research, METR, Transluce and Goodfire — inside its labs for independent evaluation of alignment and safeguards, a step this site reported earlier. Together the two moves sketch a safety posture built for scrutiny: verifiable third-party access on the research side, and measurable, shareable tooling on the abuse-fighting side. For the many platform trust-and-safety teams that cannot afford to build their own classifiers, a free, scored API from a frontier lab is genuinely useful — with the caveat that access is gated to vetted organizations, not a public endpoint.
Comments (0)
Log in to join the discussion
Log InNo comments yet