OpenAI

OpenAI releases child safety tools

OpenAI launches new AI tools to detect child sexual abuse material and support law enforcement.

10 cards · 1m 20s · tap to begin

From · · 1m 20s

OpenAI releases child safety tools

OpenAI launches new AI tools to detect child sexual abuse material and support law enforcement.

In brief

OpenAI launches new AI tools to detect child sexual abuse material and support law enforcement. OpenAI released a CSAM detector, ChatGPT warnings, developer guides, and law enforcement tools to combat online child sexual exploitation. Originally reported by OpenAI.

New CSAM detection tool

Strengthening the fight against online child sexual abuse

OpenAI releases an image classifier to detect potential child sexual abuse material including AI-generated content.

“The new classifier looks for visual signs of abuse, helping teams identify novel images that may not have been seen before”

Industry collaboration

OpenAI partners with Tech Coalition, ROOST, and Thorn to share CSAM detection tools across organizations.

“We’ve engaged the Tech Coalition and ROOST to help us share access to this new tool”

Early results

Early tests show the classifier flags 48% fewer images while finding similar numbers of abusive content.

“The new classifier could reduce review work: we estimated that it would flag 48% fewer images while finding roughly the same number of abusive images as the existing tool”

ChatGPT warnings

ChatGPT warnings explain that sexualizing children is harmful and can be illegal, suspend the account for 24 hours, and offer specialist support.

ChatGPT now warns users about sexualizing children, suspends accounts for 24 hours, and offers specialist support.

“The warning makes clear that sexualizing children causes harm, including when the requested material is ai-generated”

Developer support

OpenAI publishes CSAM guidance for developers to prevent abuse and detect harmful content.

“Developed with input from the Tech Coalition, Thorn, NCMEC, and the Internet Watch Foundation”

Law enforcement tools

OpenAI provides AI tools for child safety investigators to review chat logs and create timelines.

“The initial tool will help investigators review lengthy chat logs and legal process returns, identify potential connections, and create timelines”

Real-world impact

Early partners confirmed 72% of high-confidence flags as novel CSAM and 90% of sexualized drawings.

“In SafetyKit’s evaluations involving drawings or animations that sexualize children, the classifier detected 90% of cases”

Training and support

OpenAI offers field guides and training for law enforcement on using AI in child safety investigations.

“The initial training is planned as an overview and demonstration, supported by a practical field guide that teams can refer to afterward”

Collaborative approach

OpenAI works with Anthropic and Google to create joint guidance for handling child sexual exploitation disclosures.

“We’ve already seen this guidance lead to increased numbers of developers accessing child safety tooling and registering to report csam to ncme”

Next steps

OpenAI will continue improving tools while supporting child protection organizations and investigators.

“By working together, we can harness the power of ai to make a meaningful difference in the fight against child sexual exploitation and abuse”

The main idea

OpenAI released a CSAM detector, ChatGPT warnings, developer guides, and law enforcement tools to combat online child sexual exploitation.

Read the original on OpenAI

Read the TL;DR — this story on one card.

How we make summaries

React

Sign in to react and comment.

Comments (0)

Life is short. Keep it sweet. Respect others' opinions and be kind!

    Recommended next

    More decks on openai and related topics.