The rise of AI-generated child sexual abuse material (CSAM) poses a significant threat to online safety and the well-being of children. According to the National Center for Missing and Exploited Children (NCMEC), the reports of CSAM received through their CyberTipline surged by 12% in 2023, reaching a staggering 36.2 million reports. This alarming increase is exacerbated by the influx of AI-generated CSAM, which not only revictimizes the children depicted in the original content but also creates new avenues for exploitation and normalization of child sexual abuse.
The Scope of the Problem
According to the National Center for Missing and Exploited Children (NCMEC), the CyberTipline, a vital resource for reporting CSAM, received a staggering 36.2 million reports in 2023, marking a 12% increase from the previous year. This surge in reports is exacerbated by the influx of AI-generated CSAM, which poses unique challenges in identifying genuine victims and distinguishing between real and synthetic content.
The proliferation of AI-generated CSAM not only burdens law enforcement agencies tasked with sifting through vast amounts of digital content but also raises concerns about the potential exploitation of vulnerable individuals, especially children. As generative AI technology continues to advance, the risk of misuse by perpetrators increases, necessitating proactive measures to combat this threat.
Recommendations and Strategies
To combat the threat of AI-generated CSAM, industry experts and child safety organizations have proposed several recommendations and strategies. One notable recommendation advises companies to exercise caution in selecting datasets utilized for training AI models, opting to avoid those exclusively comprised of instances of CSAM as well as adult sexual content. This precaution is essential due to generative AI's tendency to conflate these two categories.
Additionally, social media platforms and search engines are urged to promptly remove links to websites and apps facilitating the dissemination of illicit images of children, thereby thwarting the creation of new instances of AI-generated CSAM online
Challenges in Victim Identification
Victim identification, often described as a “needle-in-the-haystack” problem, has become increasingly challenging due to the proliferation of AI-generated CSAM. Law enforcement agencies must sift through vast amounts of digital content to identify and protect vulnerable victims, and the growing volume of AI-generated material only adds to this haystack, making the task more resource-intensive and time-consuming.
Collaborative Efforts and Safety by Design Principles
Recognizing the urgency of this issue, leading AI companies, including OpenAI, Microsoft, Google, and Meta, have united with child safety organizations like Thorn and All Tech Is Human to combat the generation and dissemination of AI-generated CSAM. This groundbreaking initiative has led to the development of “Safety by Design” principles, which aim to prevent the creation and spread of AI-generated CSAM and other sexual harms against children.
The key principles outlined in the “Safety by Design for Generative AI: Preventing Child Sexual Abuse” paper include:
| Principle | Description |
|---|---|
| Develop | Responsibly source training datasets, detect and remove CSAM/CSEM, incorporate feedback loops, and address adversarial misuse during model development. |
| Deploy | Release models after evaluating for child safety, combat abusive content, and encourage developer ownership of safety. |
| Maintain | Actively understand and respond to child safety risks, remove AIG-CSAM generated by bad actors, invest in research and future solutions. |
By implementing these principles across all stages of AI development, deployment, and maintenance, companies aim to make it more difficult for bad actors to misuse generative AI for the sexual abuse of children.
The Role of Legislation and Policymakers
Policymakers and legislators also play a crucial role in addressing this issue. The REPORT Act, recently passed by the U.S. Senate, aims to enhance the reporting and combating of online child sexual exploitation by extending providers' preservation period for reported content and granting limited liability to NCMEC's vendors.
However, concerns have been raised about the potential for overreporting, which could further strain the already overburdened CyberTipline system and distract investigators from focusing on legitimate cases.
As the threat of AI-generated CSAM continues to evolve, it is imperative that policymakers and industry stakeholders collaborate to develop responsible frameworks and guidelines. These frameworks should allow companies to legally and safely test large language models to improve detection capabilities while prioritizing child safety and online protection.
On March 12 2024, John Shehan, NCMEC's Senior Vice President, Exploited Children Division & International Engagement, testified on deepfake harms to children.
Read his full testimony here
Ongoing Efforts and Future Challenges
While the commitment to Safety by Design principles is a significant step forward, the battle against AI-generated CSAM is far from over. Companies have agreed to release progress updates on their efforts, ensuring transparency and accountability. However, the rapid pace of technological advancements and the ever-evolving tactics of bad actors necessitate continuous vigilance and collaboration among all stakeholders, including technology companies, law enforcement agencies, and policymakers.
As Dr. Rebecca Portnoff, Vice President of Data Science at Thorn, emphasizes, “We're at a crossroads with generative AI, which holds both promise and risk in our work to defend children from sexual abuse“. Addressing this challenge will require a multifaceted approach, leveraging both technological solutions and robust legal frameworks to protect the most vulnerable members of society.
