AI-Generated Child Sexual Abuse Material: A Growing Threat Outpacing Tech Regulation

Online child sex abuse material, boosted by AI, is outpacing Big Tech's regulation

Generative AI is exacerbating the problem of online child sexual abuse materials (CSAM), as watchdogs report a proliferation of deepfake content featuring real victims' imagery. Published by the UK's Internet Watch Foundation (IWF), the report documents a significant increase in digitally altered or completely synthetic images featuring children in explicit scenarios, with one forum sharing 3,512 images and videos over a 30 day period. The majority were of young girls. Offenders were also documented sharing advice and even AI models fed by real images with each other. "Without proper controls, generative AI tools provide a playground for online predators to realize their most perverse and sickening fantasies," wrote IWF CEO Susie Hargreaves OBE. "Even now, the IWF is starting to see more of this type of material being shared and sold on commercial child sexual abuse websites on the internet." According to the snapshot study, there has been 17 percent increase in online AI-altered CSAM since the fall of 2023, as well as a startling increase in materials showing extreme and explicit sex acts. Materials include adult pornography altered to show a child's face, as well as existing child sexual abuse content digitally edited with another child's likeness on top. "The report also underscores how fast the technology is improving in its ability to generate fully synthetic AI videos of CSAM," the IWF writes. "While these types of videos are not yet sophisticated enough to pass for real videos of child sexual abuse, analysts say this is the 'worst' that fully synthetic video will ever be. Advances in AI will soon render more lifelike videos in the same way that still images have become photo-realistic." In a review of 12,000 new AI-generated images posted to a dark web forum over a one month period, 90 percent were realistic enough to be assessed under existing laws for real CSAM, according to IWF analysts. Another UK watchdog report, published in the Guardian today, alleges that Apple is vastly underreporting the amount of child sexual abuse materials shared via its products, prompting concern over how the company will manage content made with generative AI. In it's investigation, the National Society for the Prevention of Cruelty to Children (NSPCC) compared official numbers published by Apple to numbers gathered through freedom of information requests. While Apple made 267 worldwide reports of CSAM to the National Center for Missing and Exploited Children (NCMEC) in 2023, the NSPCC alleges that the company was implicated in 337 offenses of child abuse images in just England and Wales, alone -- and those numbers were just for the period between April 2022 and March 2023. Apple declined the Guardian's request for comment, pointing the publication to a previous company decision to not scan iCloud photo libraries for CSAM, in an effort to prioritize user security and privacy. Mashable reached out to Apple, as well, and will update this article if they respond. Under U.S. law, U.S.-based tech companies are required to report cases of CSAM to the NCMEC. Google reported more than 1.47 million cases to the NCMEC in 2023. Facebook, in another example, removed 14.4 million pieces of content for child sexual exploitation between January and March of this year. Over the last five years, the company has also reported a significant decline in the number of posts reported for child nudity and abuse, but watchdogs remain wary. Online child exploitation is notoriously hard to fight, with child predators frequently exploiting social media platforms, and their conduct loopholes, to continue engaging with minors online. Now with the added power of generative AI in the hands of bad actors, the battle is only intensifying. Read more of Mashable's reporting on the effects of nonconsensual synthetic imagery: What to do if someone makes a deepfake of youExplicit deepfakes are traumatic. How to deal with the pain. The consequences of making a nonconsensual deepfake Victims of nonconsensual deepfakes arm themselves with copyright law to fight the content's spreadHow to stop students from making explicit deepfakes of each other If you have had intimate images shared without your consent, call the Cyber Civil Rights Initiative's 24/7 hotline at 844-878-2274 for free, confidential support. The CCRI website also includes helpful information as well as a list of international resources.

Online child sex abuse material, boosted by AI, is outpacing Big Tech's regulation

Generative AI is exacerbating the problem of online child sexual abuse materials (CSAM), as watchdogs report a proliferation of deepfake content featuring real victims' imagery. Published by the UK's Internet Watch Foundation (IWF), the report documents a significant increase in digitally altered or completely synthetic images featuring children in explicit scenarios, with one forum sharing 3,512 images and videos over a 30 day period. The majority were of young girls. Offenders were also documented sharing advice and even AI models fed by real images with each other. "Without proper controls, generative AI tools provide a playground for online predators to realize their most perverse and sickening fantasies," wrote IWF CEO Susie Hargreaves OBE. "Even now, the IWF is starting to see more of this type of material being shared and sold on commercial child sexual abuse websites on the internet." According to the snapshot study, there has been 17 percent increase in online AI-altered CSAM since the fall of 2023, as well as a startling increase in materials showing extreme and explicit sex acts. Materials include adult pornography altered to show a child's face, as well as existing child sexual abuse content digitally edited with another child's likeness on top. "The report also underscores how fast the technology is improving in its ability to generate fully synthetic AI videos of CSAM," the IWF writes. "While these types of videos are not yet sophisticated enough to pass for real videos of child sexual abuse, analysts say this is the 'worst' that fully synthetic video will ever be. Advances in AI will soon render more lifelike videos in the same way that still images have become photo-realistic." In a review of 12,000 new AI-generated images posted to a dark web forum over a one month period, 90 percent were realistic enough to be assessed under existing laws for real CSAM, according to IWF analysts. Another UK watchdog report, published in the Guardian today, alleges that Apple is vastly underreporting the amount of child sexual abuse materials shared via its products, prompting concern over how the company will manage content made with generative AI. In it's investigation, the National Society for the Prevention of Cruelty to Children (NSPCC) compared official numbers published by Apple to numbers gathered through freedom of information requests. While Apple made 267 worldwide reports of CSAM to the National Center for Missing and Exploited Children (NCMEC) in 2023, the NSPCC alleges that the company was implicated in 337 offenses of child abuse images in just England and Wales, alone -- and those numbers were just for the period between April 2022 and March 2023. Apple declined the Guardian's request for comment, pointing the publication to a previous company decision to not scan iCloud photo libraries for CSAM, in an effort to prioritize user security and privacy. Mashable reached out to Apple, as well, and will update this article if they respond. Under U.S. law, U.S.-based tech companies are required to report cases of CSAM to the NCMEC. Google reported more than 1.47 million cases to the NCMEC in 2023. Facebook, in another example, removed 14.4 million pieces of content for child sexual exploitation between January and March of this year. Over the last five years, the company has also reported a significant decline in the number of posts reported for child nudity and abuse, but watchdogs remain wary. Online child exploitation is notoriously hard to fight, with child predators frequently exploiting social media platforms, and their conduct loopholes, to continue engaging with minors online. Now with the added power of generative AI in the hands of bad actors, the battle is only intensifying. Read more of Mashable's reporting on the effects of nonconsensual synthetic imagery: What to do if someone makes a deepfake of youExplicit deepfakes are traumatic. How to deal with the pain. The consequences of making a nonconsensual deepfake Victims of nonconsensual deepfakes arm themselves with copyright law to fight the content's spreadHow to stop students from making explicit deepfakes of each other If you have had intimate images shared without your consent, call the Cyber Civil Rights Initiative's 24/7 hotline at 844-878-2274 for free, confidential support. The CCRI website also includes helpful information as well as a list of international resources.

Online child sex abuse material, boosted by AI, is outpacing Big Tech's regulation

New reports document large numbers of real and AI child sex abuse materials shared online. Credit: dem10 / iStock / Getty Images Plus via Getty Images Generative AI is exacerbating the problem of online child sexual abuse materials (CSAM), as watchdogs report a proliferation of deepfake content featuring real victims' imagery. Published by the UK's Internet Watch Foundation (IWF), the report documents a significant increase in digitally altered or completely synthetic images featuring children in explicit scenarios, with one forum sharing 3,512 images and videos over a 30 day period. The majority were of young girls. Offenders were also documented sharing advice and even AI models fed by real images with each other. "Without proper controls, generative AI tools provide a playground for online predators to realize their most perverse and sickening fantasies," wrote IWF CEO Susie Hargreaves OBE. "Even now, the IWF is starting to see more of this type of material being shared and sold on commercial child sexual abuse websites on the internet." According to the snapshot study, there has been 17 percent increase in online AI-altered CSAM since the fall of 2023, as well as a startling increase in materials showing extreme and explicit sex acts. Materials include adult pornography altered to show a child's face, as well as existing child sexual abuse content digitally edited with another child's likeness on top. "The report also underscores how fast the technology is improving in its ability to generate fully synthetic AI videos of CSAM," the IWF writes. "While these types of videos are not yet sophisticated enough to pass for real videos of child sexual abuse, analysts say this is the 'worst' that fully synthetic video will ever be. Advances in AI will soon render more lifelike videos in the same way that still images have become photo-realistic." In a review of 12,000 new AI-generated images posted to a dark web forum over a one month period, 90 percent were realistic enough to be assessed under existing laws for real CSAM, according to IWF analysts. Another UK watchdog report, published in the Guardian today, alleges that Apple is vastly underreporting the amount of child sexual abuse materials shared via its products, prompting concern over how the company will manage content made with generative AI. In it's investigation, the National Society for the Prevention of Cruelty to Children (NSPCC) compared official numbers published by Apple to numbers gathered through freedom of information requests. While Apple made 267 worldwide reports of CSAM to the National Center for Missing and Exploited Children (NCMEC) in 2023, the NSPCC alleges that the company was implicated in 337 offenses of child abuse images in just England and Wales, alone -- and those numbers were just for the period between April 2022 and March 2023. Apple declined the Guardian's request for comment, pointing the publication to a previous company decision to not scan iCloud photo libraries for CSAM, in an effort to prioritize user security and privacy. Mashable reached out to Apple, as well, and will update this article if they respond. Under U.S. law, U.S.-based tech companies are required to report cases of CSAM to the NCMEC. Google reported more than 1.47 million cases to the NCMEC in 2023. Facebook, in another example, removed 14.4 million pieces of content for child sexual exploitation between January and March of this year. Over the last five years, the company has also reported a significant decline in the number of posts reported for child nudity and abuse, but watchdogs remain wary. Online child exploitation is notoriously hard to fight, with child predators frequently exploiting social media platforms, and their conduct loopholes, to continue engaging with minors online. Now with the added power of generative AI in the hands of bad actors, the battle is only intensifying. Read more of Mashable's reporting on the effects of nonconsensual synthetic imagery:

More AI-generated child sex abuse material is being posted online

More AI-generated child abuse material is appearing on dark web forums.Domenick Fini / NBC News The amount of AI-generated child sexual abuse material (CSAM) posted online is increasing, a report published Monday found. The report, by the U.K.-based Internet Watch Foundation (IWF), highlights one of the darkest results of the proliferation of AI technology, which allows anyone with a computer and a little tech savvy to generate convincing deepfake videos. Deepfakes typically refer to misleading digital media created with artificial intelligence tools, like AI models and applications that allow users to "face-swap" a target's face with one in a different video. Online, there is a subculture and marketplace that revolves around the creation of pornographic deepfakes. In a 30-day review this spring of a dark web forum used to share CSAM, the IWF found a total of 3,512 CSAM images and videos created with artificial intelligence, most of them realistic. The number of CSAM images found in the review was a 17% increase from the number of images found in a similar review conducted in fall 2023. The review of content also found that a higher percentage of material posted on the dark web is now depicting more extreme or explicit sex acts compared to six months ago. "Realism is improving. Severity is improving. It's a trend that we wouldn't want to see," said Dan Sexton, the IWF's chief technology officer. Entirely synthetic videos still look unrealistic, Sexton said, and are not yet popular on abusers' dark web forums, though that technology is still rapidly improving. "We've yet to see realistic-looking, fully synthetic video of child sexual abuse," Sexton said. "If the technology improves elsewhere, in the mainstream, and that flows through to illegal use, the danger is we're going to see fully synthetic content." It's currently much more common for predators to take existing CSAM material depicting real people and use it to train low-rank adaptation models (LoRAs), specialized AI algorithms that make custom deepfakes from even a few still images or a short snippet of video. The current reliance on old footage in creating new CSAM imagery can cause persistent harm to survivors, as it means footage of their abuse is repeatedly given fresh life. "Some of these are victims that were abused decades ago. They're grown-up survivors now," Sexton said of the source material. The rise in the deepfaked abuse material highlights the struggle regulators, tech companies and law enforcement face in preventing harm. Last summer, seven of the largest AI companies in the U.S. signed a public pledge to abide by a handful of ethical and safety guidelines. But they have no control over the numerous smaller AI programs that have littered the internet, often free to use. "The content that we've seen has been produced, as far as we can see, with openly available, free and open-source software and openly available models," Sexton said. A rise in deepfaked CSAM may make it harder to track pedophiles who are trading it, said David Finkelhor, the director of the University of New Hampshire's Crimes Against Children Research Center. A major tactic social media platforms and law enforcement use to identify abuse imagery is by automatically scanning new images to see if they match a database of established instances of CSAM. But newly deepfaked material may elide those sensors, Finkelhor said. "Once these images have been altered, it becomes more difficult to block them," he said. "It's not entirely clear how courts are going to deal with this," Finkelhor said. The U.S. Justice Department has announced charges against at least one man accused of using artificial intelligence to create CSAM of minors. But the technology may also make it difficult to bring the strictest charges against CSAM traffickers, said Paul Bleakley, an assistant professor of criminal justice at the University of New Haven. U.S. law is clear that possessing CSAM imagery, regardless of whether it was created or modified with AI, is illegal, Bleakley said. But there are harsher penalties reserved for people who create CSAM, and that might be harder to prosecute if it's done with AI, he said. "It is still a very gray area whether or not the person who is inputting the prompt is actually creating the CSAM," Bleakley said. In an emailed statement, the FBI said it takes crimes against children seriously and investigates each allegation with various law enforcement agencies. "Malicious actors use content manipulation technologies and services to exploit photos and videos -- typically captured from an individual's social media account, open internet, or requested from the victim -- into sexually-themed images that appear true-to-life in likeness to a victim, then circulate them on social media, public forums, or pornographic websites," the bureau wrote. "Many victims, which have included minors, are unaware their images were copied, manipulated, and circulated until it was brought to their attention by someone else. The photos are then sent directly to the victims by malicious actors for sextortion or harassment, or until it was self-discovered on the internet."

AI advances could lead to more child sexual abuse videos, watchdog warns

IWF warns of more AI-made child sexual abuse videos as tools behind them get more widespread and easier to use Advances in artificial intelligence are being used by paedophiles to produce AI-generated videos of child sexual abuse that could increase in volume as the technology improves, according to a safety watchdog. The majority of such cases seen by the Internet Watch Foundation involve manipulation of existing child sexual abuse material (CSAM) or adult pornography, with a child's face transplanted on to the footage. A handful of examples involve entirely AI-made videos lasting about 20 seconds, the IWF said. The organisation, which monitors CSAM around the world, said it was concerned that more AI-made CSAM videos could emerge as the tools behind them become more widespread and easier to use. Dan Sexton, chief technology officer at the IWF, said if the use of AI video tools followed the same trend as AI-made still images, which have increased in volume as the technology has improved and become more widely available, more CSAM videos could emerge. "I would tentatively say that if it follows the same trends, then we will see more videos," he said, adding that future videos could also be of "higher quality and realism". IWF analysts said the majority of videos seen by the organisation on a dark web forum used by paedophiles were partial deepfakes, where AI models freely available online are used to impose a child's face, including images of known CSAM victims, on existing CSAM videos or adult pornography. The IWF said it found nine such videos. A smaller number of wholly AI-made videos were of a more basic quality, according to the analysts, but they said this would be the "worst" that fully synthetic video would be. The IWF added that AI-made CSAM images have become more photo-realistic this year compared with 2023, when it first started seeing such content. Its snapshot study this year of a single dark web forum - which anonymises users and shields them from tracking - found 12,000 new AI-generated images posted over a month-long period. Nine out of 10 of those images were so realistic they could be prosecuted under the same UK laws covering real CSAM, the IWF said. The organisation, which operates a hotline for the public to report abuse, said it had found examples of AI-made CSAM images being sold online by offenders in place of non-AI made CSAM. The IWF's chief executive, Susie Hargreaves, said: "Without proper controls, generative AI tools provide a playground for online predators to realise their most perverse and sickening fantasies. Even now, the IWF is starting to see more of this type of material being shared and sold on commercial child sexual abuse websites on the internet." The IWF is pushing for law changes that will criminalise making guides to generate AI-made CSAM as well as making "fine-tuned" AI models that can produce such material. The cross-bench peer and child safety campaigner Baroness Kidron tabled an amendment to the proposed data protection and digital information bill this year that would have criminalised creating and distributing such models. The bill fell by the wayside after Rishi Sunak called the general election in May. Last week the Guardian reported that AI-made CSAM was overwhelming US law enforcement's ability to identify and rescue real-life victims.

Europol warns of uptick in AI-aided child abuse images

AFP - Artificial intelligence (AI)-linked images of child sex abuse are on the rise, Europe's policing agency warned yesterday, saying the material makes it increasingly difficult to identify victims and perpetrators. Criminals have been adopting AI tools and services to carry out a range of crimes from online fraud and cyberattacks, to creating explicit images of children, Europol said. "Cases of AI-assisted and AI-generated child sexual abuse material have been reported," the Hague-based agency said in a new report. "The use of AI which allows child sex offenders to generate or alter child sex abuse material is set to further proliferate in the near future," Europol added in a 37-page report, looking at current online threats facing Europe. The production of artificial abuse images increases "the amount of illicit material in circulation and complicates the identification of victims as well as perpetrators," Europol said. More than 300 million children a year were victims of online sexual exploitation and abuse, researchers at the University of Edinburgh said in May. Offences ranged from so-called sextortion, where predators demand money from victims to keep images private, to the abuse of AI technology to create deepfake videos and pictures, the university's Childlight Global Safety Institute said. The advent of AI has caused growing concern around the world that the technology can be used for malicious purposes such as the creation of so-called "deepfakes" - computer-generated, often realistic images and video, based on a real template. "The volume of self-generated sexual material now constitutes a significant and growing part of child abuse sexual material online," Europol said. "Even in the cases when the content is fully artificial and there is no real victim depicted, AI-generated child sex abuse material still contributes to the objectification and sexualisation of children," Europol said.

AI being used to generate deepfake child sex abuse images based on real victims, report finds

The tools used to create the images remain legal in the UK, the Internet Watch Foundation says, even though AI child sexual abuse images are illegal. Artificial intelligence (AI) is being used to generate deepfake child sexual abuse images based on real victims, a report has found. The tools used to create the images remain legal in the UK, the Internet Watch Foundation (IWL) said, even though AI child sexual abuse images are illegal. It gave the example of one victim of child rape and torture, whose abuser uploaded images of her when she was between three and eight years old. The non-profit organisation reported that Olivia, not her real name, was rescued by police in 2023 - but years later dark web users are using AI tools to computer-generate images of her in new abusive situations. Offenders are compiling collections of images of named victims, such as Olivia, and using them to fine-tune AI models to create new material, the IWL said. One model for generating new images of Olivia, who is now in her 20s, was available to download for free, it found. A dark web user reportedly shared an anonymous webpage containing links to AI models for 128 different victims of child sexual abuse. Other fine-tuned models can generate AI child sexual material of celebrity children, the IWL said. IWL analysts found 90% of AI images were realistic enough to be assessed under the same law as real child sexual abuse material. They also found AI images are becoming increasingly extreme. Read more: New AI tool could be game-changer in battle against Alzheimer's Why Google's greenhouse gas emissions have surged 48% in five years 'Incredibly concerning but also preventable' The IWL warned "hundreds of images can be spewed out at the click of a button" and some have a "near flawless, photo-realistic quality". Its chief executive Susie Hargreaves said: "We will be watching closely to see how industry, regulators and government respond to the threat, to ensure that the suffering of Olivia, and children like her, is not exacerbated, reimagined and recreated using AI tools." Richard Collard of the NSPCC said: "The speed with which AI-generated child abuse is developing is incredibly concerning but is also preventable. Too many AI products are being developed and rolled out without even the most basic considerations for child safety, retraumatising child victims of abuse. "It is crucial that child protection is a key pillar of any government legislation around AI safety. We must also demand tough action from tech companies now to stop AI abuse snowballing and ensure that children whose likeness are being used are identified and supported."

Grim Report Discovers First AI-Generated Child Sex Abuse Videos

A report by the Internet Watch Foundation (IWF) has found that generative AI models are being used to create deepfakes of real child sex abuse victims. The disturbing investigation by the UK-based IWF has found that the rise of AI video means that synthetic child sexual abuse videos are beginning to proliferate. The IWF, which describes itself as the "front line against online child sexual abuse", says it has identified AI models tailor-made for over 100 child sex abuse victims. It gave the example of one real-life abuse victim whose abuser uploaded images of her when she was between three and eight years old. The non-profit organization reports that Olivia, not her real name, was rescued by police in 2023 -- but years later dark web users are using AI tools to computer-generate images of her in new abusive situations. The criminals are collecting images of victims, such as Olivia who is now in her 20s, and using them to fine-tune AI models to create new material. Some of these models are freely available to download online, according to the report. AI video technology has made great strides this year and unfortunately, this is reflected in the report. The snapshot study was made between March and April this year, the IWF identified nine deepfake videos on one dark web forum dedicated to child sexual abuse material (CSAM) -- none had been previously found when IWF analysts investigated the forum in October. Some of the deepfake videos feature adult pornography which is altered to show a child's face. While others are existing videos of child sexual abuse that have had another child's face superimposed. Because the original videos of sexual abuse are of real children, IWF analysts say the deepfakes are especially convincing. Free, open-source AI software appears to be behind many of the deepfake videos seen by the IWF. The methods shared by offenders on the dark web are similar to those used to generate deepfake adult pornography. The IWF fears that as AI video technology improves, AI CSAM will become photorealistic. This comes as the IWF has already seen a steady increase in the number of reports of illegal AI images.

Labour urged to ban AI 'paedophile manuals' being shared online

Loophole in law allows predators to share instructions on how to create deepfake images Labour has been urged to ban AI "paedophile manuals" that teach predators how to generate images and videos of child abuse. The Internet Watch Foundation (IWF), a Cambridge-based watchdog, urged the Government to close a loophole in the law that allows paedophiles to create instructions on how to generate illegal "deepfakes" of children. Since 2014, it has been against the law to download or possess so-called "grooming manuals", which are shared by abusers to teach others how to target victims. However, the IWF warned a loophole remained meaning paedophiles could use online forums to train others to create AI child sexual abuse images. Dan Sexton, chief technology officer at the IWF, said there was a "very experienced" technical community among abusers who are "training, helping and skilling up others". "It is possible now to share a guide and all the open source tools openly, and it is only illegal once you have created an image," he said, calling for "prompt action from the new government". The IWF, which helps block child abuse images from the web, identified 3,512 illegal child abuse images generated by AI in a "snapshot" study of a dark web forum in March and April this year. That was an increase on the 2,978 images uncovered in a September survey. The number of "category A" images - the most serious that depict rape, torture or bestiality - had risen from 22pc to 32pc in the latest study. Using the latest AI tools, people can generate images, which can be highly photorealistic, using only text prompts. Dan Sexton, chief technology officer at the IWF, said: "The realism has improved. We know it's synthetic, but it looks real enough to pass as a real child." The IWF added that it should be made an offence for someone to use personal data to create AI models that can generate abuse images, while AI chatbots should be banned from initiating sexual communications with children. It also called for a block on so-called "nudifying" apps which can take pictures of people and "remove" their clothing without consent. Criminals have been taking AI software, which is often freely available to download online, and modifying it by "training" the technology with images of abuse. Mr Sexton added the IWF had now seen "clear evidence" that abuse imagery was being created using an AI tool known as Stable Diffusion, which was initially developed by British start-up Stability AI. He said: "The evidence is very clear. The foundation models that are being referenced - it has been early versions of Stable Diffusion." While Stability AI helped develop the image generation technology, it has since made some of its technology freely available to download and modify. The company has previously said: "Stability AI is committed to preventing the misuse of AI and prohibit the use of our image models and services for unlawful activity, including attempts to edit or create CSAM." The IWF also warned against plans being considered by OpenAI, the Silicon Valley business behind ChatGPT, to "responsibly" allow its technology to be used to create adult images. Mr Sexton said there were "obvious concerns" such tools could be misused. A government spokesman said: "We welcome the Internet Watch Foundation report and will carefully consider their recommendations. "We are committed to further measures to keep children safe online and go after those that would cause harm, including where AI is used to do so." Some in Whitehall had expected Labour to reveal plans for an AI Bill, which would have included new safety measures. However, this was absent from last week's King's Speech. An OpenAI spokesman said: "We have strong safeguards to prevent deepfakes or the creation or spreading of material harmful to children." Stability AI was contacted for comment.