Global crackdown on sexual deepfakes intensifies as US, Europe, and New Zealand enact strict laws

NZ is criminalising sexualised deepfakes - banning apps that make them should be next

New Zealand is changing the law to make sexualised deepfakes a crime. But this alone may not be enough to counter the rise in AI-generated fake sexual material. This week the Deepfake Digital Harm and Exploitation Bill is set to go through its first reading, with support across the political spectrum. The amendment will make creating, sharing or selling sexually explicit deepfakes without consent a criminal offence. It comes in response to the rapid spread of such material, including the rollout of Elon Musk's Grok AI chatbot on X, which people used to digitally undress women and girls and generate potentially three million sexualised images. New Zealand is not alone in confronting the problem. The United Kingdom, Australia, South Korea and the United States have already introduced or expanded laws to criminalise creating and sharing of non-consensual deepfakes. Criminalisation is an important first step and brings New Zealand in line with developments elsewhere. But stemming the tide of sexualised deepfakes will also require regulation of the technology itself. Deepfakes mostly target women Deepfakes are AI-generated images, audio or video designed to make it appear that a real person said or did something that never actually happened. With image-based sexual abuse, this usually means using AI to create convincing but fake sexual material of a person without their consent. Sometimes, technology is used to manipulate ordinary photos pulled from social media into explicit imagery. Other AI systems can generate entirely fabricated sexual content from a text prompt alone. The abuse is overwhelmingly directed at women. One widely cited study found 98% of deepfake videos online were pornographic and mostly targeted women. For victim-survivors, the harms are significant regardless of whether the image is created from a real original or completely fabricated. People report humiliation, fear, anxiety, loss of control and violation of sexual autonomy. Gaps in New Zealand law The new bill is designed to fill gaps in the current law. At present, New Zealand does not have a criminal offence specifically directed at sexualised deepfakes. Existing laws may apply, but they were not designed for AI-generated abuse. The Harmful Digital Communications Act 2015 criminalises posting harmful digital communications and has already been used in at least one prosecution involving a sexualised deepfake. However, the offence requires proof that the defendant intended to cause serious emotional distress and that this actually resulted. Those requirements can be difficult hurdles for victim-survivors of image-based sexual abuse. A new offence introduced into the act in 2022 sought to address this for so-called "revenge porn", making it a crime to share an intimate visual recording without consent. This sat alongside existing offences in the Crimes Act. The concept of an "intimate visual recording" was introduced in the early 2000s to deal with covert filming with hidden cameras. Parliament was responding to conduct known as "upskirting" and "downblousing" - forms of abuse overwhelmingly targeting women and girls in spaces where they had a reasonable expectation of privacy, such as bathrooms or changing rooms. The law therefore focused on whether a person had been secretly recorded. But deepfakes complicate that framework because no recording may have occurred at all. This means current law may be clearer when an intimate image is real than when it is entirely fabricated. Beyond criminalisation The deepfakes bill attempts to remove that uncertainty by expanding the legal definition of an "intimate visual recording" to include images that are "created, synthesised or altered". But the bill also reflects a broader pattern in New Zealand's response to image-based sexual abuse: the law tends to evolve only after new technologies expose gaps in existing protections. First, it was hidden cameras and covert recordings. Then it was revenge porn. Now generative AI. Criminal law has been reactive and is not future-proofed enough to cover new technology-facilitated sexual harm. The rapid development of new technologies means criminalisation alone is unlikely to stop the spread of sexualised deepfakes. The tools to create deepfakes are cheap, fast and increasingly easy to access. A recent investigation by the Tech Transparency Project identified dozens of "nudify" and face-swap apps available through both the Apple and Google app stores. These apps can generate sexualised images from ordinary photographs within seconds. Despite app store policies prohibiting sexually explicit or degrading content, many of these tools remain readily accessible, often disguised as image-editing apps. Addressing regulation of high-risk apps Generative AI systems rely on enormous datasets scraped from the internet, frequently including images of women and girls used without their knowledge or consent. The result is that women's bodies are increasingly becoming both the raw material for AI systems and the targets of abuse generated by them. That is why New Zealand needs to think beyond criminal law and address regulation. Australia is moving to ban nudification apps and websites. So is the United Kingdom and the European Union. New Zealand should follow suit. More broadly, New Zealand should consider a regulatory framework for high-risk AI systems, particularly for technologies capable of generating non-consensual sexual content. This could include mandatory safety guardrails in image-generation systems, stronger obligations on app stores and platforms distributing these tools, and transparency requirements around AI training data. The Deepfake Digital Harm and Exploitation Bill is an important step forward and parliament should pass it. But if New Zealand wants to meaningfully address image-based sexual abuse in the age of generative AI, criminal law cannot be the end of the conversation.

FTC's Strict Anti-Deepfake Rule Kicks in Today. Here's What That Means for Grok

Roughly one year ago today, President Trump signed into law the Take It Down Act (TIDA), a first-of-its-kind federal law aimed at curbing the proliferation of nonconsensual intimate photos and videos online. Now, starting on Tuesday, the Federal Trade Commission will finally begin enforcing one of its key provisions, Section 3. When it was enacted into law, TIDA allowed the prosecution of anyone who knowingly distributes deepfakes or intimate images of any minor or non-consenting adult. Last month, a 37-year-old man from Columbus, Ohio, became "the first person in the United States to be convicted under the Take It Down Act," in the words of U.S. Attorney Dominick S. Gerace II. The perpetrator used more than 120 AI platforms and models to create and distribute sexual images of minors and non-consenting adult women on various digital platforms. Though the law has been in effect for a year now, the FTC gave digital platforms operators until May 19, 2026, to comply with Section 3, which holds the platforms partially liable for the distribution of such images. That deadline has now arrived. Specifically, Section 3 of the Act requires digital platforms to establish a clear and easy process for victims of nonconsensual intimate imagery to request the removal of photos or videos and any identical copies within 48 hours. Victims won't be required to find and report each identical copy themselves. The FTC is also requiring the platform operators to make the reporting process available for everyone, regardless of whether or not the victim has an existing account on the platform. Once a request is submitted, the victims should be provided with an identifying number to track their request, which they can also provide to law enforcement. "The FTC will vigorously enforce TIDA," FTC Chairman Andrew Ferguson wrote in a letter sent to more than a dozen tech companies earlier this month, threatening potential civil penalties of $53,088 per violation. The recipients of that letter included Amazon, Apple, Alphabet, Bumble, Match Group, Discord, Meta, Microsoft, Reddit, TikTok, and X, which has been embroiled in controversy over the past few months due to the rampant spread of nonconsensual deepfakes on the platform. Victims of online sexual abuse have been advocating for tougher regulation for years, even before the radical advances in AI image and video generation compounded the problem. TIDA is a welcome step in that direction, but its critics are worried that it won't be an effective solution. One reason is that since the Act became law a year ago, nonconsensual sexual deepfakes haven't gone anywhere, even becoming trendier on some platforms. Elon Musk's Grok chatbot has become the notorious poster child of this online epidemic. Late last year, X users started a disturbing trend where they used Grok to generate nonconsensually undressed or otherwise sexualized photos, often of women and children. Musk engaged with the trend, reposting photos of sexualized toaster machines on his own X profile before seemingly trying to make money off of the freshly popular feature. Though TIDA could not yet hold Musk and his company liable for these photos, xAI got into considerable trouble abroad, where such regulations already existed. Even once Section 3 is enforced, though, the bill might not be the most effective at curbing the negative impact of sexual deepfakes. "The reality, though, is that this means that the burden of identifying, documenting, and reporting harmful AI-generated content still lies with the victims themselves," Columbia University PhD candidate Kaylee Williams told Gizmodo. "Research shows that the reporting process can expose victims to immense trauma, on top of the adverse consequences they're likely to face as a result of the initial abuse." Williams is also unsure whether the major social media companies "will make a good faith effort to comply with the new rules." "At the end of the day, the question is whether these fines will provide enough incentive for the companies to build digital infrastructures to make a meaningful difference for victims, or simply absorb the costs of noncompliance, as many have done in the past," Williams said. "Abusive content is now easier to make and distribute than it has ever been before, so policing this harm at scale will be both costly and laborious, both for the platforms and the FTC." The TIDA penalties are high considering that it's per violation, but are considerably less harsh than those imposed by a similar online safety law enacted in the United Kingdom, where companies can be fined up to 10% of their qualifying worldwide revenue. For a company like Meta, for example, that could be worth millions of dollars. Other critics are worried that the Act will pose a significant threat to online privacy. "The Take It Down Act gives the powerful a dangerous new route to manipulate platforms into removing lawful speech that they simply don't like," digital rights non-profit Electronic Frontier Foundation's director of federal affairs India McKinney told Gizmodo. "Lawmakers should have focused on strengthening and enforcing existing legal protections for victims, rather than inventing new takedown regimes that are ripe for abuse." The concern is that, due to the 48-hour turnaround, digital platforms could choose to solely rely on the report in deciding to take down certain posts. Bad actors could abuse this as a loophole to take down anything they don't want to see on the internet, including posts by government critics. "President Trump himself has said that he would use the law to censor his critics, and that’s exactly the kind of abuse the public should fear," McKinney said. When the bill had just passed the Senate and was still awaiting a House vote in March 2025, Trump told a joint session of Congress that he was eagerly looking forward to it. "I'm going to use that bill for myself too if you don't mind, because nobody gets treated worse than I do online, nobody," Trump said.

US enforces law to crack down on sexual deepfakes

Washington (United States) (AFP) - The United States on Tuesday began enforcing a law requiring tech platforms to remove sexual deepfakes and other non-consensual intimate imagery, but experts warned of shortcomings and raised online censorship concerns. President Donald Trump signed the Take It Down Act last year, criminalizing the online distribution of non-consensual sexual imagery that is often created using cheap and widely available artificial intelligence tools. The Federal Trade Commission said that starting Tuesday tech platforms were required to establish a process allowing victims to request the removal of such content and must take it down within 48 hours of receiving a valid request -- or face penalties. "We stand ready to monitor compliance, investigate violations, and enforce the Take It Down Act," FTC Chairman Andrew Ferguson said after sending letters to over a dozen tech firms -- including Meta, TikTok, X, and Snapchat -- ahead of the enforcement. "Protecting the vulnerable -- especially children -- from this harmful abuse is a top priority for this agency and this administration." In a post on Monday, X's Safety account said there is "no place in our society for predators to share intimate photos and videos of others without their consent." "X has zero tolerance for non-consensual intimate images, unwanted sexual content, or any kind of exploitative behavior." 'Shoot first, ask questions never' The assurance comes after Elon Musk's AI tool Grok faced international backlash earlier this year for virtually undressing women and minors without their consent. Researchers said Grok generated an estimated three million sexualized images in a matter of days. Still, some experts warn the Take It Down Act is far from a perfect solution. Riana Pfefferkorn, a policy fellow at the Stanford Institute for Human-Centered Artificial Intelligence, said that the law creates a "shoot first, ask questions never" dynamic. The incentive structure is in the "direction of just remove it, remove it, remove it," she was quoted as saying in Indicator, a newsletter focused on investigating digital deception. Pfefferkorn also expressed concern that the law could be weaponized against trans people, sex workers whose content is consensual, and political speech the administration disagrees with. The law's takedown provision has also alarmed other free speech advocates, who warn it could encourage tech firms to zealously moderate non-offending content to reduce risk. An online boom in non-consensual deepfakes is currently outpacing efforts to regulate the technology around the world due to a proliferation of AI tools, including "nudification" apps, experts say. While high-profile politicians and celebrities, including singer Taylor Swift, have been victims of deepfake porn, researchers say women not in the public eye are equally vulnerable. A wave of AI porn scandals have been reported at schools across US states -- from California to New Jersey -- with hundreds of teenagers targeted by their own classmates. Such non-consensual imagery can lead to harassment, bullying or blackmail, sometimes causing devastating mental health consequences, experts warn.

"Not without my consent": how Europe plans to ban non-consensual "nudifier apps"

The European Parliament and Council outlaw "nudifier apps". Starting 2 December, the ban targets companies developing AI systems for sexual deepfakes and users creating false intimate content of real people without consent. On 7 May, co-legislators agreed to ban "nudifier apps" under the Digital Omnibus on AI. These controversial tools can generate AI-created sexual images or videos that "undress" individuals without consent, raising concerns over privacy and ethical use of technology. New-generation AI makes the creation of synthetic content increasingly affordable and realistic. About 8 million deepfakes were online in 2025, with 90 per cent of online content set to be AI-generated by 2026, the European Parliament Research Service found. So far, EU law has addressed deepfakes indirectly by treating them as violations of privacy and transparency, sparking calls for stronger protection under an outright EU-wide ban. "[...], there was perceived to be a lacuna in the law in addressing them [deepfakes]. That's why the Omnibus was seen as an opportunity to address that", said MEP Michael McNamara from Renew Europe in the European Parliament and co-rapporteur for the Civil Liberties, Justice and Home Affairs committee. While co-legislators still need to formally greenlight their position, Europe is already giving a clear signal: "nudifier apps" are a serious form of sexual digital abuse that must be banned before they enter the market. "Nudification" apps like "undressers", "nudify tools", and "clothes removers" manipulate ordinary photos to create increasingly realistic, sexually explicit images ("deepfakes") using generative AI. According to current estimates, as many as 96 percent were created without the subject's knowledge. These tools use deep-learning models, image recognition, and body reconstruction technology to synthesise realistic-looking images based on the lighting, pose, and skin tone of the original photo. They essentially trace or deduce the subject's shape through their clothes and invent a nude body that matches. This technology is accessible through many mobile applications - the App Store used to host apps like "DeepNude" and "ClothOff" - specialised websites, and automated bots on platforms like Telegram, frequently marketed as "AI art" or entertainment. Elon Musk's X made it extremely easy to access these apps; by early January 2026, chatbot Grok was creating approximately 6.700 sexualised images per hour, dozens of which involved children. The European Commission launched a formal investigation into Grok's digital safety laws, and the backlash caused X to implement restrictions. Now, generating images of real people in "revealing clothing" is blocked in some countries. The feature is limited to paying subscribers - though users on the site still find ways to trick the AI using prompts. These tools turn generative AI into a targeted instrument for harassment and digital abuse, stripping away the subject's bodily autonomy. Non-consensual explicit content directly violates fundamental privacy rights and undermines digital ethical standards. These programs are increasingly categorised as predatory technologies. "There are certain practices that are not jokes. It's about people. And in this battle, dignity should always be on the winning side", said German Greens MEP Sergey Lagodinsky. These tools cause psychological distress and reputational damage to victims: a 2026 UNICEF study across 11 countries revealed that at least 1.2 million children had their image manipulated into sexual deepfakes in 2025. A study from the same year noted that threats to post non-consensual, sexually explicit media increase the odds of suicide plans, attempts, and self-harm. The tools also disproportionately target women and girls, who represent 99percent of deepfake victims. They fuelled a 26.385 percent increase in generated child sexual abuse imagery (CSAM) since 2024 and an increase crimes like sextortion and blackmail. The 7 May provisional agreement targets AI creators across the board, banning any system specifically designed to generate this kind of content (including images, video, and audio). It makes developers of large-scale AI models directly responsible if their systems are used to create non-consensual nude images. These companies must now build permanent safety blocks into their core software to stop users from generating it. It also forbids realistic depictions of intimate parts and sexually explicit acts. The ban will affect providers, any company that places these systems on the EU market or offers them to people in the bloc.It affects organisations that use or host these softwares and allow non-consensual explicit content, and, crucially, users caught exploiting the AI to generate this kind of content. The EU is shifting the primary burden of responsibility away from individual end users and onto the companies building the models. As Lagodinsky said, "we cannot enforce human behaviour here. So, we are going against the technology itself." Providers are now forced to assess any "foreseeable misuse" of their technology before it reaches the public. They must implement measures to prevent users from bypassing filters with clever prompts or minor image alterations. To ensure compliance, the AI Office will monitor whether these safeguards are integrated into the model's core architecture. "Platforms would restrict access to certain prompts and to certain conduct, just like ChatGPT or Grok already say certain requests are not permissible", said Lagodinsky. "Legislative processes are much slower than innovation. We will only be able to cope with this if we have a principled way of regulating based on risk. That's why, for example, there are possibilities for the Commission to add certain new technologies as risky technologies in the AI Act", he added. If a company fails to implement these rules, it faces severe enforcement actions under the AI Act's framework. Penalties for non-compliance are hefty, with fines reaching up to €35 million or 7 percent of a company's total global turnover. The agreement also empowers national authorities to pull unsafe AI products from the EU market entirely. By 2 December 2026, all providers must prove their systems meet these safety standards or face financial sanctions. This oversight applies to both EU-based firms and international developers who offer their AI services to European residents. "I don't think policymakers necessarily underestimated it [the AI evolution]. Certainly, there has been a big lack of legal certainty until now", said McNamara. Prior to the ban, the EU primarily labelled deepfakes and "nudification" as content issues and violations of privacy and transparency. While no single law specifically prohibited these activities, a mix of regulations on data protection, image rights, privacy, and platform liability regulated them as general-purpose AI (GPAI) or limited-risk systems. "One the key contentions has been [...] whether to regulate or not to regulate [AI-generated content] with the current administration of the United States advocating a hands-off approach", McNamara told Euronews. The Digital Services Act (DSA) is a key regulation for online platforms. Yet it serves as a reactive tool, requiring Very Large Online Platforms (VLOPs) to address illegal content and misinformation by removing deepfakes only after they become aware of them. It also mandates deepfake-notification mechanisms and other measures to mitigate systemic risks arising from their platforms. The AI Act's current deepfake rules do not prevent the creation or sharing of non-consensual images. They only require providers to disclose the use of AI in online content creation and users to clearly label synthetic content. Redress for victims is also not foreseen. The act handles non-consensual intimate images (NCII) and child sexual abuse material (CSAM) in terms of transparency, allowing significant discretion to the provider under the GPAI Code of Practice. The General Data Protection Regulation (GDPR) is a general privacy regulation, not specifically tailored to synthetic content. It addresses the unlawful processing of individuals' data without prohibiting deepfakes in their existing form or creating victim-centred remedies. Non-consensual intimate images lead to humiliation and reputational harm, which require remedies beyond data protection, including harassment, defamation, and criminal law. The EU's 2024/1385 directive on online and offline violence against women criminalises technology-facilitated gender-based violence (TFGBV). These include digital tools to monitor, harass, and silence women and girls. While guaranteeing legal protections against deepfake sexual content, the text does not specifically target nudifier apps.

Big Donald Trump order on AI, cybersecurity coming soon as White House begins crackdown on deepfakes

Donald Trump on AI: This comes as the United States on Tuesday began enforcing a law requiring tech platforms to remove deepfakes and other non-consensual intimate imagery. The White House plans to release an executive order ⁠on ⁠cybersecurity and artificial intelligence as soon as this week, Axios ⁠reported on Tuesday, citing ⁠sources familiar with the matter. The order outlines plans for a voluntary framework for AI developers to inform ⁠the government about new releases, Axios reported, citing a readout, as per Reuters. This comes as the United States on Tuesday began enforcing a law requiring tech platforms to remove sexual deepfakes and other non-consensual intimate imagery, but experts warned of shortcomings and raised online censorship concerns. President Donald Trump signed the Take It Down Act last year, criminalizing the online distribution of non-consensual sexual imagery that is often created using cheap and widely available artificial intelligence tools. The Federal Trade Commission said that starting Tuesday tech platforms were required to establish a process allowing victims to request the removal of such content and must take it down within 48 hours of receiving a valid request -- or face penalties. "We stand ready to monitor compliance, investigate violations, and enforce the Take It Down Act," FTC Chairman Andrew Ferguson said after sending letters to over a dozen tech firms -- including Meta, TikTok, X, and Snapchat -- ahead of the enforcement. "Protecting the vulnerable -- especially children -- from this harmful abuse is a top priority for this agency and this administration." In a post on Monday, X's Safety account said there is "no place in our society for predators to share intimate photos and videos of others without their consent." "X has zero tolerance for non-consensual intimate images, unwanted sexual content, or any kind of exploitative behavior." - 'Shoot first, ask questions never' - The assurance comes after Elon Musk's AI tool Grok faced international backlash earlier this year for virtually undressing women and minors without their consent. Researchers said Grok generated an estimated three million sexualized images in a matter of days. Still, some experts warn the Take It Down Act is far from a perfect solution. Riana Pfefferkorn, a policy fellow at the Stanford Institute for Human-Centered Artificial Intelligence, said that the law creates a "shoot first, ask questions never" dynamic. The incentive structure is in the "direction of just remove it, remove it, remove it," she was quoted as saying in Indicator, a newsletter focused on investigating digital deception. Pfefferkorn also expressed concern that the law could be weaponized against trans people, sex workers whose content is consensual, and political speech the administration disagrees with. The law's takedown provision has also alarmed other free speech advocates, who warn it could encourage tech firms to zealously moderate non-offending content to reduce risk. An online boom in non-consensual deepfakes is currently outpacing efforts to regulate the technology around the world due to a proliferation of AI tools, including "nudification" apps, experts say. While high-profile politicians and celebrities, including singer Taylor Swift, have been victims of deepfake porn, researchers say women not in the public eye are equally vulnerable. A wave of AI porn scandals have been reported at schools across US states -- from California to New Jersey -- with hundreds of teenagers targeted by their own classmates. Such non-consensual imagery can lead to harassment, bullying or blackmail, sometimes causing devastating mental health consequences, experts warn.