New Zealand and Europe crack down on sexualized deepfakes with sweeping new legislation

NZ is criminalising sexualised deepfakes - banning apps that make them should be next

New Zealand is changing the law to make sexualised deepfakes a crime. But this alone may not be enough to counter the rise in AI-generated fake sexual material. This week the Deepfake Digital Harm and Exploitation Bill is set to go through its first reading, with support across the political spectrum. The amendment will make creating, sharing or selling sexually explicit deepfakes without consent a criminal offence. It comes in response to the rapid spread of such material, including the rollout of Elon Musk's Grok AI chatbot on X, which people used to digitally undress women and girls and generate potentially three million sexualised images. New Zealand is not alone in confronting the problem. The United Kingdom, Australia, South Korea and the United States have already introduced or expanded laws to criminalise creating and sharing of non-consensual deepfakes. Criminalisation is an important first step and brings New Zealand in line with developments elsewhere. But stemming the tide of sexualised deepfakes will also require regulation of the technology itself. Deepfakes mostly target women Deepfakes are AI-generated images, audio or video designed to make it appear that a real person said or did something that never actually happened. With image-based sexual abuse, this usually means using AI to create convincing but fake sexual material of a person without their consent. Sometimes, technology is used to manipulate ordinary photos pulled from social media into explicit imagery. Other AI systems can generate entirely fabricated sexual content from a text prompt alone. The abuse is overwhelmingly directed at women. One widely cited study found 98% of deepfake videos online were pornographic and mostly targeted women. For victim-survivors, the harms are significant regardless of whether the image is created from a real original or completely fabricated. People report humiliation, fear, anxiety, loss of control and violation of sexual autonomy. Gaps in New Zealand law The new bill is designed to fill gaps in the current law. At present, New Zealand does not have a criminal offence specifically directed at sexualised deepfakes. Existing laws may apply, but they were not designed for AI-generated abuse. The Harmful Digital Communications Act 2015 criminalises posting harmful digital communications and has already been used in at least one prosecution involving a sexualised deepfake. However, the offence requires proof that the defendant intended to cause serious emotional distress and that this actually resulted. Those requirements can be difficult hurdles for victim-survivors of image-based sexual abuse. A new offence introduced into the act in 2022 sought to address this for so-called "revenge porn", making it a crime to share an intimate visual recording without consent. This sat alongside existing offences in the Crimes Act. The concept of an "intimate visual recording" was introduced in the early 2000s to deal with covert filming with hidden cameras. Parliament was responding to conduct known as "upskirting" and "downblousing" - forms of abuse overwhelmingly targeting women and girls in spaces where they had a reasonable expectation of privacy, such as bathrooms or changing rooms. The law therefore focused on whether a person had been secretly recorded. But deepfakes complicate that framework because no recording may have occurred at all. This means current law may be clearer when an intimate image is real than when it is entirely fabricated. Beyond criminalisation The deepfakes bill attempts to remove that uncertainty by expanding the legal definition of an "intimate visual recording" to include images that are "created, synthesised or altered". But the bill also reflects a broader pattern in New Zealand's response to image-based sexual abuse: the law tends to evolve only after new technologies expose gaps in existing protections. First, it was hidden cameras and covert recordings. Then it was revenge porn. Now generative AI. Criminal law has been reactive and is not future-proofed enough to cover new technology-facilitated sexual harm. The rapid development of new technologies means criminalisation alone is unlikely to stop the spread of sexualised deepfakes. The tools to create deepfakes are cheap, fast and increasingly easy to access. A recent investigation by the Tech Transparency Project identified dozens of "nudify" and face-swap apps available through both the Apple and Google app stores. These apps can generate sexualised images from ordinary photographs within seconds. Despite app store policies prohibiting sexually explicit or degrading content, many of these tools remain readily accessible, often disguised as image-editing apps. Addressing regulation of high-risk apps Generative AI systems rely on enormous datasets scraped from the internet, frequently including images of women and girls used without their knowledge or consent. The result is that women's bodies are increasingly becoming both the raw material for AI systems and the targets of abuse generated by them. That is why New Zealand needs to think beyond criminal law and address regulation. Australia is moving to ban nudification apps and websites. So is the United Kingdom and the European Union. New Zealand should follow suit. More broadly, New Zealand should consider a regulatory framework for high-risk AI systems, particularly for technologies capable of generating non-consensual sexual content. This could include mandatory safety guardrails in image-generation systems, stronger obligations on app stores and platforms distributing these tools, and transparency requirements around AI training data. The Deepfake Digital Harm and Exploitation Bill is an important step forward and parliament should pass it. But if New Zealand wants to meaningfully address image-based sexual abuse in the age of generative AI, criminal law cannot be the end of the conversation.

"Not without my consent": how Europe plans to ban non-consensual "nudifier apps"

The European Parliament and Council outlaw "nudifier apps". Starting 2 December, the ban targets companies developing AI systems for sexual deepfakes and users creating false intimate content of real people without consent. On 7 May, co-legislators agreed to ban "nudifier apps" under the Digital Omnibus on AI. These controversial tools can generate AI-created sexual images or videos that "undress" individuals without consent, raising concerns over privacy and ethical use of technology. New-generation AI makes the creation of synthetic content increasingly affordable and realistic. About 8 million deepfakes were online in 2025, with 90 per cent of online content set to be AI-generated by 2026, the European Parliament Research Service found. So far, EU law has addressed deepfakes indirectly by treating them as violations of privacy and transparency, sparking calls for stronger protection under an outright EU-wide ban. "[...], there was perceived to be a lacuna in the law in addressing them [deepfakes]. That's why the Omnibus was seen as an opportunity to address that", said MEP Michael McNamara from Renew Europe in the European Parliament and co-rapporteur for the Civil Liberties, Justice and Home Affairs committee. While co-legislators still need to formally greenlight their position, Europe is already giving a clear signal: "nudifier apps" are a serious form of sexual digital abuse that must be banned before they enter the market. "Nudification" apps like "undressers", "nudify tools", and "clothes removers" manipulate ordinary photos to create increasingly realistic, sexually explicit images ("deepfakes") using generative AI. According to current estimates, as many as 96 percent were created without the subject's knowledge. These tools use deep-learning models, image recognition, and body reconstruction technology to synthesise realistic-looking images based on the lighting, pose, and skin tone of the original photo. They essentially trace or deduce the subject's shape through their clothes and invent a nude body that matches. This technology is accessible through many mobile applications - the App Store used to host apps like "DeepNude" and "ClothOff" - specialised websites, and automated bots on platforms like Telegram, frequently marketed as "AI art" or entertainment. Elon Musk's X made it extremely easy to access these apps; by early January 2026, chatbot Grok was creating approximately 6.700 sexualised images per hour, dozens of which involved children. The European Commission launched a formal investigation into Grok's digital safety laws, and the backlash caused X to implement restrictions. Now, generating images of real people in "revealing clothing" is blocked in some countries. The feature is limited to paying subscribers - though users on the site still find ways to trick the AI using prompts. These tools turn generative AI into a targeted instrument for harassment and digital abuse, stripping away the subject's bodily autonomy. Non-consensual explicit content directly violates fundamental privacy rights and undermines digital ethical standards. These programs are increasingly categorised as predatory technologies. "There are certain practices that are not jokes. It's about people. And in this battle, dignity should always be on the winning side", said German Greens MEP Sergey Lagodinsky. These tools cause psychological distress and reputational damage to victims: a 2026 UNICEF study across 11 countries revealed that at least 1.2 million children had their image manipulated into sexual deepfakes in 2025. A study from the same year noted that threats to post non-consensual, sexually explicit media increase the odds of suicide plans, attempts, and self-harm. The tools also disproportionately target women and girls, who represent 99percent of deepfake victims. They fuelled a 26.385 percent increase in generated child sexual abuse imagery (CSAM) since 2024 and an increase crimes like sextortion and blackmail. The 7 May provisional agreement targets AI creators across the board, banning any system specifically designed to generate this kind of content (including images, video, and audio). It makes developers of large-scale AI models directly responsible if their systems are used to create non-consensual nude images. These companies must now build permanent safety blocks into their core software to stop users from generating it. It also forbids realistic depictions of intimate parts and sexually explicit acts. The ban will affect providers, any company that places these systems on the EU market or offers them to people in the bloc.It affects organisations that use or host these softwares and allow non-consensual explicit content, and, crucially, users caught exploiting the AI to generate this kind of content. The EU is shifting the primary burden of responsibility away from individual end users and onto the companies building the models. As Lagodinsky said, "we cannot enforce human behaviour here. So, we are going against the technology itself." Providers are now forced to assess any "foreseeable misuse" of their technology before it reaches the public. They must implement measures to prevent users from bypassing filters with clever prompts or minor image alterations. To ensure compliance, the AI Office will monitor whether these safeguards are integrated into the model's core architecture. "Platforms would restrict access to certain prompts and to certain conduct, just like ChatGPT or Grok already say certain requests are not permissible", said Lagodinsky. "Legislative processes are much slower than innovation. We will only be able to cope with this if we have a principled way of regulating based on risk. That's why, for example, there are possibilities for the Commission to add certain new technologies as risky technologies in the AI Act", he added. If a company fails to implement these rules, it faces severe enforcement actions under the AI Act's framework. Penalties for non-compliance are hefty, with fines reaching up to €35 million or 7 percent of a company's total global turnover. The agreement also empowers national authorities to pull unsafe AI products from the EU market entirely. By 2 December 2026, all providers must prove their systems meet these safety standards or face financial sanctions. This oversight applies to both EU-based firms and international developers who offer their AI services to European residents. "I don't think policymakers necessarily underestimated it [the AI evolution]. Certainly, there has been a big lack of legal certainty until now", said McNamara. Prior to the ban, the EU primarily labelled deepfakes and "nudification" as content issues and violations of privacy and transparency. While no single law specifically prohibited these activities, a mix of regulations on data protection, image rights, privacy, and platform liability regulated them as general-purpose AI (GPAI) or limited-risk systems. "One the key contentions has been [...] whether to regulate or not to regulate [AI-generated content] with the current administration of the United States advocating a hands-off approach", McNamara told Euronews. The Digital Services Act (DSA) is a key regulation for online platforms. Yet it serves as a reactive tool, requiring Very Large Online Platforms (VLOPs) to address illegal content and misinformation by removing deepfakes only after they become aware of them. It also mandates deepfake-notification mechanisms and other measures to mitigate systemic risks arising from their platforms. The AI Act's current deepfake rules do not prevent the creation or sharing of non-consensual images. They only require providers to disclose the use of AI in online content creation and users to clearly label synthetic content. Redress for victims is also not foreseen. The act handles non-consensual intimate images (NCII) and child sexual abuse material (CSAM) in terms of transparency, allowing significant discretion to the provider under the GPAI Code of Practice. The General Data Protection Regulation (GDPR) is a general privacy regulation, not specifically tailored to synthetic content. It addresses the unlawful processing of individuals' data without prohibiting deepfakes in their existing form or creating victim-centred remedies. Non-consensual intimate images lead to humiliation and reputational harm, which require remedies beyond data protection, including harassment, defamation, and criminal law. The EU's 2024/1385 directive on online and offline violence against women criminalises technology-facilitated gender-based violence (TFGBV). These include digital tools to monitor, harass, and silence women and girls. While guaranteeing legal protections against deepfake sexual content, the text does not specifically target nudifier apps.