EU moves to ban AI nudify apps after Grok made them mainstream, targeting platforms over users

Musk's tactic of blaming users for Grok sex images may be foiled by EU law

The European Union may soon ban nudify apps after Elon Musk's chatbot Grok emerged as a prime example of the dangers of an AI platform failing to block outputs that sexualized images of real people, including children. In a joint press release, the European Parliament's Internal Market and Civil Liberties committees confirmed that lawmakers voted 101-9 (with 8 abstentions) to simplify the Artificial Intelligence Act and "propose bans on AI 'nudifier' systems." The vote came after the European Commission concluded earlier this year that the AI Act does not prohibit "AI systems that generate child sexual abuse material (CSAM) or sexually explicit deepfake nudes." At that time, the Commission signaled that Parliament members were already proposing ways to amend the law to strengthen protections against such harmful content. If the amendment passes, which seems likely, it would foil Elon Musk's plan to blame users for harmful outputs. Earlier this year, xAI declined to introduce safeguards to block outputs, vowing to suspend and hold users legally accountable for any CSAM or non-consensual intimate imagery they generate. Instead, the feature was paywalled, limited to subscribers who could reportedly continue generating explicit content without the consent of real people whose images were fed into Grok. In the US, xAI has seemingly faced few consequences for Grok's outputs, but had the Take It Down Act been in play -- it takes effect in May -- the company could have risked billions in fines. It's possible that Musk's tactic of paywalling the feature and blocking Grok from spouting harmful outputs in response to prompts on X was intended to mitigate some of that risk ahead of that law's enforcement. But if the EU bans nudify apps, perhaps as early as August, Musk would finally be forced to intervene, fine-tuning Grok to be less "spicy" than Musk likely wants or else risking violating the AI Act. That could cost xAI too much at a time when competing with its biggest rivals in the AI race demands substantial investments, with possible fines of up to 7 percent of its total worldwide annual turnover. Why officials want to go after platforms, not users Officials "want to introduce a new ban on so-called 'nudifier' systems that use AI to create or manipulate images that are sexually explicit or intimate and resemble an identifiable real person without that person's consent," the press release explained. However, "the ban would not apply to AI systems with effective safety measures preventing users from creating such images," officials said. As Bloomberg noted, the ban would radically shift the EU's approach to regulating explicit deepfakes, moving beyond just prosecuting users to also punishing platforms. The Grok scandal "epitomized" why such a regulatory shift was needed, Bloomberg reported, noting that "this amendment is the first" EU policy "to specifically target AI platforms" that produce and allow sharing of "sexual material without the subject's consent." While EU officials did not directly mention Grok in the press release, regulators had already been probing the AI system while pondering the implications of xAI's controversy for other, less visible nudify apps. Submitting questions to the European Commission earlier this year, lawmakers warned: Recent shocking reports of AI-powered nudity applications, such as Grok on X, but also other tools that are freely available online, highlight an increase in AI-driven tools that allow users to generate manipulated intimate images of individuals without their consent, facilitating gender-based cyberviolence and the creation of child sexual abuse material. "These systems should be banned from the EU market," lawmakers urged, particularly since "individual perpetrators" -- who "can often be punished under national criminal law" -- "are often hard to find." A more proactive plan, lawmakers suggested, would be to "prevent widespread image-based sexual violence from the outset." With apparent backing from Parliament members, the amendment's likely passage is sure to frustrate Musk, who is also facing legal challenges in the US seeking injunctions against Grok's nudify outputs. In January, a mother of one of Musk's children, Ashley St. Clair, became one of the first victims to file a lawsuit. And more recently, three young girls in Tennessee filed a proposed class action representing all children harmed by Grok's alleged CSAM outputs. In the EU, similar public pressure is mounting for regulators to intervene, as xAI seems unwilling prevent Grok from undressing real people. A civil liberties committee member, Michael McNamara, said in the press release that he believes the proposal to ban nudify apps "is something that our citizens expect."

EU Moves to Ban AI That Creates Non-Consensual Sexual Images

The prohibition targets AI technology, in addition to its users, due to improvements in models' capabilities to produce convincing likenesses of real people, as seen in the case of Elon Musk's Grok service. Artificial intelligence tools that can be used to create nonconsensual sexual images may be banned in the European Union, following outcry after users used Elon Musk's Grok service to generate thousands of undressed pictures of women and children. The European Parliament's civil liberties committee on Wednesday approved the bloc's draft text of its Artificial Intelligence Omnibus law. That included an amendment to outlaw any AI system that generates realistic images "so as to depict sexually explicit activities or the intimate parts of an identifiable natural person" without their consent. The ban wouldn't apply if a company has imposed measures restricting the creation of such deepfakes. The amendment's approval puts the parliament in line with European governments, who agreed on a similar ban, making it likely the prohibition will be approved and put into law later this year. While Europe has a number of rules that make it illegal to produce and share sexual material without the subject's consent, this amendment is the first to specifically target AI platforms. Lawmakers opted to target AI technology besides its users, due to swift improvements in models' capabilities to produce convincing likeness of real people. That shift was epitomized by the way Grok -- a chatbot accessible to anyone using Musk's X social network -- was used to generate and publish online vast amounts of sexualized images based on pictures of fully clothed people in January. Parent company xAI Corp. restricted the feature after widespread criticism. A representative for xAI did not immediately reply to a request for comment. The prohibition adds to a growing corpus of EU laws targeting child abuse sexual material and nonconsensual intimate image sharing. A 2024 directive on violence against women made it an offense to use AI to produce sexual images without the subject's consent, and the Digital Services Act also punishes social networks that allow the spread of illegal content, including child abuse material. If the ban is adopted into EU law, developers of advanced AI capable of generating all sorts of audiovisual material will have to prove they have set up restrictions on the images their models create. It is unclear how the AI systems would verify whether the person portrayed in an image has given their consent or not. The EU is working to pass the AI Omnibus, a comprehensive regulation, that aims to simplify the EU's sweeping Artificial Intelligence Act. The omnibus is also expected to delay implementation of parts of the AI Act on high-risk applications, originally scheduled to kick in in August 2026, and now pushed to December 2027 and August 2028. That will allow specialist organizations to draft detailed guidance on how to comply with the EU's AI law, and give companies more clarity. Regulators in the EU and the UK are formally investigating X and xAI over the Grok incident in January to establish whether it had breached laws on content moderation and online safety.

EU lawmakers support ban on AI apps generating explicit images

BRUSSELS, March 18 (Reuters) - Key EU lawmakers, who are influential in shaping changes to Europe's AI Act, on Wednesday backed a ban on AI apps which create unauthorised sexually explicit images and urged that the bloc's landmark rules agreed two years ago include this ban. The move comes a week after EU countries voted on a similar proposal ahead of talks between the lawmakers and EU governments ⁠on the European Commission's plan to water down parts of the AI Act so as not to lose ground in the global tech race. The European Parliament will vote on the group's proposal on March 26. Lawmakers and EU governments will then have to thrash out their differences before the changes proposed by the EU executive can be implemented. "A proposal to ban so-called nudification apps I believe ⁠is something that our citizens expect of the co-legislators," said lawmaker Michael McNamara, who is leading the forthcoming discussion with EU countries. Sexually explicit content generated by Elon Musk's xAI chatbot Grok on X, as ⁠well as sexually intimate deepfakes produced by Grok, have triggered a crackdown by governments and regulators from Europe to Asia. The lawmakers also voted to delay ⁠certain rules on high-risk AI systems to December 2 next year, saying that key standards may not be finalised by ⁠August 2, which could create uncertainty for companies. This was in line with EU countries' position. The AI Act will be implemented in phases. Reporting by Foo Yun Chee Editing by Bernadette Baum Our Standards: The Thomson Reuters Trust Principles., opens new tab * Suggested Topics: * Litigation * Data Privacy * Public Policy Foo Yun Chee Thomson Reuters An agenda-setting and market-moving journalist, Foo Yun Chee is a 21-year veteran at Reuters. Her stories on high profile mergers have pushed up the European telecoms index, lifted companies' shares and helped investors decide on their next move. Her knowledge and experience of European antitrust laws and developments helped her break stories on Microsoft, Google, Amazon, Meta and Apple, numerous market-moving mergers and antitrust investigations. She has previously reported on Greek politics and companies, when Greece's entry into the eurozone meant it punched above its weight on the international stage, as well as on Dutch corporate giants and the quirks of Dutch society and culture that never fail to charm readers.

Europe takes first step to banning AI-generated child sexual abuse images

BRUSSELS, March 13 (Reuters) - Europe on Friday took the first step towards outlawing artificial intelligence practices which generate child sexual abuse material after EU governments proposed to add this provision to the bloc's landmark AI rules adopted two years ago. Governments and regulators from Europe to Asia are cracking down on sexually explicit ⁠content generated by Elon Musk's xAI chatbot Grok on X as well as sexually intimate deepfakes produced by Grok. EU tech regulators and national watchdogs in Britain, Ireland and Spain are currently investigating Grok's sexualised AI deepfakes. The EU countries will need the backing of the European Parliament before their proposal can ⁠be adopted. Lawmakers are scheduled to vote on their own similar proposition on Wednesday. Both sides have to stake out their positions on the matter and other ⁠issues ahead of negotiations on the European Commission's proposal to water down parts of the AI Act, a move welcomed ⁠by tech giants and some businesses but criticised by civic groups and privacy campaigners for ⁠bowing to Big Tech. The discussions will likely take a year before any changes can be implemented. Reporting by Foo Yun Chee; Editing by Kirsten Donovan Our Standards: The Thomson Reuters Trust Principles., opens new tab * Suggested Topics: * Artificial Intelligence * Public Health Foo Yun Chee Thomson Reuters An agenda-setting and market-moving journalist, Foo Yun Chee is a 21-year veteran at Reuters. Her stories on high profile mergers have pushed up the European telecoms index, lifted companies' shares and helped investors decide on their next move. Her knowledge and experience of European antitrust laws and developments helped her break stories on Microsoft, Google, Amazon, Meta and Apple, numerous market-moving mergers and antitrust investigations. She has previously reported on Greek politics and companies, when Greece's entry into the eurozone meant it punched above its weight on the international stage, as well as on Dutch corporate giants and the quirks of Dutch society and culture that never fail to charm readers.

EU backs ban on AI generating sexualised deepfakes after Grok backlash

EU states on Friday supported new rules banning AI tools that generate sexualised deepfakes, following a backlash over images made with Elon Musk's AI chatbot Grok. EU nations on Friday backed a ban on AI systems generating sexualised deepfakes, after an outcry over such images produced by Elon Musk's AI chatbot Grok. European ambassadors agreed to prohibit "practices regarding the generation of non-consensual sexual and intimate content or child sexual abuse material", a spokesperson for Cyprus, which holds the EU's rotating presidency, said. EU member states and the European Parliament introduced the ban as part of the proposals to amend the bloc's comprehensive rules on AI. EU lawmakers are set to approve the ban during a vote by committees Wednesday. X, the platform on which Grok is available, in January said it had "zero tolerance" for sexualised deepfakes of children and women, and implemented measures it said would stop such image creation after the global outrage. Read moreFrance's Macron calls Musk 'over-subsidised,' urges Europe to support domestic industry The European Commission, the bloc's digital watchdog, in January kickstarted an investigation into Grok under the EU's online content rules. The ban will become law after negotiations on a final text including the changes to the AI rulebook between the EU parliament and member states. Ambassadors on Friday also approved a fixed timeline for the delayed application of high-risk AI rules: December 2027 for stand-alone high-risk AI systems and August 2028 for high-risk AI systems embedded in products. Rules on high-risk AI are due to come into effect in August 2026 and August 2027 unless the delay becomes law.

EU lawmakers support ban on AI apps generating explicit images

Key EU lawmakers are pushing for a ban on AI apps that generate unauthorized explicit images. This move aims to strengthen Europe's AI Act. The European Parliament will vote on the proposal next month. Lawmakers and EU governments will then negotiate differences. This comes after concerns over AI-generated explicit content and a potential watering down of the AI Act. Key EU lawmakers, who are influential in shaping changes to Europe's AI Act, on Wednesday backed a ban on AI apps which create unauthorised sexually explicit images and urged that the bloc's landmark rules agreed two years ago include ⁠this ⁠ban. The move comes a week after EU countries voted on a similar proposal ahead of talks between the lawmakers and EU governments on the European Commission's plan to water down parts of the AI Act so as not to lose ground in the global tech race. The European ⁠Parliament will vote on the group's proposal on March 26. Lawmakers and EU governments will then have ⁠to thrash out their differences before the changes proposed by the EU executive can be implemented. "A proposal to ban so-called nudification apps I believe is something that our citizens expect of the co-legislators," said lawmaker Michael McNamara, who is leading the forthcoming discussion with EU countries. Sexually explicit content generated by Elon Musk's xAI chatbot Grok on X, as well as sexually intimate deepfakes produced by Grok, have triggered a crackdown by ⁠governments and regulators from Europe to Asia. The lawmakers also voted to delay certain rules on high-risk AI systems to December 2 next year, saying that key standards may not be finalised by August 2, which could create uncertainty for companies. This was in line with EU countries' position. The AI Act will be implemented in phases.

Europe takes first step to banning AI-generated child sexual abuse images

Governments and regulators from Europe to Asia are cracking down on sexually explicit content generated by Elon Musk's xAI chatbot Grok on X as well as sexually intimate deepfakes produced by ⁠Grok. Europe on Friday took the first step towards outlawing artificial intelligence practices which generate child sexual abuse material after EU governments proposed to add this provision to the bloc's ⁠landmark ⁠AI rules adopted two years ago. Governments and regulators from Europe to Asia are cracking down on sexually explicit content generated by Elon Musk's xAI chatbot Grok on X as well as sexually intimate deepfakes produced by ⁠Grok. EU tech regulators and national watchdogs in Britain, Ireland and Spain are currently investigating ⁠Grok's sexualised AI deepfakes. The EU countries will need the backing of the European Parliament before their proposal can be adopted. Lawmakers are scheduled to vote on their own similar proposition on Wednesday. Both sides have to stake out their positions on the matter and other issues ahead of negotiations on the European Commission's proposal to water ⁠down parts of the AI Act, a move welcomed by tech giants and some businesses but criticised by civic groups and privacy campaigners for bowing to Big Tech. The discussions will likely take a year before any changes can be implemented.