Indonesia and Malaysia block Grok as governments worldwide demand AI regulation over deepfakes

Indonesia and Malaysia block Grok over non-consensual, sexualized deepfakes | TechCrunch

Officials from Indonesia and Malaysia have said they are temporarily blocking access to xAI's chatbot Grok. These are the most aggressive moves so far from government officials responding to a flood of sexualized, AI-generated imagery -- often depicting real women and minors, and sometimes depicting violence -- posted by Grok in response to requests from users on the social network X. (X and xAI are part of the same company.) In a statement shared Saturday with the Guardian and other publications, Indonesia's communications and digital minister Meutya Hafid said, "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space." The ministry has also reportedly summoned X officials to discuss the issue. The New York Times said the Malaysian government announced a similar ban on Sunday. Varied governmental responses over the past week include an order from India's IT ministry for xAI to take action to prevent Grok from generating obscene content, as well as an order from the European Commission for the company to retain all documents related to Grok, potentially setting the stage for an investigation. In the United Kingdom, the communications regulator Ofcom has said that it will "undertake a swift assessment to determine whether there are potential compliance issues that warrant investigation." Prime Minister Keir Starmer said in an interview Ofcom has his "full support to take action." And while in the United States, the Trump administration appears to be staying silent on the issue (xAI CEO Elon Musk is a major Trump donor and led the administration's controversial Department of Government Efficiency last year), Democratic senators have called on Apple and Google to remove X from their app stores. xAI initially responded by posting a seemingly first-person apology to the Grok account, acknowledging that a post "violated ethical standards and potentially US laws" around child sexual abuse material. It later restricted the AI image-generation feature to paying subscribers on X, though that restriction did not appear to affect the Grok app itself, which still allowed anyone to generate images. In response to a post wondering why the U.K. government wasn't taking action against other AI image generation tools, Musk wrote, "They want any excuse for censorship."

Indonesia blocks Grok over non-consensual, sexualized deepfakes | TechCrunch

Indonesian officials said Saturday that they are temporarily blocking access to xAI's chatbot Grok. This is one of the most aggressive moves so far from government officials responding to a flood of sexualized, AI-generated imagery -- often depicting real women and minors, and sometimes showing assault and abuse -- posted by Grok in response to requests from users on the social network X. (X and xAI are part of the same company.) In a statement shared with the Guardian and other publications, Indonesia's communications and digital minister Meutya Hafid said, "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space." The ministry has also reportedly summoned X officials to discuss the issue. Varied governmental responses over the past week include an order from India's IT ministry for xAI to take action to prevent Grok from generating obscene content, as well as an order from the European Commission for the company to retain all documents related to Grok, which could be setting the stage for an investigation. In the United Kingdom, the communications regulator Ofcom has said that it will "undertake a swift assessment to determine whether there are potential compliance issues that warrant investigation." Prime Minister Keir Starmer said in an interview Ofcom has his "full support to take action." And while in the United States, the Trump administration appears to be staying silent on the issue (xAI CEO Elon Musk is a major Trump donor and led the administration's controversial Department of Government Efficiency last year), Democratic senators have called on Apple and Google to remove X from their app stores. xAI initially responded by posting a seemingly first-person apology to the Grok account, acknowledging that a post "violated ethical standards and potentially US laws" around child sexual abuse material. It later restricted the AI image-generation feature to paying subscribers on X, though that restriction did not appear to affect the Grok app itself, which still allowed anyone to generate images. In response to a post wondering why the U.K. government wasn't taking action against other AI image generation tools, Musk wrote, "They want any excuse for censorship."

Indonesia Bans Grok Amid Bikini Deepfake Controversy, UK Mulls Crackdown

Indonesia has become the first country to ban X's chatbot Grok amid recent controversy over the chatbot generating non-consensual, sexually suggestive deepfake images. The chatbot was hit with widespread criticism last week after it admitted to creating suggestive images of minors wearing bikinis. The pictures were part of a viral trend on the social media platform that saw numerous celebrities and public figures targeted, with some nonprofits alleging many of Grok's targets were underage. "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space," the country's Communications and Digital Minister Meutya Hafid said in a statement to Reuters. Indonesia, the world's fourth-most-populous country and the world's largest Muslim-majority nation, has strict pornography laws. Though Grok has been blocked, the wider X platform remains accessible. Though Indonesia is so far the only country to ban Grok outright, regulators in several other countries have voiced concerns or threatened potential action. Earlier this week, UK Prime Minister Keir Starmer said that Grok's generation of sexualized deepfakes was "unlawful." "This is disgraceful. It's disgusting," Starmer told Greatest Hits Radio. "And it's not to be tolerated." While a potential ban was not explicitly mentioned, government sources told BBC News they would expect Ofcom, the UK's media regulator, "to use all powers at its disposal in regard to Grok and X," adding that they had been in contact with both X and xAI. Meanwhile, European Union regulators said earlier this week they were "very seriously looking" into allegations that Elon Musk's AI tool Grok is being used to generate and share sexually explicit, childlike images. X recently warned users that anyone "using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content." It also restricted the chatbot's image generation capabilities for the majority of its users earlier this week, making them exclusive to Premium subscribers. There's plenty of political outcry being directed toward Grok domestically as well. Several Democratic senators have called on Apple and Google to pull the Grok app from their app stores, citing the controversy over the chatbot creating sexualized images of real people. Sens. Ron Wyden, Edward Markey, and Ben Ray Luján alleged that the chatbot's outputs violated both platforms' terms and conditions in a letter directed to Apple CEO Tim Cook and Google CEO Sundar Pichai.

Grok fallout: Tech giants must be held accountable for technology-assisted gender-based violence

The new image and video editing feature for xAI's chatbot, Grok, has generated thousands of non-consensual, sexually explicit images of women and minors since Grok announced the editing feature on Christmas Eve. It was promoted as enabling the addition of Santa Claus to photos. The growing ease of perpetrating sexual violence with novel technologies reflects the urgent need for tech companies and policymakers to prioritize AI safety and regulation. I am a PhD candidate in public health. My research has largely focused on the intersection of gender-based violence and health, previously working on teams that leverage AI as a tool to support survivors of violence. The potential and actual harms of AI on a such a wide scale require new regulations that will protect the health of mass populations. 'Nudifying' apps Concern about sexually explicit "deepfakes" has been publicly debated for some time now. In 2018, the public heard that Reddit threads profiled machine learning tools being used to face-swap celebrities like Taylor Swift onto pornographic material. Read more: Taylor Swift deepfakes: new technologies have long been weaponised against women. The solution involves us all Other AI-powered programs for "nudifying" could be found in niche corners of the internet. Now, this technology is easily accessible at anyone's fingertips. Grok can be accessed either through its website and app or on the social media platform, X. Some users have noted that when prompted to create pornographic images, Grok says it's programmed not to do this, but such apparent guardrails are being easily bypassed. xAI's owner, Elon Musk, released a statement via X that the company takes action against illegal content on X by removing it, "permanently suspending accounts, and working with local governments and law enforcement as necessary." However, it's unclear how or when these policies will be implemented. This is nothing new Technologies have long been used as a medium for sexual violence. Technology-facilitated sexual violence encompasses a range of behaviours as digital technologies are used to facilitate both virtual and face-to-face sexually based harms. Women, sexual minorities and minors are the most often victimized. One form of this violence that has received significant attention is "revenge porn" -- referring to the non-consensual distribution of an individual's images and videos on the internet. Victims have reported lifelong mental health consequences, damaged relationships and social isolation. Some social media websites have policies forbidding the distribution of non-consensual intimate content and have implemented mechanisms for reporting and removing such content. Search engines like Google and Bing will also review requests to remove links from search results if they're in violation of their personal content policies. Canada has criminalized "revenge porn" under the Criminal Code, which is punishable by up to five years in prison. Similar to revenge porn, victims of deepfakes have reported mental distress, including feelings of helplessness, humiliation and embarrassment, while some have even been extorted for money. Creators of sexually explicit deepfakes have also targeted prominent female journalists and politicians as a method of cyberbullying and censorship. Now what? This latest Grok controversy reflects a predictable major lapse in AI safeguards. Prominent AI safety experts and child safety organizations warned xAI months ago that the feature was "a nudification tool waiting to be weaponized." On Jan. 9, xAI responded by moving the image-editing feature behind a subscription for X users (though it can still be accessed for free on the Grok app) and has stopped Grok from automatically uploading the generated image to the comments. However, X users are still generating sexualized images with the Grok tab and manually posting them onto the platform. Some countries have taken action to block access to Grok. Looking to the future This isn't the first time, nor will it be the last time, a tech company demonstrates such a major lapse in judgment over their product's potential for user-perpetrated sexual violence. Canada needs action that includes: 1. Criminalizing the creation and distribution of non-consensual sexually explicit deepfakes. Legal scholars have advocated for the criminalization of creating and distributing non-consensual sexually explicit deepfakes, similar to existing "revenge porn" laws. 2. Regulate AI companies and hold them accountable. Canada has yet to pass any legislation to regulate AI, with the proposed Artificial Intelligence and Data Act and Online Harms Act dying when Parliament was prorouged in January 2025. Canada's AI minister referenced this in his response to these Grok issues, but the response lacks a dedicated timeline and a sense of urgency. Read more: Why Canada's reaction to the Grok scandal is so muted in the midst of a global outcry As AI progresses, major regulatory actions need to be taken to prevent further harms of sexual violence. Tech companies need to undergo thorough safety checks for their AI products, even if it comes at the expense of slowing down. It also raises questions about who should be responsible for the harms caused by the AI's outputs. Three American senators have called on Apple and Google to remove Grok from their app stores for its clear policy violations, citing the recent examples of these companies' abilities to promptly remove apps from their store. 3. Expand the scope of sexual violence social services to support those affected by non-consensual sexually explicit deepfakes. As the perpetration of sexual violence via AI technologies becomes more prevalent, sexual violence organizations can expand their scope to support those affected by non-consensual sexually explicit deepfakes. They can do so by leveraging existing services, including mental health care and legal supports. 4. Dismantle the underlying rape culture that perpetuate these forms of violence. The root of sexual violence is the dominance of rape culture, which is fostered in online environments where sexualized abuse and harassment is tolerated or encouraged. Dismantling rape culture requires holding perpetrators accountable and speaking out against behaviour that normalizes such behaviours.

MPs slam UK delay on banning AI nudification apps

Committee raises concerns over delays and loopholes in proposed law The Science, Innovation and Technology Committee has criticized the UK government's handling of AI nudification tools, saying it is taking too long to ban apps, and that expedited legislation does not encompass multi-purpose platforms used to create nude images. Grok, the AI chatbot owned and run by Elon Musk's xAI, caused controversy at the start of this year when users prompted it to create images of naked or barely dressed people - mostly women, some underage - from real photos. Over a 24-hour period between January 5 and 6, it generated 6,700 sexualized images every hour. Regulators in the UK subsequently launched a probe and the government came under pressure to penalize X, formerly Twitter. Comms watchdog Ofcom, which polices the Online Safety Act (OSA), is now formally investigating the social media platform, which was bought by xAI in March 2025. Grok's nudity capabilities are still switched on for paying users. Today's comments from Dame Chi Onwurah, chair of the Science, Innovation and Technology Committee, are in response to a letter from technology minister Liz Kendall, who tried to assure her that the government is tackling the issue. Kendall said in her letter to Dame Onwurah, dated January 12 but made public this morning: "xAI's action to restrict this ability to paying users is a further insult to victims, effectively monetizing this horrific crime." She said the OSA was built to deal with this situation and "intimate image abuse has been designated a 'priority offence,'" adding: "Ofcom has the mandate it needs to hold services to account for horrific illegal content on their sites and they have the government's unequivocal backing to use the full force of the powers that Parliament has granted them, up to and including, if they deem necessary, the power to apply to the courts to block services from being accessed in the UK if they refuse to comply with UK law." The government is also banning nudification tools, and Kendall said: "We will bring forward this legislation as a priority, making amendments to the Crime and Policing Bill going through Parliament now." Dame Onwurah responded, saying "significant questions remain" about the approach being taken, and asking why it has taken "so long" to introduce the nudification ban "when reports of these disturbing Grok deepfakes appeared in August 2025." "It's also unclear whether this ban - which appears to be limited to apps that have the sole function of generating nude images - will cover multi-purpose tools like Grok." Kendall also said if there are "gaps" in the OSA, the government will address them. To this, Dame Onwurah said: "This comes months after rejecting the committee's recommendations to explicitly regulate generative AI and put greater responsibility on platforms like X and Grok. I urge the government to adopt our recommendations and embed core principles - such as responsibility and transparency - into the online safety regime. These are essential principles to build a strong regulatory framework that protects users online." The Register has asked X to respond. Instead of a poop emoji, which is what X used to send journalists contacting it for comment, these days the automated messages states "Legacy Media Lies." We're hoping it sends us a specific statement on the matters above. ®

Musk's Grok AI Blocked in Indonesia, Malaysia Over Sexual Images

Indonesia and Malaysia restricted access to Elon Musk's Grok AI over the weekend, becoming the first countries to ban the artificial intelligence system over its generation of sexual content. Indonesia's Communications and Digital Affairs Ministry is imposing a temporary ban on Grok "to protect women, children, and the entire community from the risk of fake pornographic content generated using artificial intelligence technology," according to a statement issued on Saturday. The ministry has asked platform X to immediately provide clarification regarding the matter, it said.

Malaysia restricts access to Grok AI as backlash over sexualised images widens

KUALA LUMPUR, Jan 12 (Reuters) - Malaysia on Sunday temporarily blocked access to Grok, joining a growing list of countries taking action after the generative artificial intelligence chatbot sparked a global backlash by allowing users to create and publish sexualised images. xAI, the Elon Musk-led firm behind Grok, on Thursday said it would restrict image generation and editing to paying subscribers as it addressed lapses that allowed users on X to produce sexualised content of others, often without consent. On Saturday, Indonesia became the first country to temporarily deny access to the bot. In a statement on Sunday, the Malaysian Communications and Multimedia Commission (MCMC) said it would restrict access to Grok following repeated misuse of the tool "to generate obscene, sexually explicit, indecent, grossly offensive, and non-consensual manipulated images, including content involving women and minors." MCMC said it issued notices to X and xAI this month to demand the implementation of effective technical and moderation safeguards, but the received responses relied primarily on user-initiated reporting mechanisms and failed to address the risks posed by the design and operation of the AI tools. "MCMC considers this insufficient to prevent harm or ensure legal compliance," it said. xAI replied to a Reuters email seeking comment with what seemed to be an automated response: "Legacy Media Lies." X did not immediately respond to a request for comment. MCMC said access to Grok would be restricted until effective safeguards were implemented, adding that it was open to engaging with the firms. Muslim-majority Malaysia has strict laws governing online content, including a ban on obscene and pornographic materials. It has put internet companies under greater scrutiny in recent years in response to what it calls a rise in harmful content. Malaysia is considering barring users younger than 16 from accessing social media. Reporting by Rozanna Latiff; Editing by Thomas Derpinghaus Our Standards: The Thomson Reuters Trust Principles., opens new tab

He could just turn it off

Generative AI, we are repeatedly told, is a transformative and complicated technology. So complicated that its own creators are unable to explain why it acts the way it does, and so transformative that we'd be fools to stand in the way of progress. Even when progress resembles a machine for undressing strangers without their consent on an unprecedented scale, as has been the case of late with Elon Musk's Grok chatbot. UK Prime Minister Kier Starmer seems to have so fully bought into the grand lie of the AI bubble that he was willing to announce: Not that it currently is in compliance. Nor a timeline in which it is expected to do so. Just that he seems satisfied that someday, eventually, Musk's pet robot will stop generating child sexual abuse material. This statement comes just under two days after Starmer was quoted as saying "If X cannot control Grok, we will." What could Elon possibly have said to earn this pathetic capitulation. AI is difficult? Solutions take time? These are entirely cogent technical arguments until you remember: He could just turn it off. Elon Musk has the power to disable Grok, if not in whole (we should be so lucky) than its image generation capabilities. We know this intuitively, but also because he rate-limited Grok's image generation after this latest scandal: after a few requests, free users are now prompted to pay $8 per month to continue enlisting a wasteful technology to remove articles of clothing from women. Sweep it under the rug, make a couple bucks along the way. Not only is it entirely possible for image generation to be turned off, it's the only responsible option. Software engineers regularly roll back updates or turn off features that work less than optimally; this one's still up and running despite likely running afoul of the law. That we have now gone the better part of a month aware this problem exists; that the "feature" still remains should tell Starmer and others all they need to know. Buddy, you're carrying water for a bozo who does not seem to care that one such victim was reportedly Ashley St Clair, the mother of one of his (many) children. Some countries -- namely Malaysia and Indonesia -- chose to turn Grok off for their citizens by blocking the service. Indonesia's Communication and Digital Affairs Minister was quoted as saying "The government sees nonconsensual sexual deepfakes as a serious violation of human rights." Imagine if everyone in the business of statecraft felt that way. The UK (not to mention the US, but please, expect nothing from us, we're busy doing authoritarianism) has a lot more sway over X, and by extension Elon, than either of those countries. Musk does, and is looking to do even more, business in the UK. Even if Musk were not perhaps the world's most well known liar, Grok can still make images and that should speak for itself. Grok should be well out of second chances by now, and it's up to government leaders to say no more until they can independently verify it's no longer capable of harm.

Malaysia will take legal action against Musk's X and xAI over misuse of Grok chatbot

KUALA LUMPUR, Malaysia (AP) -- Malaysian authorities said Tuesday they will take legal action against Elon Musk's social media platform X and its artificial intelligence unit xAI, accusing the companies of failing to ensure the safety of users of its Grok chatbot. The move came just days after Malaysia and Indonesia became the first countries to block access to Grok, as concerns grow that it is being misused to generate sexually explicit and nonconsensual images. The Malaysian Communications and Multimedia Commission said in a statement that it has identified the misuse of Grok to generate and distribute harmful content including sexually explicit, indecent, extremely offensive as well as non-consensual manipulated images. It said it served notices to X and xAI this month to remove the harmful content but no action has been taken. "Content allegedly involving women and children is a matter of great concern. Such conduct is against Malaysian law and undermines the security commitments" stated by the companies, it said. The commission has appointed a lawyer and said legal proceedings would begin soon. There is growing scrutiny of generative AI tools that can produce realistic images, sound and text, and concern that existing safeguards are failing to prevent their abuse. Launched in 2023, Grok is free to use on X. An image generator feature, Grok Imagine, was added last year and included a so-called spicy mode that can generate adult content. Grok has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Grok has come under pressure in the European Union, India and the United Kingdom, which said Monday it was moving to criminalize "nudification apps." Britain's media regulator also launched an investigation into whether Grok broke the law by allowing users to share sexualized images of children. Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn't fully address the problem. Musk and his companies have not publicly commented on the Southeast Asian restrictions. xA1 has been giving an automated reply to media queries which stated, "Legacy Media Lies."

Malaysia and Indonesia block Elon Musk's Grok due to obscene, non-consensual content

Responses from xAI and CEO Elon Musk have so far failed to satisfy the concerns of regulators in Indonesia and Malaysia, as well as other jurisdictions like India. Malaysia and Indonesia blocked access to Elon Musk's artificial intelligence chatbot Grok over the weekend due to concerns that the tool was being used to generate non-consensual, sexually explicit and obscene content. Malaysian regulators ordered temporary restrictions be placed on the chatbot from xAI on Sunday following "repeated failures by X Corp" to address content risks associated with the AI tool. The move came just one day after Indonesia stepped in to deny access temporarily to Grok due to similar concerns and asked X officials to clarify on the matter, according to CNBC's translation of the statement. The Southeast Asian countries' actions come after it was discovered that some users of the AI tool generated non-consensual explicit images and deepfakes, including depictions of scantily clad minors. Musk's company had recently updated its Grok Imagine features, enabling easier image generation from text-based prompts on the platform, which is integrated with Musk's social media platform X, giving it a wide reach. Amid escalating concerns over Grok's content moderation policy, xAI announced it would limit image generation and editing features to paying subscribers, in an effort to patch safeguard gaps that permitted sexualized outputs. Musk, responding on X, has also asserted that users creating illegal content via Grok would face consequences equivalent to uploading such material directly to the social media platform. CNBC attempted to reach out to xAI for comment regarding the developments over the weekend. A press email for the company returned an automatic message that read "Legacy Media Lies."

Malaysia and Indonesia Block Access to Grok Because of Sexually Explicit Content

Grok, the chatbot created by Elon Musk's artificial intelligence company, has produced a flood of explicit images featuring real people in recent weeks. Indonesia and Malaysia said they were blocking access to Grok, the artificial intelligence chatbot created by Elon Musk, this weekend amid mounting outrage over the bot producing sexualized images of real people. Indonesia announced that it was temporarily blocking the chatbot from xAI, Mr. Musk's A.I. company, on Saturday, and the Malaysian government made a similar announcement on Sunday. While officials elsewhere have expressed concern and called for bans over the images, Indonesia and Malaysia are the first countries to formally ban the application. "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity and the security of citizens in the digital space," Meutya Hafid, Indonesia's minister of communications and digital affairs, said in a statement released on Saturday. X Corp, an xAI subsidiary, did not immediately respond to a request for comment on the bans. In recent weeks, sexually explicit images generated by Grok have flooded X, the social media platform owned by Mr. Musk where users can request the chatbot to produce images of real people, including putting them in sexually explicit situations, sometimes removing their clothes or positioning them in suggestive poses. Indonesia's government has long taken a strict stance against pornographic content online, restricting access to websites such as Pornhub and OnlyFans. In 2018, the authorities briefly blocked TikTok, citing the prevalence of content that officials said posed risks to children, including sexually explicit material. In Malaysia, regulators have said they plan to bar children under 16 from social media, partly because of several online bullying episodes that have resulted in the high-profile deaths of minors. Grok's production of sexually explicit images has drawn outrage from officials across the world, including Britain's prime minister, Keir Starmer, and U.S. Senators Ron Wyden, Democrat of Oregon; Ed Markey, Democrat of Massachusetts; and Ben Ray Lújan, Democrat of New Mexico, who sent a letter last week asking Apple and Google to remove the application from their online stores. While many A.I. chatbots have safeguards that restrict them from producing images of real people, Mr. Musk -- known for pushing the limit on allowable speech -- has so far refused to rein in his own platforms, where harassment and hate-filled language has thrived. Grok's creation of nonconsensual, sexually explicit imagery could potentially violate laws in multiple countries against possessing and sharing such content, particularly as some of the images have included children and minors. In a post on X this month, Mr. Musk said any users who directed Grok to produce sexually explicit images of children would suffer "consequences." Last week, the Grok feature on X restricted its image-generation abilities to subscribers, who pay a fee for access to extra features on the app. But regulators say the move does little to solve the central problem, and merely makes generating inappropriate images a premium service. On Friday, Mr. Starmer's spokesman said limiting image creation to X subscribers was "not a solution" and was "insulting" to victims of misogyny and sexual violence. Hasya Nindita contributed reporting from Yogyakarta, Indonesia, and Zunaira Saieed from Kuala Lumpur, Malaysia.

Elon Musk's Grok AI Blocked in Indonesia Over Sexualized Content

Indonesia blocked access to Grok after an investigation into the artificial intelligence system's generation of sexual content. The Communications and Digital Affairs Ministry is imposing a temporary ban on Grok "to protect women, children, and the entire community from the risk of fake pornographic content generated using artificial intelligence technology," according to a statement issued on Saturday. The ministry has asked platform X to immediately provide clarification regarding the matter, it said. "The government views non-consensual deepfake sexual practices as a serious violation of human rights, dignity, and national security in the digital space," Minister of Communications and Digital Meutya Hafid said in the statement. Indonesia's move came as Elon Musk's firm xAI, which owns Grok, decided on Friday to restrict the image-generation feature for most users on the X social media platform after the AI tool drew widespread condemnation for generating undressed images of women and children. Users now need a paid subscription to generate and edit images. Those features were initially introduced on X for free with daily limits. The standalone Grok app, which operates separately from the social network, still allows users to generate pictures without subscribing. "Sorry for the inconvenience," Grok posted on the X platform about the Indonesian block. "We are working to resolve this issue."

Indonesia temporarily blocks access to Grok over sexualised images

JAKARTA, Jan 10 (Reuters) - Indonesia temporarily blocked Elon Musk's Grok chatbot on Saturday due to the risk of AI-generated pornographic content, becoming the first country to deny access to the AI tool. The move comes after governments and regulators from Europe to Asia have condemned and some have opened inquiries into sexualised content on the app. xAI, the startup behind Grok, said on Thursday it was restricting image generation and editing to paying subscribers as it tried to fix safeguard lapses that had allowed sexualised outputs including depictions of scantily clad children. "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space," Communications and Digital Minister Meutya Hafid said in a statement. The ministry has also summoned X officials to discuss the matter. Musk said on X that anyone using Grok to make illegal content would suffer the same consequences as if they had uploaded illegal content. xAI replied to Reuters' email seeking comment with what seemed to be an automated response: "Legacy Media Lies". X did not immediately respond to a request for comment. Indonesia, with the world's biggest Muslim population, has strict rules that ban the sharing online of content deemed obscene. Reporting by Fransiska Nangoy; Editing by Stephen Coates Our Standards: The Thomson Reuters Trust Principles., opens new tab

Malaysia, Indonesia become first to block Musk's Grok over AI deepfakes

KUALA LUMPUR, Malaysia (AP) -- Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk's xAI, after authorities said it was being misused to generate sexually explicit and non-consensual images. The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse. The Grok chatbot, which is accessed through Musk's social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Regulators in the two Southeast Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesia's Communication and Digital Affairs Minister Meutya Hafid said in a statement Saturday. The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI. Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director general of digital space supervision, said in a separate statement. He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm. In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday after what it said was "repeated misuse" of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors. The regulator said notices issued this month to X Corp. and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they've directly created or replies to posts from other users. Last summer the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The Southeast Asian restrictions come amid mounting scrutiny of Grok elsewhere, including in the European Union, Britain, India and France. Grok last week limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it did not fully address the problem.

Two countries block Grok app over AI-generated CSAM

Two countries have blocked the Grok app after it was widely used to generate non-consensual near-nude deepfakes of women and children. A third country is currently carrying out an investigation. Three US senators have asked Apple to temporarily remove both X and Grok from the US App Store due to "sickening content generation," and we are still awaiting the company's response ... The Grok AI tool is available both as a standalone app and through the X app. It is also available through the Grok tab on the X website. There has been abundant evidence of Grok generating non-consensual, near-nude deepfakes of real individuals, taking a clothed photo and then digitally removing clothing to replace it with a bikini or other revealing clothing. Even more worryingly, some of these deepfakes were of children. While nude imagery is theoretically blocked by Grok, some users have been using prompt language that works around this. On Friday, three U.S. senators asked Apple to temporarily remove both apps from the App Store, noting that the non-consensual imagery included child sexual abuse materials (CSAM). Senators Ron Wyden, Ed Markey, and Ben Ray Luján penned an open letter to the CEOs of Apple and Google, asking both companies to pull X and Grok apps "pending a full investigation" of "mass generation of nonconsensual sexualized images of women and children." The letter notes that ex-CEO Elon Musk has failed to act, and contrasts the lack of action by Apple and Google with their rapid removal of the ICEBlock app at the request of the White House. Musk's only response has been to limit X image generation to paid subscribers, which seems the most cynical possible action, but the same feature is accessible to anyone through the Grok tab on both the X website and app. Associated Press reports that Malaysia and Indonesia have blocked the app within their countries. Regulators in the two Southeast Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. Britain's media regulator Ofcom has also opened a formal investigation. There have been deeply concerning reports of the Grok AI chatbot account on X being used to create and share undressed images of people - which may amount to intimate image abuse or pornography - and sexualised images of children that may amount to child sexual abuse material [...] Ofcom has decided to open a formal investigation to establish whether X has failed to comply with its legal obligations under the Online Safety Act. As of the time of writing, there has been no public response from either Apple or Google, and the apps remain available on their respective US app stores. We've reached out to Apple for comment and will update with any response.

Malaysia, Indonesia become first to block Musk's Grok over AI deepfakes

KUALA LUMPUR, Malaysia -- Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk's xAI, after authorities said it was being misused to generate sexually explicit and non-consensual images. The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse. The Grok chatbot, which is accessed through Musk's social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Regulators in the two Southeast Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesia's Communication and Digital Affairs Minister Meutya Hafid said in a statement Saturday. The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI. Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director general of digital space supervision, said in a separate statement. He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm. In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday after what it said was "repeated misuse" of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors. The regulator said notices issued this month to X Corp. and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they've directly created or replies to posts from other users. Last summer the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The Southeast Asian restrictions come amid mounting scrutiny of Grok elsewhere, including in the European Union, Britain, India and France. Grok last week limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it did not fully address the problem.

X still allowing users to post sexualised images generated by Grok AI tool

Despite restrictions announced this week, Guardian reporters find standalone app continues to allow posting of nonconsensual content X has continued to allow users to post highly sexualised videos of women in bikinis generated by its AI tool Grok, despite the company's claim to have cracked down on misuse. The Guardian was able to create short videos of people stripping to bikinis from photographs of fully clothed, real women. It was also possible to post this adult content on to X's public platform without any sign of it being moderated, meaning the clip could be viewed within seconds by anyone with an account. It appeared to offer a straightforward workaround to restrictions announced by Elon Musk's social network this week. These had been welcomed by the prime minister, Keir Starmer, who had described the photographs generated by Grok as "disgusting" and "shameful". After weeks of rising public concern, X said late on Wednesday it had "implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis". It said the restriction would apply to all users, including paid subscribers, and it had "zero tolerance for any forms of child sexual exploitation, nonconsensual nudity, and unwanted sexual content". But it did not specify whether people would still be able to create such images on the standalone Grok app, and then share this material publicly on X. The Guardian found that this standalone version of Grok, known as Grok Imagine - which is easily accessible through a web browser - was still responding to prompts to digitally remove the clothes from images of women. Reporters uploaded still images of fully clothed real-life women, and prompted the AI tool to dress them in bikinis. The platform responded by going further than the request, to create short videos of the women removing their clothes in the manner of a sexually provocative striptease. X has been contacted for comment. Rebecca Hitchen, the head of policy and campaigns at the End Violence Against Women Coalition, said such a simple workaround should not be possible. "The continued ease of access to sophisticated nudification tools clearly demonstrates that X isn't taking the issue of online violence against women and girls seriously enough," she said. Hitchen called on the UK government and Ofcom, the media regulator, to pressure X and other platforms "to stop the proliferation of image-based sexual abuse". Reuters has also reported that its journalists, including a reporter in Britain, used Grok to create on-demand sexualised photos after the announcements by X. "It's hard to believe that xAI and Elon Musk can't work out how to prevent these images from being spewed out by Grok," said Penny East, the chief executive of the Fawcett Society. "First, Musk decided the solution was to preserve nudification as a privilege only for those users who pay for X. Then he pledged to stop it entirely. And yet it has not stopped. "The truth is Musk and the tech sector simply do not prioritise safety or dignity in the products they create. It's a pretty low bar for women to expect that they can converse online without men undressing them. And yet seemingly even that is impossible." While Downing Street had said it felt "vindicated" by the steps X had taken, there was also caution among ministers about the extent of the changes and how they would be delivered. On Thursday, Liz Kendall, the technology secretary, who had described the sexual manipulation of images of women and children as "despicable and abhorrent", welcomed the move and thanked "those who have spoken out against this abuse, above all the victims". However, she added: "I will expect the facts to be fully and robustly established by Ofcom's ongoing investigation." Starmer also demanded X to act without delay. "Free speech is not the freedom to violate consent," he said. "Young women's images are not public property, and their safety is not up for debate. I welcome that X is now acting to ensure full compliance with UK law - it must happen immediately." Ofcom said its formal investigation into X, launched on Monday, remained ongoing and it was "working round the clock to progress this and get answers into what went wrong and what's being done to fix it". Canada's privacy watchdog said it was investigating xAI, while authorities in the Philippines said they were moving to block Grok, with Malaysian authorities planning to take legal action. But the controversy may have been helpful for boosting public awareness of Grok. On Thursday, Musk shared a post claiming "popularity and real world usage are skyrocketing globally" - alongside a graph of "Grok" as a search term hitting a new high on Google Trends. Musk simply added: "Try Grok.com." A government spokesperson said: "The Online Safety Act already requires platforms like X to prevent illegal content, including nonconsensual intimate images and child abuse material, from appearing on their services. "The secretary of state has said she expects X's compliance with UK laws to be fully and robustly established by Ofcom's investigation, which is already under way, and the government will not rest until social media companies meet their legal duties. "We are also taking further action through the proposed 'nudification' offence, which will target tools designed specifically to generate nonconsensual intimate images."

UK Regulators Open Investigation into X's Grok Amid Nudify Scandal

British, Southeast Asian, and other regulators are moving against X and its owner Elon Musk over Grok, the platform's AI chatbot, after it generated and publicly posted large volumes of sexualized images of real people, including women and children, without consent. In the U.K., media regulator Ofcom has opened a formal investigation into whether X violated laws designed to prevent the spread of illegal content, including nonconsensual intimate images and child sexual abuse material. Ofcom says it is examining whether the platform has failed in its legal duties to protect users in Britain. "Platforms must protect people in the U.K. from content that's illegal in the U.K., and we won't hesitate to investigate where we suspect companies are failing in their duties, especially where there's a risk of harm to children," the watchdog says in a statement. Under British law, X could face fines of up to £18 million or 10 percent of its global revenue. If the company is found in breach and refuses to comply with Ofcom's requests, the regulator could seek court orders restricting payment providers and advertisers from working with X. In extreme cases where a court finds "significant harm" to individuals, internet service providers could be required to block access to the platform in the U.K. The investigation follows widespread criticism after Grok automatically generated sexualized images in response to user prompts. Users were able to tag Grok under images and request edits such as "put her in a bikini," and in many cases, the bot complied, producing manipulated images of real people in sexualized contexts. Untold numbers of women and some children have had their likenesses sexualized without permission, including one of the mothers of Musk's children. In response to the furore, the platform announced that Grok's image-generation tool would be limited to paying subscribers. A spokesperson for Prime Minister Keir Starmer describes the change as "insulting" to victims of misogyny and sexual violence, adding that it "simply turns an AI feature that allows the creation of unlawful images into a premium service." Meanwhile, Indonesia and Malaysia became the first countries to block Grok entirely. Both governments imposed temporary bans after concluding that the chatbot lacked sufficient safeguards to prevent nonconsensual pornographic content based on real individual "The government sees nonconsensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," says Indonesian Communication and Digital Affairs Minister Meutya Hafid. Other governments and regulators in Europe and beyond have also signaled reviews or investigations. Policy experts say the scale of government reaction is unusual for a social media platform but reflects the nature of the alleged violations. "Making child sexual abuse [material] is flagrantly illegal, pretty much everywhere on Earth," Riana Pfefferkorn of Stanford University tells NPR. In the United States, political reaction has been more limited. Senator Ted Cruz says the images "should be taken down and guardrails should be put in place," and adds he is "encouraged that X has announced that they're taking these violations seriously and working to remove any unlawful images and offending users from their platform."

These countries may block Grok over deepfake unease

Grok faces global suspension over concerns about deepfakes and child exploitation. Credit: Yui Mok- PA Images / Contributor / PA Images via Getty Images Grok, xAI's all-purpose helper, fact-checker, and content generator, is under close watch from international regulators who worry the chatbot is dangerous. In early January, X users spotted that Grok was complying with requests to create sexualized depictions of minors in revealing clothing, a form of child sexual abuse material (CSAM). xAI staff said they were looking to strengthen the chatbot's guardrails. But it's not a simple fix. Recent investigations by Reuters, The Atlantic, and Wired found widespread flaws in the model's safeguards, which have allowed for a proliferation of nonconsensual, sexual, and sometimes graphically violent content to be generated by Grok at the request of users. On X, investigators have uncovered a mass "undressing" problem, marked by numerous accounts of users who have had their publicly posted images edited by the integrated chatbot to don revealing attire. RAINN, the national anti-sexual assault nonprofit, has called this a form of AI or tech-enabled sexual abuse. Other chatbots and AI-powered image generators have also come under scrutiny for failing to implement safeguards regarding sexual content, including Meta's AI companions. Replying to user concerns on X, CEO Elon Musk stated that the company takes the generation of illegal content seriously, but argued that the legal ramifications should fall on the users. Musk later responded to comments from UK's leaders about a potential sitewide block, calling it a form of censorship and alleging that foreign governments are eager to silence free speech online. In response to slow movement from X's teams, countries have begun tamping down access to xAI's bot and standalone app, as several conduct investigations into Grok's safeguards, xAI's response, and the possibility that the company is violating various online safety laws. The company has already faced several investigations before recent claims that Grok enabled CSAM. In addition to ongoing investigation of X's content moderation and algorithms, members have pushed for the European Union to probe Grok for several alleged violations under its Digital Services Act, including potentially dangerous and "erratic" behavior. In recent weeks, the body ordered X to retain all data and internal documents regarding Grok as it expands its investigation. Despite a commission spokesperson calling Grok's conduct "illegal," tech watchdogs are skeptical the body would invoke its measures to block the site completely. The UK's communication regulator, Ofcom, also announced an official investigation into Grok, with a potential revenue fine of up to 10 percent (about $24 million) if it's found to have violated the Online Safety Act. Organizations in the U.S. have called on federal agencies to investigate xAI, as well, citing the recently passed Take It Down Act, which criminalizes publishing or threatening to publish nonconsensual intimate imagery (NCII) and grants the Federal Trade Commission (FTC) authority to sue social media platforms that don't comply with take-down requests. Other countries, meanwhile, are moving forward with outright bans until the company complies with investigations or overhauls Grok's safety structure.

Grok blocked in Malaysia and Indonesia as sexual deepfake scandal builds | Fortune

There is growing scrutiny of generative AI tools that can produce realistic images, sound and text, and concern that existing safeguards are failing to prevent their abuse. The Grok chatbot, accessed through Musk's social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. "The government sees nonconsensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesian Communication and Digital Affairs Minister Meutya Hafid said in a statement. Scrutiny of Grok is growing, including in the European Union, India, France and the United Kingdom, which said Monday it was moving to criminalize "nudification apps." Britain's media regulator also launched an investigation into whether Grok broke the law by allowing users to share sexualized images of children. Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn't fully address the problem. An emailed request for comment by The Associated Press to xAI resulted in an automated reply from the media support address which stated, "Legacy Media Lies." This was the same message received from a different address when asked for comment regarding the global backlash. Regulators in the two Southeast Asian nations said existing controls weren't preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. Initial findings showed Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director-general of digital space supervision, said in a statement. He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent. The Malaysian Communications and Multimedia Commission noted "repeated misuse" of the tool to generate obscene, sexually explicit and nonconsensual manipulated images, including content involving women and minors. The regulator said notices were issued this month to X Corp. and xAI demanding stronger safeguards. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. Launched in 2023, Grok is free to use on X. An image generator feature, Grok Imagine, was added last year and included a so-called spicy mode that can generate adult content. The U.K.'s media regulator said it launched an investigation into whether Grok violated its duty to protect people from illegal content. The regulator, Ofcom, said Grok-generated images of children being sexualized or people being undressed may amount to pornography or child sexual abuse material. Technology Secretary Liz Kendall called the AI-generated images "weapons of abuse" and said the center-left Labour government would target the source of the problem by making it a crime for companies to supply tools to create nude images without consent. Kendall said X could face a possible Ofcom fine of up to 10% of its qualifying global revenue depending on the investigation's outcome and a possible court order blocking access to the site. "They can choose to act sooner to ensure this abhorrent and illegal material cannot be shared on their platform," Kendall said in Parliament. Musk over the weekend called the British government "fascist" and said it is trying stifle free speech. ___ Tarigan reported from Jakarta, Indonesia. Associated Press writers Sylvia Hui and Brian Melley in London contributed.

How AI 'deepfakes' became Elon Musk's latest scandal

Paris (France) (AFP) - Elon Musk's company xAI has faced global backlash in recent days over sexualised "deepfake" images of women and children created by its Grok chatbot. Here are the essential facts about the scandal, how governments have responded and the company's attempts to cool the controversy. 'Put her in a bikini' Grok -- Musk's version of the chatbots also offered by OpenAI and other generative AI companies -- has its own account on the X social network allowing users to interact with it. Until last week, users could tag the bot in posts to request image generation and editing, receiving the image in a reply from Grok. Many took advantage of the service by sending Grok photos of women or tagging the bot in replies to women's photo posts. They would ask it to "put her in a bikini" or "take her clothes off" -- receiving photorealistic altered images in response. Such AI-powered nonconsensual "nudifying" services had previously been available on niche websites, but Grok became the first to take it mainstream with social media integration and offer it for free. Outrage grew as some users were discovered generating sexualised images of children and minors. Still others used the tool to generate bikini images of women killed in the deadly New Year fire at Swiss ski resort Crans-Montana, as well as the woman shot and killed by an immigration officer in Minneapolis. Last week, an analysis of more than 20,000 Grok-generated images by Paris non-profit AI Forensics found that more than half depicted "individuals in minimal attire" -- most of them women, and two percent appearing to be under-18s. - How have countries reacted? Indonesia on Saturday became the first country to block access to Grok entirely, with neighbouring Malaysia following on Sunday. India said Sunday that X had removed thousands of posts and hundreds of user accounts in response to its complaints. Speaking on condition of anonymity, a government source told AFP 3,500 posts and 600 accounts had been removed. Britain's Ofcom media regulator -- which can fine companies up to 10 percent of global revenue -- said Monday it was opening a probe into whether X failed to comply with UK law over the sexual images. "If X cannot control Grok, we will -- and we'll do it fast," British Prime Minister Keir Starmer told MPs from his Labour Party. France's commissioner for children Sarah El Hairy said Tuesday she had referred Grok's generated images to French prosecutors, the Arcom media regulator and the European Union. Digital affairs minister Anne Le Henanff had earlier called the restriction of image creation to paying users "insufficient and hypocritical". And the European Commission, which acts as the EU's digital watchdog, has ordered X to retain all internal documents and data related to Grok until the end of 2026 in response to the uproar. The bloc has already been investigating X over potential breaches of its digital content rules since 2023. "We will not be outsourcing child protection and consent to Silicon Valley," Commission chief Ursula von der Leyen said Monday. "If they don't act, we will." - How did the company respond? "We take action against illegal content... including Child Sexual Abuse Material (CSAM) by removing it, permanently suspending accounts, and working with local governments and law enforcement," X's safety team posted on January 4. Musk himself said last week that anyone using Grok to "make illegal content will suffer the same consequences as if they upload illegal content". But he made light of the controversy in a separate post, adding laughing emojis as he reshared to his 232 million followers on X a post featuring a toaster wrapped in a bikini. By January 9, Grok began responding to all requests for image generation or editing by saying the service was restricted to paying subscribers. Musk has also fired back at politicians demanding action. Critics of X and Grok "just want to suppress free speech" Musk posted on January 10.

Malaysia to take legal action against Musk's X and xAI over Grok

Authorities said Grok was being misused to generate sexually explicit and non-consensual AI images. Malaysian authorities said on Tuesday they will take legal action against Elon Musk's social media platform X and its artificial intelligence unit xAI, accusing the companies of failing to ensure the safety of users of its Grok chatbot. The move came just days after Malaysia and Indonesia became the first countries to block access to Grok, as concerns grow that it is being misused to generate sexually explicit and nonconsensual images. The Malaysian Communications and Multimedia Commission said in a statement that it has identified the misuse of Grok to generate and distribute harmful content including sexually explicit, indecent, extremely offensive as well as non-consensual manipulated images. It said it served notices to X and xAI this month to remove the harmful content but no action has been taken. "Content allegedly involving women and children is a matter of great concern. Such conduct is against Malaysian law and undermines the security commitments" stated by the companies, it said. The commission has appointed a lawyer and said legal proceedings would begin soon. There is growing scrutiny of generative AI tools that can produce realistic images, sound and text, and concern that existing safeguards are failing to prevent their abuse. Launched in 2023, Grok is free to use on X. An image generator feature, Grok Imagine, was added last year and included a so-called spicy mode that can generate adult content. Grok has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Grok has come under pressure in the European Union, India and the United Kingdom, which said Monday it was moving to criminalize "nudification apps." Britain's media regulator also launched an investigation into whether Grok broke the law by allowing users to share sexualized images of children. Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn't fully address the problem. Musk and his companies have not publicly commented on the Southeast Asian restrictions. xA1 has been giving an automated reply to media queries which stated, "Legacy Media Lies."

Indonesia temporarily blocks access to Elon Musk's Grok over sexualised images

Indonesia temporarily blocked Elon Musk's Grok chatbot on Saturday due to the risk of AI-generated pornographic content, becoming the first country to deny access to the AI tool. The move comes after governments and regulators from Europe to Asia have condemned and some have opened inquiries into sexualised content on the app. xAI, the startup behind Grok, said on Thursday it was restricting image generation and editing to paying subscribers as it tried to fix safeguard lapses that had allowed sexualised outputs including depictions of scantily clad children. "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space," Communications and Digital Minister Meutya Hafid said in a statement. The ministry has also summoned X officials to discuss the matter. Musk said on X that anyone using Grok to make illegal content would suffer the same consequences as if they had uploaded illegal content. xAI replied to Reuters' email seeking comment with what seemed to be an automated response: "Legacy Media Lies." X did not immediately respond to a request for comment. Indonesia, with the world's biggest Muslim population, has strict rules that ban the sharing online of content deemed obscene.

Indonesia suspends Grok AI over sexualized images

Indonesia suspended Elon Musk's AI chatbot Grok on Saturday over growing concerns about AI-generated sexualized photos of people without their consent. Elon Musk's platform is facing global backlash after reports emerged that its image creation feature allowed users to sexualize pictures of women and children using simple text prompts. CBS News has verified that Grok fulfilled user requests asking it to edit images of women to show them in bikinis or little clothing, including prominent public figures such as first lady Melania Trump. Indonesia is the first country to deny all access to the tool, which was restricted to paying subscribers elsewhere following backlash. "In order to protect women, children, and the public from the risks of fake pornographic content generated using the artificial intelligence technology, the government... has temporarily blocked access to the Grok application," Communication and Digital Affairs Minister Meutya Hafid said in a statement. "The government views non-consensual deepfake practices as a serious violation of human rights, dignity, and the security of citizens in the digital space." According to AFP journalists in Jakarta, the Grok X account was still active and replying to queries, including in Bahasa, Indonesia, on Saturday evening. Musk-owned startup xAI, which developed Grok, told CBS News in a statement on Saturday that "Legacy Media Lies," without elaborating further. The message appears to be an automatic reply to inquiries from media outlets. European officials and tech campaigners slammed the move to limit Grok's features to paying subscribers, saying it failed to address concerns about sexually explicit deepfake content. U.K. Prime Minister Keir Starmer said Thursday that he wants "all options on the table," including a potential ban for Grok in Britain. "This is disgraceful, it's disgusting and it's not to be tolerated. X has got to get a grip of this," Starmer said in an interview with a U.K. radio station. "It's unlawful. We're not going to tolerate it. I've asked for all options to be on the table." A source in Starmer's office reiterated to CBS News on Friday that "nothing is off the table" when it comes to regulating X in Britain. In the U.S., Republican Sen. Ted Cruz said in a post on X that "many of the recent AI-generated posts are unacceptable and a clear violation of my legislation -- now law -- the Take It Down Act, as well as X's terms and conditions." "These unlawful images pose a serious threat to victims' privacy and dignity. They should be taken down and guardrails should be put in place," Cruz said, adding that he was encouraged by steps taken by X to remove unlawful images. Musk said last week in response to a post about the explicit images that anyone using Grok to "make illegal content will suffer the same consequences as if they upload illegal content."

Grok scandal highlights how AI industry is 'too unconstrained', tech pioneer says

Musk's X to block Grok AI from creating sexualised images of real people The scandal over the flood of intimate images on Elon Musk's X created non-consensually by its Grok AI tool has underlined how the artificial intelligence industry is "too unconstrained", according to a pioneer of the technology. Yoshua Bengio, a computer scientist described as one of the modern "godfathers of AI", said tech companies were building systems without appropriate technical and societal guardrails. Bengio spoke to the Guardian as he appointed the historian Yuval Noah Harari and the former Rolls-Royce chief executive Sir John Rose to the board of his AI safety lab. X has announced it is stopping Grok from manipulating pictures of real people to show them in revealing clothes such as bikinis, including for premium subscribers, after a public and political backlash. Asked what the furore showed about the state of the AI industry, Bengio said the situation across the sector was "not completely a free for all" but needed to be addressed. "It is too unconstrained and, because frontier AI companies are building increasingly powerful systems without the appropriate technical and societal guardrails, this is starting to have more and more visible negative effects on people," Bengio said. Part of the solution was better governance, he said, including placing moral heavyweights on company boards. As well as Harari and Rose, Bengio has appointed Maria Eitel, the founder of the Nike Foundation - a philanthropic wing of the multinational sports group - as the chair of his safety lab, LawZero, which launched last year. The former Swedish prime minister Stefan Löfven will be the first member of the NGO's global advisory council. Harari, the author of Sapiens: A Brief History of Humankind, has been a prominent voice of caution on AI development and recently published a book, Nexus, outlining his concerns. "It's not only a technical discussion for companies building frontier AI systems," said Bengio. "It also comes down to what choices are made about AI that we consider to be morally right." Bengio, a professor of computer science at the University of Montreal, has secured $35m (£26m) of funding for LawZero. It is building a system called Scientist AI that will work alongside autonomous systems - knows as AI agents - and flag potentially harmful behaviour. "The whole construction of the board has been guided by the idea that we need a group of people who are extremely reliable in a moral sense, who can help us keep to LawZero's mission of delivering technical solutions for trustworthy, highly capable, safe-by-design AI systems as a global public good," said Bengio. Bengio earned the "godfather of AI" moniker after winning the 2018 Turing award, seen as the equivalent of a Nobel prize for computing. He shared it with Geoffrey Hinton, who later won a Nobel, and Yann LeCun, a former chief AI scientist at Mark Zuckerberg's Meta. Last month, Bengio warned against granting AI rights, saying it was showing signs of self-preservation - a key area of concern for AI safety campaigners - and humans should not be impeded from pulling the plug on such systems.

Grok blocked by Indonesian and Malaysian governments as UK threatens full ban

Governments are cracking down on Grok as deepfake problem worsens. Credit: Arda Kucukkaya / Contributor / Anadolu via Getty Images Grok access has been cut off for users in Indonesia and Malaysia following concerns that the xAI chatbot's safeguards were ineffective. Both countries issued temporary suspensions intended to remain in effect until xAI implements safeguards that comply with regulators' demands. "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space," wrote Indonesia's minister of communications and digital affairs Meutya Hafid in a statement released Saturday. Indonesia has sweeping internet censorship laws governing content deemed "obscene." Malaysia had recently launched an investigation into "misuse of artificial intelligence (AI) tools on the X platform," part of a wave of intervening actions taken by regulators. That investigation followed on the heels of an Indian IT ministry notice directing X to take immediate action on Grok's alleged misuse. The notice alluded to potential violations of the country's Information Technology Act. French authorities in the Paris prosecutor's office and other foreign governments, including the UK and an EU probe, have announced similar investigations into xAI's technology and its regulation under relevant online safety laws. Australian prime minister Anthony Albanese also issued a statement concerning Grok's deepfake problems, reiterating the country's decision to ban social media for users under the age of 16. In addition, the U.S. based National Center on Sexual Exploitation (NCOSE) has called on the Department of Justice and Federal Trade Commission to investigate X under existing child sexual abuse material (CSAM) laws and the recent Take It Down Act. UK technology secretary Liz Kendall recently said she would support blocking X outright, should the UK's Office of Communications (Ofcom) conclude the platform violates the Online Safety Act. Kendall said she expects a decision to made in the next few days. Elon Musk, who has previously said users should face consequences for "uploading illegal content" on X, responded to the threats of a ban by accusing the UK government of being overly eager to censor the company. In a post on X, Musk wrote: "They just want to suppress free speech." Musk also shared several X posts claiming that Grok is being targeted by government officials. A recent Wired investigation found that the desktop and app versions of Grok Imagine were able to produce sexually violent and graphic material, depictions of celebrities in sexual scenarios, and potential AI-generated CSAM, despite built-in safeguards. The chatbot has a now well-documented track record of producing sexualized deepfakes, including at the request of users who ask Grok to non-consensually "undress" individuals in publicly-uploaded photos.

Governments around the world are considering bans on Grok's app over AI sexual image scandal

Both Indonesia and Malaysia ordered that restrictions be put in place over the weekend. Malaysian officials blocked access to Grok on Sunday, citing "repeated misuse ... to generate obscene, sexually explicit, indecent, grossly offensive, and non-consensual manipulated images." Officials also cited "repeated failures by X Corp." to prevent such content. Indonesia had blocked the chatbot the previous day for similar reasons. In a statement accompanying Grok's suspension, Meutya Hafid, Indonesia's Minister of Communication and Digital, said. "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space." The responses could be just the beginning of Grok's problems, though. Several other countries, including the U.K., India, and France, are thinking of following suit.

Indonesia and Malaysia Block Access to xAI's Grok | AIM

While numerous countries around the world have voiced their concerns and sought a response from Musk's xAI on the issue, the two countries are the first to enforce a ban. In a response to mounting concerns over harmful AI-generated content, both Indonesia and Malaysia have moved to block access to Grok, the AI chatbot developed by Elon Musk's xAI, citing risks associated with AI-generated sexualised and non-consensual imagery. On January 10, Indonesia's Ministry of Communication and Digital Affairs temporarily blocked access to Grok after widespread reports that users were exploiting the tool to generate sexually explicit deepfake images -- some depicting scantily clad minors -- without consent. A report from Reuters quoted the country's communications minister, Meutya Hafid, who said the government views such AI misuse as a "serious violation of human rights, dignity and the security of citizens in the digital space." The block came despite xAI's decision days earlier to limit Grok's image-generation features to paying users, as part of efforts to tighten safeguards. The following day, Malaysia's Communications and Multimedia Commission (MCMC) imposed a temporary restriction on Grok's accessibility, after finding that repeated misuse of the chatbot had produced obscene, indecent and non-consensual manipulated content. MCMC officials said their formal notices to Grok and its parent platform X for stronger moderation and technical safeguards had not produced adequate results, prompting the restriction. While numerous countries around the world have voiced their concerns and sought a response from Musk's xAI on the issue, the two countries are the first to enforce a ban. Having said that, Indonesia has been one of the more proactive countries who has banned social media services that exist as a medium to share pornographic content. In 2014-2015, Indonesia blocked access to Reddit and Vimeo amid an online pornography crackdown, when authorities determined those platforms hosted nudity and other content inconsistent with local norms and laws. While responding to a post on X about the United Kingdom's government taking a similar approach to restricting Grok in the country, Musk said, "They want any excuse for censorship." However, Musk also previously stated that "Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content."

Malaysia and Indonesia move to ban Musk's Grok AI over sexually-explicit deepfakes | Fortune

Two Southeast Asian nations -- Malaysia and Indonesia -- have become the world's first countries to take action against Elon Musk's Grok chatbot and its ability to generate deepfake images. On Jan. 10, Indonesia announced it would temporarily restrict access to Grok. Meutya Hafid, the nation's Communication and Digital minister, wrote in a statement that the ban was imposed to protect "women, children and the larger community" from fake pornographic content created by AI. Then, the next day, the Malaysian Communications and Multimedia Commission (MCMC) said it would also "temporarily restrict" access to Grok following "repeated misuse ... to generate obscene, sexually explicit, indecent, grossly offensive, and non-consensual manipulated images." In a statement, MCMC said it had issued notices to both X and xAI -- Grok's corporate parents -- on Jan. 3 and Jan. 8 respectively, yet deemed their responses "insufficient to prevent harm or ensure legal compliance." Both Indonesia and Malaysia have strict rules against online pornography. Indonesia, home to the world's largest Muslim population, has been particularly aggressive, and has charged both Indonesian and foreign OnlyFans creators under its 2008 Pornography Act. On Jan. 9, X restricted image generation to paying users in a bid to constrain a flood of deepfaked images, where users would ask Grok to alter images of women, to show them dressed in revealing clothing. Victims have attacked X for allowing the nonconsensual creation of sexualized content, including of minors. Government officials, too, are considering taking action against X and Grok. xAI sent an email with only the text "Legacy Media Lies" after Fortune reached out for comment on Indonesia and Malaysia's ban. Persistent safety lapses Government moves against X and Grok were likely inevitable, given how easy it became to generate deepfaked content. "Grok's guardrails are easy to bypass," says Chew Han Ei, a senior research fellow at the Lee Kuan Yew School of Public Policy in the National University of Singapore (NUS). "When a system can be nudged so readily into producing or amplifying harmful synthetic content, that points to a design weakness." Indonesia and Malaysia's moves against Grok and X comes amid broader worries in the region about social media and deepfakes. Several governments throughout Asia have passed regulations against the production of deepfakes, usually in the context of preventing cybercrime, fraud, and election interference. In 2024, South Korea criminalized watching or producing deepfake pornography. Last year, both Australia and Malaysia banned access to social media for under-16s, citing concerns about online dangers like cyberbullying, sexual exploitation and financial scams. On Monday, Meta disclosed that it had closed 550,000 accounts across Facebook, Instagram and Threads to comply with Australia's new ban. "Governments are becoming less willing to tolerate a release-first, fix-later approach. If safeguards are not credible, access to the tool becomes a legitimate policy question," Chew says.

Malaysia will take legal action against Musk's X and xAI over misuse of Grok chatbot

KUALA LUMPUR, Malaysia (AP) -- Malaysian authorities said Tuesday they will take legal action against Elon Musk's social media platform X and its artificial intelligence unit xAI, accusing the companies of failing to ensure the safety of users of its Grok chatbot. The move came just days after Malaysia and Indonesia became the first countries to block access to Grok, as concerns grow that it is being misused to generate sexually explicit and nonconsensual images. The Malaysian Communications and Multimedia Commission said in a statement that it has identified the misuse of Grok to generate and distribute harmful content including sexually explicit, indecent, extremely offensive as well as non-consensual manipulated images. It said it served notices to X and xAI this month to remove the harmful content but no action has been taken. "Content allegedly involving women and children is a matter of great concern. Such conduct is against Malaysian law and undermines the security commitments" stated by the companies, it said. The commission has appointed a lawyer and said legal proceedings would begin soon. There is growing scrutiny of generative AI tools that can produce realistic images, sound and text, and concern that existing safeguards are failing to prevent their abuse. Launched in 2023, Grok is free to use on X. An image generator feature, Grok Imagine, was added last year and included a so-called spicy mode that can generate adult content. Grok has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Grok has come under pressure in the European Union, India and the United Kingdom, which said Monday it was moving to criminalize "nudification apps." Britain's media regulator also launched an investigation into whether Grok broke the law by allowing users to share sexualized images of children. Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn't fully address the problem. Musk and his companies have not publicly commented on the Southeast Asian restrictions. xA1 has been giving an automated reply to media queries which stated, "Legacy Media Lies."

Malaysia will take legal action against Musk's X and xAI over misuse of Grok chatbot

KUALA LUMPUR, Malaysia -- Malaysian authorities said Tuesday they will take legal action against Elon Musk's social media platform X and its artificial intelligence unit xAI, accusing the companies of failing to ensure the safety of users of its Grok chatbot. The move came just days after Malaysia and Indonesia became the first countries to block access to Grok, as concerns grow that it is being misused to generate sexually explicit and nonconsensual images. The Malaysian Communications and Multimedia Commission said in a statement that it has identified the misuse of Grok to generate and distribute harmful content including sexually explicit, indecent, extremely offensive as well as non-consensual manipulated images. It said it served notices to X and xAI this month to remove the harmful content but no action has been taken. "Content allegedly involving women and children is a matter of great concern. Such conduct is against Malaysian law and undermines the security commitments" stated by the companies, it said. The commission has appointed a lawyer and said legal proceedings would begin soon. There is growing scrutiny of generative AI tools that can produce realistic images, sound and text, and concern that existing safeguards are failing to prevent their abuse. Launched in 2023, Grok is free to use on X. An image generator feature, Grok Imagine, was added last year and included a so-called spicy mode that can generate adult content. Grok has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Grok has come under pressure in the European Union, India and the United Kingdom, which said Monday it was moving to criminalize "nudification apps." Britain's media regulator also launched an investigation into whether Grok broke the law by allowing users to share sexualized images of children. Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn't fully address the problem. Musk and his companies have not publicly commented on the Southeast Asian restrictions. xA1 has been giving an automated reply to media queries which stated, "Legacy Media Lies."

Malaysia suspends access to Musk's Grok AI: regulator

Kuala Lumpur (AFP) - Malaysia suspended access to Elon Musk's chatbot Grok over AI-generated pornographic content, the country's tech regulator said on Sunday. The decision follows global backlash after it emerged that Grok's image creation feature allowed users to sexualise pictures of women and children using simple text prompts. On Saturday Indonesia became the first country to deny all access to the tool, which has been restricted to paying subscribers elsewhere. The Malaysian Communications and Multimedia Commission said in a statement it had "directed a temporary restriction on access to the Grok artificial intelligence for users in Malaysia" with immediate effect. When an AFP reporter in the Malaysian capital Kuala Lumpur fed Grok prompts on Sunday, there was no response. "This action follows repeated misuse of Grok to generate obscene, sexually explicit, indecent, grossly offensive and non-consensual manipulated images," the regulator said. The statement cited "content involving women and minors, despite prior regulatory engagement and formal notices" issued to Musk's X Corp. and xAI startup which developed Grok. The AI tool is integrated into social media platform X. The Malaysian regulator said it deemed the platform's safeguards inadequate, adding that access would resume only after the required changes are verified. X Corp. had "failed to address the inherent risks posed by the design and operation of the AI tool", relying "primarily on user-initiated reporting mechanisms", the regulator said. European officials and tech campaigners on Friday slammed Grok after its controversial image creation feature was restricted to paying subscribers, saying the change failed to address concerns about sexualised deepfakes. Grok had appeared to deflect the criticism with a new monetisation policy, posting on X on Thursday that image generation and editing were now "limited to paying subscribers", alongside a link to a premium subscription.

Which countries are taking aim at Elon Musk's Grok AI chatbot?

Global crackdown: These nations are restricting or speaking out against Grok over non-consensual sexually explicit deepfakes Governments worldwide are moving swiftly to rein in Grok, Elon Musk's artificial intelligence (AI) chatbot, amid growing concerns it generates fake, sexually explicit images. Last summer, xAI, Grok's operating company, added an image-generator feature that included a "spicy mode" that could generate adult content. In recent weeks, Grok has responded to user prompts to "undress" images of women and dress them in bikinis, creating AI-generated deepfakes with no safeguards. From outright blocks in Southeast Asia to criminal probes and regulatory warnings in Europe and Australia, authorities say existing safeguards are failing. These are the countries that have banned or heeded warnings about Grok. Which countries have banned Grok? Indonesia Indonesia was the first country to temporarily block Grok to protect women, children, and the broader community from fake pornographic content generated using AI. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesia's communication and digital affairs minister Meutya Hafid said in a statement on Saturday. Restrictions on Grok are a "preventative" measure while the authorities assess whether the platform is safe, Indonesia's National Police said. Initial findings showed that Grok does not have efficient safeguards to stop the creation and distribution of pornographic content based on real photos of Indonesian residents, according to a statement from Alexander Sabar, Indonesia's director general of digital space supervision. Sabar said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social, and reputational harm. Malaysia The Malaysian Communications and Multimedia Commission (MCMC) ordered a temporary ban on Grok on Sunday after what it said was "repeated misuse" of the tool to generate obscene, sexually explicit, and non-consensual manipulated images. The regulator said it sent two notices this month to X and xAI demanding stronger safeguards. In its replies, X said that Grok relies mostly on users to submit complaints about abusive content. The MCMC concluded that X "failed to address the inherent risks" in the design and operation of its AI platform, which it said is insufficient under Malay law. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. How have other countries responded? European Union The European Commission announced it is looking into cases of sexually suggestive and explicit images of young girls generated by Grok. "I can confirm from this podium that the Commission is also very seriously looking into this matter," a Commission spokesperson told journalists in Brussels last week. Reuters reported that the Commission ordered X to retain all documents relating to Grok until the end of the year so the bloc can evaluate whether it complies with EU rules. A Commission spokesperson told Reuters that it doesn't mean that a formal investigation has been launched. Ursula Von der Leyen, president of the European Commission, said in an interview with the Italian newspaper Corriere della Sera that she was "outraged that a technology platform allows users to digitally strip women and children online". Without directly naming X or Grok, von der Leyen said the Commission "will not outsource child protection and consent to Silicon Valley. If they don't act, we will," she said. United Kingdom The United Kingdom's media watchdog launched an investigation into X, xAI's parent company, and Musk's social media platform, over the use of Grok to generate sexually explicit and non-consensual images. Ofcom said in a statement that there had been "deeply concerning reports" of the chatbot being used to create and share undressed images of people, as well as "sexualised images of children". The media watchdog could also seek a court order to force internet service providers to block access to Grok if X doesn't comply with Ofcom's requirements. X could face a fine of up to 10 percent of its worldwide revenue or £18 million (€20 million). _"_The content which has circulated on X is vile. It's not just an affront to decent society, it is illegal," Liz Kendall, the technology secretary, told parliament. France The Paris Prosecutor's Office expanded an investigation into X in early January to include Grok, local media reported. The initial probe, ongoing since last July, focused on suspected organised interference with X's computer systems and the illegal extraction of data. The decision to widen the investigation was made this week after five politicians accused the platform of generating and disseminating fake sexually explicit videos featuring minors, according to French newspaper Le Parisien. It reported that France's Regulatory Authority for Audiovisual and Digital Communication (Arcom) is investigating X's potential breaches of the Digital Services Act, the European regulation rules for digital services. Italy On January 8, Italy's Data Protection Authority (Garante) warned that anyone using Grok or other AI platforms to remove people's clothing and those distributing these images risk criminal charges. Using the tools without the permission of the person in the photo could be considered "[a] serious violation of the fundamental rights and freedoms of the individuals involved," the body said. The authority also reminded Grok and other AI providers that they must design, develop, and make products available that comply with privacy regulations. The Italian regulator said it is working with Ireland's Data Protection Commission, the lead privacy authority for X, because the company's headquarters are based in Ireland. In October, Italy's authority blocked ClothOff, another AI-generated platform that removes clothing. The platform lets anyone, including minors, create photos and videos that portray real people in the nude or in sexual positions. In September, Italy added a new articleunder its criminal code to punish those disseminating AI deepfakes with up to five years in prison. Germany Germany said that it will soon present a "concrete proposal" for a new law against digital violence. Anna-Lena Beckfeld, a spokesperson for Germany's justice ministry, said in a press conference in January that the eventual digital violence law will be a way to support victims of this "type of digital violence," by making it "easier for them to take direct action against violations of their rights online". When asked specifically about Grok, Beckfeld said it is "unacceptable that manipulation is being used on a large scale for systemic violations of personal rights". "We, therefore, want to see stronger measures taken against this through criminal law," she added. In 2025, Germany's three main parties agreed to reform cybercrime law and close "loopholes" in the criminal code for AI-related crimes, such as AI-generated sexual images, according to a coalition agreement. Australia In January, the office of Australia's eSafety Commissioner said it had received a small but increasing number of reports in the past couple of weeks about Grok's sexual AI content. The commissioner's office said it will use its powers, such as a removal notice, which could order a social media site to take down problematic content, if any of the content violates the country's Online Safety Act. The office has already requested more information from X about the misuse of Grok's sexual service and to evaluate whether it is complying with Australia's new social media law. It also reminded Grok that, as of March 9, online services, including AI companies, will have to block children's access to sexual, violent or harmful content.

Elon Musk's Grok made the world less safe - his humiliating backdown gives me hopium | Van Badham

The AI chatbot's torrent of nonconsensual deepfakes isn't its first scandal and won't be its last. Responsible governments should simply ban it Billionaire and career Bond-villain cosplayer Elon Musk has been forced by public backlash into a humiliating backdown over use of his AI chatbot, Grok. Watching the world's richest man eat a shit sandwich on a global stage represents a rare win for sovereign democracy. In its signature tone of "insufferable jerk who's just completed his first online webinar on how to patronise girls", Musk's chatbot appeared this week to make the world less safe, less fair and perhaps even as unpleasant as a fascist-styled, ketamine-addled rich-kid dipshit in cheese hat dancing while the world burns, if one could imagine such an awful thing. Grok's latest notoriety isn't because it's shared false information - though it has, previously, downplayed the Holocaust due to a claimed "programming error" and, more recently, spread conspiracy-style claims about the Bondi massacre. It's not because of Grok's antisemitic comments, or random claims of "white genocide" within unrelated conversations. It isn't about the fallibility of AI chatbots more broadly, either - even though consumer advocates, health professionals, media associations and perhaps everyone who's ever taught at a university ever have repeatedly warned chatbot "advice" is recklessly unreliable. If you're asking what behaviours could've possibly been left to condemn in the wake of Grok once accepting the name "MechaHitler", I envy your naivety - because the answer is: Grok released tools enabling the creation and sharing of nonconsensual sexual exploitation images. Grok's "spicy mode" capacity launched in August, and by December its host platform X was "deluged with images of women and children whose clothes (were) digitally removed". Last week, researchers in Paris reported finding 800 pornographic images created by Grok's tools, including depictions of sexual violence. A UK-based internet-monitoring group reported users of a dark web forum boasting about Grok creating "sexualised and topless imagery of girls aged between 11 and 13". Formerly confined to the internet's darker corners, "nudifying" deepfake tools have been used for the image-based abuse of children and adults from Bacchus Marsh, Australia to Almendralejo, Spain, creating content so "vomit-inducing" a bipartisan US Congress prohibited it in the Take it Down Act last year. Yet Grok placed tools with similar functionality within reach of any aspirational sex offender with X access. Public complaints metastasised over new year while the platform generated up to 7,751 sexualised images per hour. Across Australia, the US, the UK, EU states, and many others, it's not just the consumers of child sexual abuse material and nonconsensual image-based abuse who are criminalised. It's also the makers. It's the publishers. It's the hosts. Musk's response? First, he posted laugh-cry emojis at "bikinified" images, then his company claimed it had somehow restricted the service just by paywalling their generation. Condemned as insufficient, Musk subsequently published a statement saying that using Grok to make "illegal content" would draw equivalent punishment to uploading it - ignoring Grok's facilitation role. When Britain joined other countries - notably Malaysia, Indonesia, Australia and Brazil - accelerating investigations into X's compliance with local laws, Musk cried censorship ... and shared deepfaked images of Keir Starmer, tits out, in bikini. As scrutiny intensified this week, he ultimately declared he was not aware of any "naked underage images" being generated on the platform. Now, the tool's been removed. Note, 15 civil society, internet and child safety groups wrote to xAI last August, warning "a torrent of obviously nonconsensual deepfakes" was "entirely predictable". The definition of addiction is the compulsive repetition of harmful behaviour. My name's Van Badham, and I'm hooked on hopium, jonesing for any sign there's a democratic government left on earth now inspired to go full Gandalf against the Balrog and slap Musk down. X/Grok hosted image-based abuse, its owner was contemptuous of our sovereignty. It wasn't its first scandal, it won't be its last: responsible governments should simply ban it. We all know why they haven't: Musk uses his influence to wade into electoral contests of countries he doesn't even live in. His $44bn purchase, X, operates as a personal propaganda fountain, platforming his preferred flavours of far-right crap at such strength and volume that Stockholm-syndromed Twitter remnants mistake it for public opinion. It's not, but self-recruited digital stormtroopers mobilise from its public permission structures into acts of unforgivable cruelty. Images of Renee Good's dead body was being digitally altered by Grok within days of her being killed by an ICE agent. There was a time when leaders sought government to influence history, not to roll over supine to unelectable dweebs they would have rightfully avoided at high school. US SecDef Pete Hegseth's incomprehensible announcement this week that - yes - Musk's very same Grok will be the AI integrated into the Pentagon's military systems guarantees an IT lesson in "garbage in, garbage out" on an epic historical scale; but the political timeliness of global outrage and horror towards X/Grok's cumulation of reckless behaviours may be everyone else's best chance to escape it. The alternative is give in, give up ... and accept reality in the image of the Grok that Musk built: ugly as a cybertruck, unfunny as a sink - and as powerless as a child stripped naked by adults while other adults stand around them doing nothing.

Malaysia Will Take Legal Action Against Musk's X and XAI Over Misuse of Grok Chatbot

KUALA LUMPUR, Malaysia (AP) -- Malaysian authorities said Tuesday they will take legal action against Elon Musk's social media platform X and its artificial intelligence unit xAI, accusing the companies of failing to ensure the safety of users of its Grok chatbot. The move came just days after Malaysia and Indonesia became the first countries to block access to Grok, as concerns grow that it is being misused to generate sexually explicit and nonconsensual images. The Malaysian Communications and Multimedia Commission said in a statement that it has identified the misuse of Grok to generate and distribute harmful content including sexually explicit, indecent, extremely offensive as well as non-consensual manipulated images. It said it served notices to X and xAI this month to remove the harmful content but no action has been taken. "Content allegedly involving women and children is a matter of great concern. Such conduct is against Malaysian law and undermines the security commitments" stated by the companies, it said. The commission has appointed a lawyer and said legal proceedings would begin soon. There is growing scrutiny of generative AI tools that can produce realistic images, sound and text, and concern that existing safeguards are failing to prevent their abuse. Launched in 2023, Grok is free to use on X. An image generator feature, Grok Imagine, was added last year and included a so-called spicy mode that can generate adult content. Grok has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Grok has come under pressure in the European Union, India and the United Kingdom, which said Monday it was moving to criminalize "nudification apps." Britain's media regulator also launched an investigation into whether Grok broke the law by allowing users to share sexualized images of children. Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn't fully address the problem. Musk and his companies have not publicly commented on the Southeast Asian restrictions. xA1 has been giving an automated reply to media queries which stated, "Legacy Media Lies."

2 Countries Ban Musk's Grok AI Over Sexually Explicit Images

KUALA LUMPUR, Malaysia (AP) -- Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk's company xAI, as concerns grew among global authorities that it was being misused to generate sexually explicit and nonconsensual images. The moves reflect growing scrutiny of generative AI tools that can produce realistic images, sound and text and concern that existing safeguards are failing to prevent their abuse. The Grok chatbot, which is accessed through Musk's social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn't fully address the problem. An emailed request for comment by The Associated Press to xAI resulted in an automated reply from the media support email address which stated, "Legacy Media Lies." This was the same message received from a different email when asked for comment regarding the global backlash.

Elon Musk's NSFW Grok AI chatbot is forcing countries to take this extreme step

Indonesia and Malaysia are the first two countries to ban Elon Musk's artificial intelligence tool Grok, after the generative AI essentially flooded the social media platform X with sexually explicit, lewd images of young girls and women, made without their consent. Musk folded the generative AI tool into X when he took over Twitter, promising "free speech." However, critics say it is instead an example of how generative AI, without clear guardrails and regulation, can result in harm. Here's what to know. In short, users are typing simple prompts into the AI tool on X to "digitally undress" girls and women, some of which appear to be minors, triggering the chatbot to remove clothing from the real photos and even placing the subjects in sexually suggestive poses.

Grok Banned in Indonesia and Malaysia Following Deepfake Image Concerns

Recently, Grok's image editing on X was limited to verified users Grok has been temporarily suspended in Indonesia and Malaysia. The xAI's artificial intelligence (AI) chatbot received global criticism after it began editing images of users posted publicly on X (formerly Twitter). The feature was widely exploited by some users to generate sexually inappropriate images of others without consent, and many countries, including the European Union (EU), India, Indonesia, and Malaysia, had questioned the social media platform and xAI over safety guardrails. Recently, the feature was limited to only verified users, but the companies did not address the core feature that resulted in the sexualised deepfakes. Indonesia and Malaysia Suspend Grok In a post on X, the official handle of the Malaysian Communications and Multimedia Commission (MCMC) issued a media statement highlighting that Grok was being blocked temporarily in the country after the Elon Musk-owned social media platform failed to implement protective measures. "This action was taken following repeated misuse of Grok to generate pornographic content, sexually explicit content, grossly indecent content and non-consensual manipulated images, including content involving women and children, despite regulatory engagements having been issued to X and xAI," the official statement mentioned (translated via Google). Notably, the regulator said it sent notice to both entities on January 3 and on January 8 to demand the implementation of effective technical protection measures and content oversight. MCMC claimed that in response, X only focused on user-initiated reporting mechanisms and did not provide a solution to directly address the risks that arise from an AI tool that can generate realistic deepfakes. On Saturday, Indonesia's communication and digital minister, Meutya Hafid, also issued a statement announcing the temporary ban on Grok's services in the country. The statement read, "To protect women, children, and the entire community from the risk of fake pornographic content generated using artificial intelligence technology, the Government, through the Ministry of Communication and Digital, has temporarily suspended access to the Grok application." Additionally, the minister called the practice of generating non-consensual sexual deepfakes a "serious violation of human rights, dignity, and the security of citizens in the digital space." The ministry has also asked X to provide clarification about the feature on an urgent basis.

Malaysia, Indonesia become first to block Musk's Grok over AI deepfakes

KUALA LUMPUR, Malaysia (AP) -- Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk's xAI, after authorities said it was being misused to generate sexually explicit and non-consensual images. The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse. The Grok chatbot, which is accessed through Musk's social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Regulators in the two Southeast Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesia's Communication and Digital Affairs Minister Meutya Hafid said in a statement Saturday. The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI. Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director general of digital space supervision, said in a separate statement. He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm. In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday after what it said was "repeated misuse" of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors. The regulator said notices issued this month to X Corp. and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they've directly created or replies to posts from other users. Last summer the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The Southeast Asian restrictions come amid mounting scrutiny of Grok elsewhere, including in the European Union, Britain, India and France. Grok last week limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it did not fully address the problem.

Malaysia, Indonesia become first to block Musk's Grok over AI deepfakes

KUALA LUMPUR, Malaysia -- Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk's xAI, after authorities said it was being misused to generate sexually explicit and non-consensual images. The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse. The Grok chatbot, which is accessed through Musk's social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Regulators in the two Southeast Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesia's Communication and Digital Affairs Minister Meutya Hafid said in a statement Saturday. The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI. Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director general of digital space supervision, said in a separate statement. He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm. In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday after what it said was "repeated misuse" of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors. The regulator said notices issued this month to X Corp. and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they've directly created or replies to posts from other users. Last summer the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The Southeast Asian restrictions come amid mounting scrutiny of Grok elsewhere, including in the European Union, Britain, India and France. Grok last week limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it did not fully address the problem.

Indonesia and Malaysia are first countries to block Grok AI

Elon Musk's AI chatbot Grok is facing mounting scrutiny, including in the European Union, Britain, India and France, for being misused to generate sexually explicit and non-consensual images. Malaysia and Indonesia blocked Grok, the artificial intelligence (AI) chatbot developed by Elon Musk's xAI, after authorities said it was being misused to generate sexually explicit and non-consensual images. The countries are the first to ban the generative AI tool, reflecting a growing global concern that it can produce realistic images, sound, and text, while existing safeguards fail to prevent their abuse. The Grok chatbot, which is accessed through Musk's social media platform X, has been criticised for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Regulators in the two Southeast Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesia's Communication and Digital Affairs Minister Meutya Hafid said in a statement Saturday. The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI. Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director general of digital space supervision, said in a separate statement. He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm. In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday after what it said was "repeated misuse" of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors. The regulator said notices issued this month to X Corp. and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they've directly created or replies to posts from other users. Last summer, the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The Southeast Asian restrictions come amid mounting scrutiny of Grok elsewhere, including in the European Union, Britain, India, and France. Grok last week limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it did not fully address the problem.

Malaysia to take legal action against X, xAI over Grok concerns - The Economic Times

The Malaysian Communications and Multimedia Commission (MCMC) already suspended access to Grok, which is integrated into both the X and xAI platforms, on Sunday. It has now appointed solicitors to commence "legal proceedings" against X and xAI, the regulatory body said, without specifying what form the proceedings would take.Malaysian regulators said Tuesday they would take legal action against Elon Musk's X and xAI over user safety concerns sparked by their in-built artificial intelligence chatbot Grok. Grok is facing a global backlash for allowing users to create and share sexualised pictures of women and children using simple text prompts. The Malaysian Communications and Multimedia Commission (MCMC) already suspended access to Grok on Sunday. It has now appointed solicitors to commence "legal proceedings" against X and xAI, the regulatory body said, without specifying what form the proceedings would take. Grok is integrated into both the X and xAI platforms. "This action concerns their failure to ensure user safety in Malaysia in relation to the use of Grok," the MCMC said. It said authorities had "identified the misuse of Grok to generate and disseminate harmful content" deemed obscene, sexually explicit and involving non-consensual manipulated images. "Content allegedly involving women and minors is of serious concern", violated Malaysian law and undermined the companies' commitments to safety, the regulator added. It said it issued notices to X and xAI on January 3 and January 8 ordering the removal of the content but the companies took no action. It also accused the companies of failing to enforce their own internal policies over harmful content, and said they may still be held liable even if such content was generated by users. Neighbouring Indonesia on Saturday became the first country to block access to Grok entirely, and the bot's image creation feature has been limited to paying subscribers in other markets. European officials and tech campaigners have criticised the latter move, saying it fails to address concerns about sexualised deepfakes.

Malaysia, Indonesia Become First to Block Musk's Grok Over AI Deepfakes

KUALA LUMPUR, Malaysia (AP) -- Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk's xAI, after authorities said it was being misused to generate sexually explicit and non-consensual images. The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse. The Grok chatbot, which is accessed through Musk's social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Regulators in the two Southeast Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesia's Communication and Digital Affairs Minister Meutya Hafid said in a statement Saturday. The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI. Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director general of digital space supervision, said in a separate statement. He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm. In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday after what it said was "repeated misuse" of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors. The regulator said notices issued this month to X Corp. and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they've directly created or replies to posts from other users. Last summer the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The Southeast Asian restrictions come amid mounting scrutiny of Grok elsewhere, including in the European Union, Britain, India and France. Grok last week limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it did not fully address the problem.

X 'acting to comply with UK law' after outcry over sexualised images

New polling suggests 58% of Britons think X should be banned in the UK if the social network fails to crack down on nonconsensual images Elon Musk's X is understood to have told the government it is acting to comply with UK law, after nearly a fortnight of public outcry at the use of its AI tool Grok to manipulate images of women and children by removing their clothes. Keir Starmer told the House of Commons on Wednesday that photographs generated by Grok had been "disgusting" and "shameful", but said he had been informed "X is acting to ensure full compliance with UK law". "If so, that is welcome," the prime minister said. "But we are not going to back down. They must act. We will take the necessary measures. We will strengthen existing laws and prepare for legislation if it needs to go further, and Ofcom will continue its independent investigation." Ofcom, the media regulator, launched its investigation into X on Monday after a deluge of sexual images appeared on Musk's platform. Government officials are understood to have been speaking with X, but ministers are monitoring the impact of the steps taken by the Texas-based company. There is frustration that guardrails other AI providers have put in place to prevent such images being created appear not to be used by Grok. "We are keeping a close watch on the situation," Starmer said. He spoke as new polling showed 58% of Britons believe X should be banned in the UK if the platform doesn't crack down on AI-generated, nonconsensual images. More In Common's research also found 60% believe UK ministers should come off X, and 79% fear AI misuse is set to become a bigger problem. In recent days, X is understood to have restricted the @grok account, which many users have been asking to partially undress celebrities and others, so it no longer generates images of real people in revealing clothing. The sharing of nonconsensual intimate images, such as those created by asking an AI to put people in underwear and bikinis and in sexual poses, is illegal under the Online Safety Act. Last week, the Internet Watch Foundation, a UK-based watchdog, said it had seen users on a dark web forum boasting of using the Grok app to create sexualised and topless imagery of girls aged between 11 and 13. On Wednesday, Musk said he was "not aware of any naked underage images generated by Grok. Literally zero." "Obviously, Grok does not spontaneously generate images, it does so only according to user requests," he wrote in an X post. "When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state. There may be times when adversarial hacking of Grok prompts does something unexpected. If that happens, we fix the bug immediately." Meanwhile, Liz Kendall, the technology secretary, renewed criticism of xAI over its decision to limit Grok's image generation and editing functions to paying subscribers, calling it "a further insult to victims, effectively monetising this horrific crime". In a letter to MPs on the Commons select committee for science, innovation and technology, she said a wider ban on AI-enabled nudification tools "will apply to applications that have one despicable purpose only: to use generative AI to turn images of real people into fake nude pictures and videos without their permission". But the committee chair, Chi Onwuruh, has criticised the government's slowness in applying the ban given "reports of these disturbing Grok deepfakes appeared in August 2025". She said it is "unclear whether this ban - which appears to be limited to apps that have the sole function of generating nude images - will cover multipurpose tools like Grok".

Malaysia and Indonesia: The first countries to block Musk's Grok over sexualized AI images

Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot built by Elon Musk's company xAI, after authorities said it was being used to generate sexualized and nonconsensual images. Regulators in both countries said Grok's safeguards failed to stop the creation and spread of manipulated pornographic content, including images involving women and minors. Indonesia moved first over the weekend, followed by Malaysia, citing risks to privacy, dignity and public safety when real photos are altered without consent. Indonesian Communications Minister Meutya Hafid called nonconsensual sexual deepfakes a serious human rights violation, warning that AI misuse is outpacing current protections. Officials said early checks showed Grok could easily be used to fabricate explicit images of ordinary citizens. Malaysia's communications authority pointed to repeated misuse of the chatbot and said it had already issued notices to X Corp. and xAI demanding tougher controls. Access will remain blocked, it said, until effective safeguards are put in place to prevent further abuse. The Southeast Asian crackdown adds to growing pressure elsewhere. Britain has opened an investigation into Grok and is moving to outlaw so-called "nudification apps," while regulators in Europe and India are also watching closely, turning Musk's chatbot into a global test case for how far governments will go to police AI-generated content.

Malaysia And Indonesia Clamp Down On Elon Musk's Grok Over Nonconsensual Explicit Content Concerns

Enter your email to get Benzinga's ultimate morning update: The PreMarket Activity Newsletter Malaysia and Indonesia have blocked access to Elon Musk's artificial intelligence (AI) chatbot, Grok after discovering that the tool could be used to generate and distribute images containing nonconsensual explicit content and child sexual abuse material (CSAM). Malaysian regulators imposed temporary limits on the chatbot, developed by xAI, on Sunday, citing "repeated failures by X Corp" to mitigate related risks. https://x.com/MCMC_RASMI/status/2010235447205921211 The decision followed similar steps taken by Indonesia a day earlier, when authorities also summoned X officials for discussions. Global Scrutiny Mounts Over Grok AI Image Tool The ban by Malaysia and Indonesia is the latest in line after the AI image generator faced international backlash for creating nonconsensual sexualized images, including those of minors, prompting an investigation by French authorities. Regulators across Europe and India opened inquiries into the AI's image-generation tools. Brazilian lawmakers have also called for Grok to be suspended pending investigations, while the U.K. media regulator Ofcom has asked X to provide details about the AI tool's operations. xAI founder Elon Musk has since issued a warning, stating that anyone found misusing Grok to generate illegal content will face consequences, by taking down the content, permanently suspending the accounts involved, and cooperating with local governments and law enforcement when required. Image via Shutterstock Disclaimer: This content was partially produced with the help of AI tools and was reviewed and published by Benzinga editors. Market News and Data brought to you by Benzinga APIs

Malaysia suspends access to Musk's Grok AI

Malaysia suspended access to Elon Musk's chatbot Grok over AI-generated pornographic content, the country's tech regulator said on Sunday. The decision follows global backlash after it emerged that Grok's image creation feature allowed users to sexualize pictures of women and children using simple text prompts. On Saturday Indonesia became the first country to deny all access to the tool, which has been restricted to paying subscribers elsewhere. The Malaysian Communications and Multimedia Commission said in a statement it had "directed a temporary restriction on access to the Grok artificial intelligence for users in Malaysia" with immediate effect. "This action follows repeated misuse of Grok to generate obscene, sexually explicit, indecent, grossly offensive and non-consensual manipulated images," the regulator said. The statement cited "content involving women and minors, despite prior regulatory engagement and formal notices" issued to Musk's X Corp. and xAI startup which developed Grok. The AI tool is integrated into social media platform X. The Malaysian regulator said it deemed the platform's safeguards inadequate, adding that access would resume only after the required changes are verified. X Corp. had "failed to address the inherent risks posed by the design and operation of the AI tool", relying "primarily on user-initiated reporting mechanisms", the regulator said. European officials and tech campaigners on Friday slammed Grok after its controversial image creation feature was restricted to paying subscribers, saying the change failed to address concerns about sexualized deepfakes. Grok had appeared to deflect the criticism with a new monetization policy, posting on X on Thursday that image generation and editing were now "limited to paying subscribers", alongside a link to a premium subscription.

Elon Musk's Grok AI restricted in Malaysia over backlash to sexualised images

Malaysia on Sunday temporarily blocked access to Grok, joining a growing list of countries taking action after the generative artificial intelligence chatbot sparked a global backlash by allowing users to create and publish sexualised images. xAI, the Elon Musk-led firm behind Grok, on Thursday said it would restrict image generation and editing to paying subscribers as it addressed lapses that allowed users on X to produce sexualised content of others, often without consent. On Saturday, Indonesia became the first country to temporarily deny access to the bot. In a statement on Sunday, the Malaysian Communications and Multimedia Commission (MCMC) said it would restrict access to Grok following repeated misuse of the tool "to generate obscene, sexually explicit, indecent, grossly offensive, and non-consensual manipulated images, including content involving women and minors." MCMC said it issued notices to X and xAI this month demanding the implementation of effective technical and moderation safeguards, but the responses it received relied primarily on user-initiated reporting mechanisms and failed to address the risks posed by the design and operation of the AI tools. "MCMC considers this insufficient to prevent harm or ensure legal compliance," it said. 'Legacy media lies' xAI replied to a Reuters email seeking comment with what seemed to be an automated response: "Legacy Media Lies." X did not immediately respond to a request for comment. MCMC said access to Grok would be restricted until effective safeguards were implemented, adding that it was open to engaging with the firms. Muslim-majority Malaysia has strict laws governing online content, including a ban on obscene and pornographic materials. It has put internet companies under greater scrutiny in recent years in response to what it calls a rise in harmful content. Malaysia is considering banning users under 16 from accessing social media.

Malaysia, Indonesia become first to block Musk's Grok over AI deepfakes

The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse. The Grok chatbot, which is accessed through Musk's social media platform X, has been criticised for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk's xAI, after authorities said it was being misused to generate sexually explicit and non-consensual images. The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse. The Grok chatbot, which is accessed through Musk's social media platform X, has been criticised for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Regulators in the two Southeast Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesia's Communication and Digital Affairs Minister Meutya Hafid said in a statement Saturday. The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI. Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director general of digital space supervision, said in a separate statement. He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm. In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday after what it said was "repeated misuse" of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors. The regulator said notices issued this month to X Corp. and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they've directly created or replies to posts from other users. Last summer, the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The Southeast Asian restrictions come amid mounting scrutiny of Grok elsewhere, including in the European Union, Britain, India and France. Grok last week limited image generation and editing to paying users following a global backlash over sexualised deepfakes of people, but critics say it did not fully address the problem.

Elon Musk's X and xAI face legal action by Malaysia over Grok AI deepfake images

Malaysia's communications regulator said it will take legal action against Elon Musk's social media platform X and AI startup xAI (X.AI) due to concerns over user safety, just days after it restricted access to AI service Grok over deepfake images. Malaysia's communications regulator is pursuing legal action and temporarily restricting Grok access due to concerns over user-generated harmful and non-consensual manipulated images. Malaysian regulators state X Corp. and xAI are liable due to their control over Grok's design and moderation, regardless of user-generated content, and cite failed internal safeguards. Other countries like Indonesia and the U.K. are investigating or imposing restrictions, and U.S. senators have urged Apple and Google to remove related apps due to illegal content generation.

Indonesia Temporarily Blocks Access to Grok Over Sexualised Images

JAKARTA, Jan 10 (Reuters) - Indonesia temporarily blocked Elon Musk's Grok chatbot on Saturday due to the risk of AI-generated pornographic content, becoming the first country to deny access to the AI tool. The move comes after governments and regulators from ⁠Europe to Asia have condemned and some have opened inquiries into sexualised content on the app. xAI, the startup behind Grok, said on Thursday it was restricting image generation and editing to paying subscribers as it tried to fix safeguard lapses that had ⁠allowed sexualised outputs including depictions of scantily clad children. "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space," Communications and Digital Minister Meutya Hafid said in a statement. The ministry has also summoned X officials to discuss the matter. Musk said on X that anyone using Grok to make illegal content would suffer the same consequences as if they had uploaded illegal content. xAI replied to Reuters' email seeking comment with what seemed to be an automated response: "Legacy Media Lies". X did not immediately respond to a request for comment. Indonesia, with the world's biggest Muslim population, has strict rules that ban the sharing online of content deemed obscene. (Reporting by Fransiska Nangoy; Editing by Stephen Coates)

Liz Kendall's response to X 'nudification' is good - but not enough to solve the problem | Nana Nwachukwu

Big tech companies cannot be trusted. It is not enough that they remove harm when they find it - the law must make their systems prevent harm On X, a woman posts a photo in a sari, and within minutes, various users are underneath the post tagging Grok to strip her down to a bikini. It is a shocking violation of privacy, but now a familiar and commonplace practice. Between June 2025 and January 2026, I documented 565 instances of users requesting Grok to create nonconsensual intimate imagery. Of these, 389 were requested in just one day. Last Friday, after a backlash against the platform's ability to create such nonconsensual sexual images, X announced that Grok's AI image generation feature would only be available to subscribers. Reports suggest that the bot now no longer responds to prompts to generate images of women in bikinis (although apparently will still do so for requests about men). But as the technology secretary, Liz Kendall, rightly states, this action "does not go anywhere near far enough". Kendall has announced that creating nonconsensual intimate images will become a criminal offence this week, and that she will criminalise the supply of nudification apps. This is appropriate, given X's weak response. Placing the feature behind a paywall means that the platform can more directly profit off the online dehumanisation and sexual harassment of women and minors. And stopping the "bikini" responses after public censure and the threat of legislation is the least X can do - the bigger question is why it was even possible in the first place. These measures are a step forward. The shadow technology secretary, Julia Lopez, suggested in her response that the government was overreacting, that this was just "a modern-day iteration of an old problem", no different from crude drawings or Photoshop. She's wrong. The scale is different. The accessibility is different. The speed is different. With Photoshop, there is a technical skill required, as well as direct publication by a user, which places all actions except platform provision squarely on them. In this case, though, the user makes a regular text reply with a request, and Grok generates and publishes criminal abuse to a massive audience. Kendall's approach criminalises users who create or alter these images, and the companies that supply dedicated nudification tools. That's where it misses the point. Grok and most prominent image-generation tools are not dedicated nudification tools. They are general-purpose AI with weak safeguards. Kendall is not asking platforms to implement proactive detection. The law waits for harm to happen, then punishes. The drawbacks of this approach are obvious. I observed this material being generated for months before the mainstream backlash began. These are harmful images that were generated that still exist, and perhaps were saved and shared across other platforms. For the victims of this AI sexual abuse material, regulation after the fact won't help. For harm that is structurally amplified in this manner, the approach must be preventive, not reactionary. Another more fundamental problem is that while the UK pushes AI safety regulation, the US is moving in the opposite direction. The Trump administration wants to "enhance the United States' global AI dominance through a minimally burdensome national policy framework for AI". Under this framework, there is little incentive for American AI companies to regulate misuse of their products. This matters because AI regulation is incomplete without cross-border collaboration. Kendall can criminalise users in the UK, she can threaten to ban X entirely. But she cannot stop Grok from being programmed in San Francisco. She cannot force OpenAI or Anthropic or any other US company to prioritise safety over speed. Without US cooperation, we are trying to regulate a transnational technology with national laws. While this wrangling over regulation and updating policy plays out, many victims, and other women online, will be wondering what this new era of AI-enabled online sexual harassment means for them, and questioning their participation on global social media platforms. If my image has been digitally altered, how do I get justice if the perpetrator is halfway across the world? Transparency in the practices of AI companies is in decline - so how could the same companies be trusted to be accountable and audit systems that reproduce harm? The truth is that these companies cannot be trusted. This is why globally, regulation needs to shift from "remove harm when you find it" to "prove that your system prevents harm". We must code power into the process by requiring mandatory input filtering, independent audits and licensing conditions that make prevention a legal technical requirement. This means it may catch the harm before it materialises, enabling regulation to minimise harmful behaviour by these AI companies before their products are deployed. This is the type of work that we at the AI Accountability Lab in the Adapt Centre at Trinity College Dublin are pushing forward through our research. Regulation after the fact is better than nothing. However, it offers little to the victims who have already been harmed, and sidesteps the conspicuous absence of law enforcement in addressing these platform harms.

Malaysia suspends access to Musk's Grok AI - The Economic Times

Malaysia has blocked access to Elon Musk's Grok chatbot after it was widely misused to create pornographic and abusive images. Regulators said safeguards were weak, despite earlier warnings, and cited sexualised content involving women and children. Access will return only after changes are made, following similar action by Indonesia earlier.Malaysia suspended access to Elon Musk's chatbot Grok over AI-generated pornographic content, the country's tech regulator said on Sunday. The decision follows global backlash after it emerged that Grok's image creation feature allowed users to sexualise pictures of women and children using simple text prompts. On Saturday Indonesia became the first country to deny all access to the tool, which has been restricted to paying subscribers elsewhere. The Malaysian Communications and Multimedia Commission said in a statement it had "directed a temporary restriction on access to the Grok artificial intelligence for users in Malaysia" with immediate effect. When an AFP reporter in the Malaysian capital Kuala Lumpur fed Grok prompts on Sunday, there was no response. "This action follows repeated misuse of Grok to generate obscene, sexually explicit, indecent, grossly offensive and non-consensual manipulated images," the regulator said. The statement cited "content involving women and minors, despite prior regulatory engagement and formal notices" issued to Musk's X Corp. and xAI startup which developed Grok. The AI tool is integrated into social media platform X. The Malaysian regulator said it deemed the platform's safeguards inadequate, adding that access would resume only after the required changes are verified. X Corp. had "failed to address the inherent risks posed by the design and operation of the AI tool", relying "primarily on user-initiated reporting mechanisms", the regulator said. European officials and tech campaigners on Friday slammed Grok after its controversial image creation feature was restricted to paying subscribers, saying the change failed to address concerns about sexualised deepfakes. Grok had appeared to deflect the criticism with a new monetisation policy, posting on X on Thursday that image generation and editing were now "limited to paying subscribers", alongside a link to a premium subscription.

Grok Limits Image Generation On X To Paid Users

Moving Grok's image generation behind X's (formerly Twitter) premium paywall does not fix the problem. Instead, it monetises abuse. By restricting a tool already linked to the creation of non-consensual sexual imagery to paying users, X has turned access to a harmful capability into a subscription perk, rather than treating it as a safety failure that demands urgent reform. The platform still allows the same image manipulation on the standalone Grok website after basic age verification, which undercuts any claim that it has taken meaningful safeguards seriously. Moreover, this approach sends a dangerous signal. If abuse becomes a revenue stream, platforms risk normalising the very behaviour regulators across Asia, Europe and India are now trying to stop. Indonesia and Malaysia have already blocked Grok. India has threatened X with the loss of safe-harbour protections. The EU has ordered document retention. Yet X's answer is not structural reform, but a checkout page. Meanwhile, Elon Musk continues to play the boy who cried censorship. He blames users, not systems. He endorses comments that compare Grok to a pen with output solely based on the user, and its critics to enemies of free speech. When governments intervene, he reaches for the language of suppression rather than responsibility. However, this is not about speech. It is about consent, harm and accountability. Until X and xAI redesign Grok with real safeguards -- and stop selling access to a tool already proven to cause damage -- regulators will keep circling to set an example. Since Friday, Grok's responses to requests on X indicated that its image generation and editing features are now limited to paid subscribers of the social media platform. When users asked the AI chatbot to create or alter images directly in reply threads, Grok returned a message stating that image generation was accessible only to paying members, directing them toward X's premium subscription page to unlock those capabilities. This change followed a global backlash over the chatbot's use in producing sexualised and inappropriate images in replies on the platform. The policy shift comes amid controversy over Grok being used to digitally undress people in images on user demand, a trend that drew sustained criticism and scrutiny. Users on X had been prompting the AI tool to edit photos of real women and children by removing or altering clothing to create sexualised depictions, which ran counter to platform rules. Notably, these prompts appeared frequently in public posts, and Grok complied even in cases of generating Child Sexual Abuse Material (CSAM) before the recent restrictions. Despite the paid-only limitation on X itself, image generation remains possible for free on the standalone Grok website after age verification, and users continue to make image requests that the chatbot fulfils on X when they are premium subscribers. Indonesia became the first country to temporarily block access to Grok on January 10, 2026, over the generation of non-consensual sexual deepfakes. Afterwards, Malaysia said it had restricted access to Grok on January 11, 2026, with its communications regulator framing the move as temporary, adding that the ban would stay until stronger safeguards beyond user reporting mechanisms are in place. Meanwhile, in the United Kingdom, authorities escalated scrutiny rather than issuing a ban: the Information Commissioner's Office said it had contacted X and xAI to seek clarity on compliance with UK data protection law and stated that it would assess whether further action was needed after reviewing the response. Furthermore, UK Prime Minister Keir Starmer said that the use of Grok to generate non-consensual sexualised images was 'disgraceful', 'disgusting' and 'not to be tolerated'. He made clear that the government had all options "on the table", including a potential ban on X. Meanwhile in the US, three Democratic senators sent a letter to Apple and Google urging them to remove the X and Grok apps from their app stores. Elsewhere, the German justice ministry announced it was preparing measures against harmful AI-driven image manipulation, following concerns about non-consensual image generation linked to Grok. In Italy, the data protection authority issued a warning that using AI tools to create undressed deepfake images of real people without consent can trigger serious privacy violations and, in some cases, criminal offences against providers. Furthermore, at the EU level, the European Commission ordered X to retain Grok-related internal documents and data until the end of 2026 as it assessed compliance under EU rules. On January 2, 2026, the Ministry of Electronics and Information Technology (MeitY) issued a formal notice to X, telling it to immediately remove vulgar, obscene and unlawful content generated through Grok and to submit a detailed action taken report (ATR) within 72 hours. The ministry cautioned that failure to comply could lead to serious legal consequences and loss of safe harbour protections under Indian law. "Safe harbour" shields intermediaries from liability for third-party content if they make reasonable efforts to block unlawful material. The ministry told X that if Grok actively generated and published sexually explicit imagery, the company could not claim this immunity unless it demonstrated compliance with due diligence rules to prevent and remove prohibited content. If X did not make such efforts, authorities said it could face legal liability as a publisher of illegal content rather than as a neutral platform. After X sought an extension, MeitY pushed the deadline by 48 hours and then assessed X's initial ATR as inadequate, highlighting that it lacked specific information on takedowns and future prevention measures. Subsequently, Indian government sources said X acknowledged moderation lapses, removed about 3,500 items of content, and deleted more than 600 accounts, while assuring authorities that it would comply with the law and curb obscene imagery going forward. Musk has framed the backlash over Grok's misuse as a problem caused by users, not the platform itself. In early January, as Grok drew criticism for generating non-consensual explicit images, Musk posted on X that "Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content". He said this in response to a user remarking that, "Some people are saying Grok is creating inappropriate images. But that's like blaming a pen for writing something bad. A pen doesn't decide what gets written. The person holding it does. Grok works the same way. What you get depends a lot on what you put in". This framing portrays Grok as a neutral tool and places the onus for harmful output on those who prompt the AI rather than on the platform or the AI tool itself. Furthermore, as the UK considers banning X over Grok's generation of sexualised images, Musk escalated his rhetoric, describing the British government as "fascist" and claiming authorities simply wanted to "suppress free speech".

Malaysia taking legal action against Musk's X, xAI over Grok safety...

KUALA LUMPUR, Malaysia -- Malaysian authorities said Tuesday they will take legal action against Elon Musk's social media platform X and its artificial intelligence unit xAI, accusing the companies of failing to ensure the safety of users of its Grok chatbot. The move came just days after Malaysia and Indonesia became the first countries to block access to Grok, as concerns grow that it is being misused to generate sexually explicit and nonconsensual images. The Malaysian Communications and Multimedia Commission said in a statement that it has identified the misuse of Grok to generate and distribute harmful content including sexually explicit, indecent, extremely offensive as well as non-consensual manipulated images. It said it served notices to X and xAI this month to remove the harmful content but no action has been taken. "Content allegedly involving women and children is a matter of great concern. Such conduct is against Malaysian law and undermines the security commitments" stated by the companies, it said. The commission has appointed a lawyer and said legal proceedings would begin soon. There is growing scrutiny of generative AI tools that can produce realistic images, sound and text, and concern that existing safeguards are failing to prevent their abuse. Launched in 2023, Grok is free to use on X. An image generator feature, Grok Imagine, was added last year and included a so-called spicy mode that can generate adult content. Grok has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Grok has come under pressure in the European Union, India and the United Kingdom, which said Monday it was moving to criminalize "nudification apps." Britain's media regulator also launched an investigation into whether Grok broke the law by allowing users to share sexualized images of children. Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn't fully address the problem. Musk and his companies have not publicly commented on the Southeast Asian restrictions. xA1 has been giving an automated reply to media queries which stated, "Legacy Media Lies."

Malaysia and Indonesia become the first countries to block Musk's Grok over sexualized AI images

KUALA LUMPUR, Malaysia -- Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk's company xAI, as concerns grew among global authorities that it was being misused to generate sexually explicit and nonconsensual images. The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text and concern that existing safeguards are failing to prevent their abuse. The Grok chatbot, which is accessed through Musk's social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn't fully address the problem. An emailed request for comment by The Associated Press to xAI resulted in an automated reply from the media support email address which stated, "Legacy Media Lies." This was the same message received from a different email when asked for comment regarding the global backlash. Regulators in the two Southeast Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesia's Communication and Digital Affairs Minister Meutya Hafid said in a statement Saturday. The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI. Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director general of digital space supervision, said in a separate statement. He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm. In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday after what it said was "repeated misuse" of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors. The regulator said notices issued this month to X Corp. and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they've directly created or replies to posts from other users. Last summer the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The Southeast Asian restrictions come amid mounting scrutiny of Grok elsewhere, including in the European Union, the United Kingdom, India and France. Grok last week limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it did not fully address the problem. An attempt by The Associated Press to request a comment through email to xAI resulted in an automated reply from the media support email address which stated, "Legacy Media Lies." This was the same message received from a different email when asked for comment regarding the global backlash. On Monday, the U.K.'s media regulator said that it launched a formal investigation into whether Grok "complied with its duties to protect people in the U.K. from content that is illegal." The regulator, Ofcom, said that Grok-generated images of children being sexualized or people being undressed may amount to pornography or child sexual abuse material. "The content created and shared using Grok in recent days has been deeply disturbing," Technology Secretary Liz Kendall said.

Indonesia and Malaysia Ban Grok AI Over Fake Sexual Image Controversy; India May Act Next

India Issues Notice to X as Indonesia and Malaysia Ban Grok AI Over Deepfake Abuse and Online Safety Violations The misuse of AI tools has sparked fresh concerns. Indonesia and Malaysia have banned Grok AI after reports of fake and sexual images created using the AI tool. These images were shared online without consent. The issue has raised serious worries about safety, privacy, and misuse of technology. Grok AI is linked to . The problem began when people started sharing sexual images of women made using AI. These images looked real but were fake. Many women were targeted without permission. The content spread fast and caused anger among users and officials.

Use of AI to harm women has only just begun, experts warn

While Grok has introduced belated safeguards to prevent sexualised AI imagery, other tools have far fewer limits "Since discovering Grok AI, regular porn doesn't do it for me anymore, it just sounds absurd now," one enthusiast for the Elon Musk-owned AI chatbot wrote on Reddit. Another agreed: "If I want a really specific person, yes." If those who have been horrified by the distribution of sexualised imagery on Grok hoped that last week's belated safeguards could put the genie back in the bottle, there are many such posts on Reddit and elsewhere that tell a different story. And while Grok has undoubtedly transformed public understanding of the power of artificial intelligence, it has also pointed to a much wider problem: the growing availability of tools, and means of distribution, that present worldwide regulators with what many view as an impossible task. Even as the UK announces that creating nonconsensual sexual and intimate images will soon be a criminal offence, experts say that the use of AI to harm women has only just begun. Other AI tools have much stricter safeguards in place. Asked to strip a photograph of a woman into a bikini, the large language model (LLM) Claude says: "I can't do that. I'm not able to edit images to change clothing or create manipulated photos of people." ChatGPT and Google's AI tool Gemini will create bikini images, but nothing more explicit. However, there are far fewer limits elsewhere. Users of the Grok forum on Reddit have been sharing tips on how to generate the most hardcore pornographic images possible using pictures of real women. On one thread, users were complaining that Grok would allow them to make images of women topless "after a struggle", but refused to generate genitals. Others have noticed that asking for "artistic nudity" gets around safeguards around stripping women completely naked. Beyond LLMs and major platforms is a whole ecosystem of websites, forums and apps devoted to nudification and the humiliation of women. These communities are increasingly finding pipelines to the mainstream, said Anne Craanen, a researcher at the Institute for Strategic Dialogue (ISD) working on tech-facilitated, gender-based violence. Communities on Reddit and Telegram discuss how to bypass guardrails to make LLMs produce pornography, a process known as "jailbreaking." Threads on X amplify information about nudification apps, which produce AI-generated images of women with their clothing removed, and how to use them. Craanen said the route for misogynistic content to reach the wider internet has grown broader, adding: "There is a very fruitful ground there for misogyny to thrive." Research from the ISD last summer found dozens of nudification apps and websites, which collectively received nearly 21 million visitors in May 2025. There were 290,000 mentions of these tools on X in June and July last year. Research by the American Sunlight Project in September found that there were thousands of ads for such apps on Meta, despite the platform's efforts to crack down on them. "There are hundreds of apps hosted on mainstream app stores like Apple and Google that make this possible," said Nina Jankowicz, a disinformation expert who co-founded the American Sunlight Project. "Much of the infrastructure of deepfake sexual abuse is supported by companies that we all use on a daily basis." Clare McGlynn, a law professor and expert in violence against women and girls from Durham University, said that she feared things would only get worse. "OpenAI announced last November that it was going to allow 'erotica' in ChatGPT. What has happened on X shows that any new technology is used to abuse and harass women and girls. What is it that we're going to see then on ChatGPT? "Women and girls are far more reluctant to use AI. This should be no surprise to any of us. Women don't see this as exciting new technology, but as simply new ways to harass and abuse us and try and push us offline." Jess Asato, Labour MP for Lowestoft, has been campaigning on this issue and said her critics have been gleefully creating and sharing explicit imagery of her - even since the restrictions on Grok. "It's still happening to me and being posted on X because I speak up about it," she added. Asato added that AI deepfake abuse has been happening to women for years, and is not limited to Grok. "I don't know why [action] has taken so long. I have spoken to so many victims of much, much worse." While the public Grok X account no longer generates pictures for those without a paid subscription, and there appear to have been guardrails put in place to stop it generating bikini pictures, its in-app tool has far fewer restrictions. Users are still able to create sexually explicit imagery based on fully clothed pictures of real people, with no restrictions for free users of X. Asked to strip a photograph down into bondage gear, it complies. It will also place women into sexually compromising positions, and smear them in white, semen-like substances. The point of creating deepfake nudes is often not just about sharing erotic imagery, but the spectacle of it, said Craanen - especially as the images flood platforms like X. "It's the actual back and forth of it, [trying] to shut someone down by saying, 'Grok, put her in a bikini,'" she said. "The performance of it is really important there, and really shows the misogynistic undertones of it, trying to punish or silence women. That also has a cascading effect on democratic norms and women's role in society."

Indonesia temporarily blocks access to Grok over sexualised images

xAI, the startup behind Grok, said on Thursday it was restricting image generation and editing to paying subscribers as it tried to fix safeguard lapses that had allowed sexualised outputs including depictions of scantily clad children. Indonesia temporarily blocked Elon Musk's Grok chatbot on Saturday due to the risk of AI-generated ⁠pornographic content, becoming the first country to deny access to the AI tool. The move comes after governments and regulators from Europe to Asia have condemned and some have opened inquiries into sexualised content on the app. xAI, the startup behind Grok, said on Thursday it was restricting image generation and editing to paying subscribers as it tried to fix safeguard lapses that had allowed sexualised outputs including depictions of scantily clad children. "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space," Communications and Digital Minister Meutya Hafid said in a statement. The ministry has also summoned X officials to discuss the matter. Musk said on X that anyone using Grok to make illegal content would suffer the same consequences as if ⁠they had uploaded ⁠illegal content. xAI replied to Reuters' email seeking comment with what seemed to be an automated response: "Legacy Media Lies". X did not immediately respond to a request for comment. Indonesia, with the world's biggest Muslim population, has strict rules that ban the sharing online of content deemed obscene.

Malaysia, Indonesia become first to block Musk's Grok over sexualized AI images

KUALA LUMPUR, Malaysia -- Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk's xAI, after authorities said it was being misused to generate sexually explicit and non-consensual images. The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse. The Grok chatbot, which is accessed through Musk's social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Regulators in the two Southeast Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesia's Communication and Digital Affairs Minister Meutya Hafid said in a statement Saturday. The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI. Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director general of digital space supervision, said in a separate statement. He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm. In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday after what it said was "repeated misuse" of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors. The regulator said notices issued this month to X Corp. and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they've directly created or replies to posts from other users. Last summer the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The Southeast Asian restrictions come amid mounting scrutiny of Grok elsewhere, including in the European Union, Britain, India and France. Grok last week limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it did not fully address the problem. An attempt by The Associated Press to request a comment through email to xAI resulted in an automated reply from the media support email address which stated, "Legacy Media Lies." This was the same message received from a different email when asked for comment regarding the global backlash.

Malaysia and Indonesia block Musk's Grok over explicit AI images | BreakingNews

Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk's xAI, after authorities said it was being misused to generate sexually explicit and non-consensual images. The moves reflect growing global concern over generative AI tools that can produce realistic images, sound and text, while existing safeguards fail to prevent their abuse. The Grok chatbot, which is accessed through Mr Musk's social media platform X, has been criticised for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children. Regulators in the two south-east Asian nations said existing controls were not preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia's government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday. "The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesia's communication and digital affairs minister Meutya Hafid said. The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI. Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director general of digital space supervision, said in a separate statement. He said such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm. In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday after what it said was "repeated misuse" of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors. The regulator said notices issued this month to X Corp and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms. "The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing," it said, adding that access will remain blocked until effective safeguards are put in place. Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they have directly created or replies to posts from other users. Last summer the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The south-east Asian restrictions come amid mounting scrutiny of Grok elsewhere, including in the European Union, the UK, India and France. Grok last week limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it did not fully address the problem

Malaysia blocks Elon Musk's Grok AI over fake, sexualised images

Country follows Indonesia in restricting access after global outcry over X's AI tool Malaysia has become the second country to temporarily block access to Elon Musk's Grok after a global outcry over the AI tool and its ability to produce fake, sexualised images. Malaysia said it would restrict access to Grok until effective safeguards were implemented, a day after similar action was taken by Indonesia. Several governments and regulators have taken action over Grok's image tool, which is embedded in the X social media site and has provoked outrage as it allows users to manipulate images of women and children to remove their clothing and put them in sexual positions. The Musk-led company that developed Grok, xAI, said last week the ability to generate and edit images would be "limited to paying subscribers" on X. Such users have provided personal details to the company and can be identified if the function is misused. The move has done little to quell anger about Grok, however. The Malaysian Communications and Multimedia Commission (MCMC) said on Sunday it would restrict access to Grok over the AI tool's ability "to generate obscene, sexually explicit, indecent, grossly offensive, and nonconsensual manipulated images, including content involving women and minors". Grok also functions through a separate website and app. It was unclear whether the Malaysian and Indonesian bans applied to Grok on X, the Grok site and app, or both. The Guardian received reports from Indonesia that people were still able to use Grok via the app and via X, although one reported the app was very slow. MCMC said it had issued notices to X and xAI this month to demand the implementation of effective technical and moderation safeguards. However, the responses it received relied mostly on user-initiated reporting mechanisms and failed to address the inherent risks posed by Grok, the MCMC said, adding that it considered this insufficient to prevent harm or ensure legal compliance. On Saturday, Indonesia also temporarily blocked the chatbot, with the country's communications and digital minister, Meutya Hafid, saying the government viewed "the practice of nonconsensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space". The UK has also raised the possibility of a ban if action is not taken, while on Saturday, Australia's prime minister, Anthony Albanese, condemned the use of generative AI to exploit or sexualise people without their consent as "abhorrent". Across Europe, regulators and politicians have also issued warnings over recent weeks. Germany's culture and media minister, Wolfram Weimer, called on the European Commission to take legal steps, warning of the "industrialisation of sexual harassment". Italy's data protection authority said that using AI tools to create explicit images of people without consent could amount to serious privacy violations and, in some cases, criminal offences. In France, government ministers said this month they had referred sexually explicit Grok-generated content circulating on X to prosecutors and alerted the French media regulator Arcom. India's IT and electronics ministry sent a formal notice to X on 2 January in relation to explicit images allegedly created through Grok, demanding the content be taken down and requiring a report on the actions being taken within 72 hours.

Malaysia blocks access to Elon Musk's Grok AI - VnExpress International

The decision follows global backlash after it emerged that Grok's image creation feature allowed users to sexualize pictures of women and children using simple text prompts. On Saturday Indonesia became the first country to deny all access to the tool, which has been restricted to paying subscribers elsewhere. The Malaysian Communications and Multimedia Commission said in a statement it had "directed a temporary restriction on access to the Grok artificial intelligence for users in Malaysia" with immediate effect. "This action follows repeated misuse of Grok to generate obscene, sexually explicit, indecent, grossly offensive and non-consensual manipulated images," the regulator said. The statement cited "content involving women and minors, despite prior regulatory engagement and formal notices" issued to Musk's X Corp. and xAI startup which developed Grok. The AI tool is integrated into social media platform X. The Malaysian regulator said it deemed the platform's safeguards inadequate, adding that access would resume only after the required changes are verified. X Corp. had "failed to address the inherent risks posed by the design and operation of the AI tool", relying "primarily on user-initiated reporting mechanisms," the regulator said. European officials and tech campaigners on Friday slammed Grok after its controversial image creation feature was restricted to paying subscribers, saying the change failed to address concerns about sexualized deepfakes. Grok had appeared to deflect the criticism with a new monetization policy, posting on X on Thursday that image generation and editing were now "limited to paying subscribers," alongside a link to a premium subscription.

Indonesia temporarily blocks access to Grok over sexualized images - VnExpress International

Indonesia temporarily blocked Elon Musk's Grok chatbot on Saturday due to the risk of AI-generated pornographic content, becoming the first country to deny access to the AI tool. The move comes after governments and regulators from Europe to Asia have condemned and some have opened inquiries into sexualised content on the app. xAI, the startup behind Grok, said on Thursday it was restricting image generation and editing to paying subscribers as it tried to fix safeguard lapses that had allowed sexualised outputs, including depictions of scantily clad children. "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space," Communications and Digital Minister Meutya Hafid said in a statement. The ministry has also summoned X officials to discuss the matter. Musk said on X that anyone using Grok to make illegal content would suffer the same consequences as if they had uploaded illegal content. xAI replied to Reuters' email seeking comment with what seemed to be an automated response: "Legacy Media Lies". X did not immediately respond to a request for comment. Indonesia, with the world's biggest Muslim population, has strict rules that ban the sharing online of content deemed obscene.

Indonesia blocks Musk's Grok chatbot due to risk of pornographic content

Move comes after governments and regulators from Europe to Asia have condemned the AI tool and some have opened inquiries into sexualised content Indonesia temporarily blocked Elon Musk's Grok chatbot on Saturday due to the risk of AI-generated pornographic content, becoming the first country to deny access to the AI tool. The move comes after governments, researchers and regulators from Europe to Asia have condemned and some have opened inquiries into sexualised content on the app. xAI, the startup behind Grok, said on Thursday it was restricting image generation and editing to paying subscribers as it tried to fix safeguard lapses that had allowed sexualised outputs, including depictions of scantily clad children. "The government views the practice of non-consensual sexual deepfakes as a serious violation of human rights, dignity, and the security of citizens in the digital space," communications and digital minister Meutya Hafid said in a statement. The ministry has also summoned X officials to discuss the matter. Musk said on X that anyone using Grok to make illegal content would suffer the same consequences as if they had uploaded illegal content. xAI replied to Reuters' email seeking comment with what seemed to be an automated response: "Legacy Media Lies". X did not immediately respond to a request for comment. Indonesia, with the world's biggest Muslim population, has strict rules that ban the sharing online of content deemed obscene. Indonesia's block follows Grok switching off its image creation function on Friday for the vast majority of users after the widespread outcry about its use to create sexually explicit and violent imagery. Musk has also been threatened with fines, regulatory action and reports of a possible ban on X in the UK. The tool has also been used to manipulate images of women to remove their clothes and put them in sexualised positions. The function to do so has been switched off except for paying subscribers. Australian prime minister Anthony Albanese has also expressed concern over the use of artificial intelligence on the Grok chatbot to enable exploitative sexual content. The prime minister on Saturday joined a growing list of international leaders including British counterpart Keir Starmer in criticising the social media platform. "The use of generative AI to exploit or sexualise people without their consent is abhorrent," he told reporters in Canberra. "The fact that this tool was used so that people were using its image creation function through Grok is, I think, just completely abhorrent. "It, once again, is an example of social media not showing social responsibility and Australians and indeed, global citizens deserve better." While the number of reports received by Australia's eSafety Office remains small, it says there has been a recent increase relating to the use of Grok to create sexualised or exploitative imagery. The watchdog warned on Friday it would use its powers including removal notices where such material meets the thresholds defined in the Online Safety Act. "X, Grok and a wide range of other services are also subject to systemic safety obligations to detect and remove child sexual exploitation material and other unlawful material as part of Australia's world-leading industry codes and standards," it said.

Indonesia and Malaysia ban Grok AI amid explicit image generation, will India be next?

India has asked for quick removal instead of a ban, but the situation has sparked debate on whether stronger action may follow. The controversy around Grok AI generating sexually explicit content shows no sign of slowing down as countries around the world take strict actions. While governments have been issuing notices to the Elon Musk-owned social media platform X regarding the removal of AI-generated obscene images, two countries have taken a stronger stance by blocking the AI tool entirely. This move is also seen as the strongest response to the growing risks to women's privacy. The incident has reignited the debate on whether India should also block Grok AI instead of merely issuing notices. Also read: Samsung Galaxy Z Fold 8 to launch soon: Check the display, specs, launch timeline, price and other details The issue started when people began sharing sexually explicit images of women made using AI on X. Many countries in Southeast Asia reacted strongly, and some even warned that they might block the platform. Malaysia and Indonesia were the first to suspend access to X in their region. Their governments said this step was taken because X failed to stop the spread of sexual deepfake images made without consent. They said such content is dangerous, especially for women and children, and breaks their national laws. Also read: Samsung Galaxy S26 Ultra India launch date, specifications, price and other latest leaks Earlier, the UK government also criticised the spread of sexual AI images. It called the content unacceptable and asked the media regulator to take strict action against platforms that do not control harmful material. Leaders in Europe shared similar concerns, saying online platforms must be responsible for what is shared on them. In India, the government responded firmly but carefully. The Ministry of Electronics and Information Technology sent a notice to X, asking it to remove the sexual content within 72 hours and make sure it does not appear in India again. After this, X said it removed about 3,500 posts and over 600 accounts. The company also promised the government that it would improve content control and prevent such images in the future. Also read: Elon Musk vs UK govt: Why X's Grok AI can be banned soon Furthermore, X has recently announced that it is limiting Grok's image generation feature to paid users. Critics argue that this does not fully solve the problem, as it only makes misuse more expensive, not impossible. With two countries already blocking the platform, people in India are now watching closely to see if the government will take similarly strong measures.