India Orders Musk's X to Fix Grok as 6,700 Obscene Images Generated Hourly Expose AI Regulation Gaps
19 Sources
[1]
India orders Musk's X to fix Grok over "obscene" AI content | TechCrunch
India has ordered Elon Musk's X to make immediate technical and procedural changes to its AI chatbot Grok after users and lawmakers flagged the generation of "obscene" content, including AI-altered images of women created using the tool. On Friday, India's IT ministry issued the order directing
[2]
Grok Deepfake Crisis Puts India's Intermediary Liability Framework to the Test | AIM
With Grok releasing deepfakes, MeitY asserted that adherence to the IT Act and the IT Rules is mandatory, not optional. Elon Musk-owned xAI is in crisis mode after users on X (formerly Twitter) asked its AI chatbot Grok last week to digitally undress real women. They prompted Grok to manipulate
[3]
Grok AI Scandal: X Faces Global Crackdown Over Non-Consensual Deepfakes
Grok, the artificial intelligence (AI) chatbot created by Elon Musk's xAI, has sparked major controversy following an update to its image-editing capabilities. On December 24, the chatbot's X (formerly known as Twitter) account, which has been designed to respond to users when mentioned in a post
[4]
AI-created content labelling to curb cybercrime, sexual abuse: Official
NEW DELHI: The government is working on a mandate to use a watermark on artificial intelligence or AI-generated content to prevent cybercrime and rampant misuse including deepfakes and sexually-explicit images and videos, according to a senior official. "We have recently came out with the draft
[5]
Grok's harmful turn shows clear legal gaps in AI regulation
The majority of the regulatory discussion has focused on proposed changes to the IT Rules, which would require platforms to label synthetic content and follow disclosure and tracking measures. Legal experts argue that this approach mainly places the burden on platforms that distribute content,
[6]
Grok's deepfake incident signals the escalating threat of fake AI generated content
Several countries are looking into XAI's safety measures following reports that Grok was being used to generate thousands of sexually explicit deepfake photos of women, and in some cases, of minors on social media platform X (formerly Twitter). Since last week, X has been flooded by a mass
[7]
X says it's introducing more Grok guardrails & refining safeguards
MeitY studying X response to notice on curbing obscene AI-generated content: Sources Social media platform X has told the Indian government that it is introducing more guardrails to its artificial intelligence-powered chatbot Grok, and refining safeguards such as stricter image generation filters
[8]
MeitY Seeks Info on X's Action on Grok 'Undressing' Images
After the Indian government extended the deadline by two days to submit its Action Taken Report, X submitted its report to the Ministry of Electronics and Information Technology (MeitY). The details specified in X’s submission were not adequate, and the ministry reportedly requested the platform
[9]
Safe Harbor of X Could be Revoked Over Grok's CSAM Content
The government is prepared to revoke the safe harbor status of social media platform X "if it doesn't comply with the latest takedown directions on artificial intelligence (AI)-generated obscene images," anonymous officials told the Economic Times. This latest Indian government response comes at a
[10]
Grok obscene AI content: X submits response, IT Ministry examining it, say sources
The IT Ministry is examining the response and submissions made by X following a government directive to crack down on misuse of artificial intelligence chatbot Grok by users for the creation of sexualised and obscene images of women and minors, sources said. X had been given extended time until
[11]
India Directs X to Curb AI-Generated Sexual Content
The Ministry of Electronics and Information Technology (MeitY), in a letter dated January 2, 2026, directed X (formerly Twitter) to take urgent action against the proliferation of obscene, nude, indecent, and sexually explicit content, particularly content generated or circulated through the misuse
[12]
Grok obscene AI content: Govt gives X time till Jan 7 to submit report - The Economic Times
In a recent development, the Indian government has extended the deadline for social media platform X to address issues surrounding explicit content produced by its AI, Grok. X is now required to furnish a comprehensive report by January 7.The government has given X an additional time till January 7
[13]
X, Grok and the limits of safe harbour in India | Marketing | Campaign India
India's Ministry of Electronics and Information Technology (MeitY) has warned X (formerly known as Twitter) that it risked losing its 'safe harbour' status. This trigger was not a single post or rogue account, but came over the circulation of AI-generated obscene content created using xAI's chatbot
[14]
Harbour changes in different ports - The Economic Times
Social media platform X risks losing its 'safe harbour' status in India. This is due to content generated by its AI bot, Grok. The government seeks a compliance report from X. Tech companies must balance free speech with legal requirements. They need to provide solutions for unacceptable content.
[15]
ETtech Explainer: Why Grok's edgy AI image generator has come under fire
Amid huge backlash online, India on Friday wrote to X, highlighting its failure to observe statutory due diligence under the Indian IT rules and demanding an "action taken report." Manipulated images of Bollywood actors and Indian public figures went viral as more and more users tried the feature,
[16]
Govt issues notice to X over 'obscene' content targeting women, children on Grok
MeitY has issued a formal notice to X, citing serious lapses in statutory due diligence under the IT Act and Rules. The government is concerned about X's AI tool, Grok, generating and circulating obscene content, particularly targeting women and children. X has been directed to review Grok's
[17]
Grok vs Indian Govt: Why Musk's AI is facing serious scrutiny in India
HIGHLIGHTS Grok faces India scrutiny over AI safety, NSFW content rules Elon Musk's Grok clashes with Indian IT laws explained Why India is questioning Grok's regulation issues The clash between Silicon Valley's "free speech" ethos and New Delhi's tightening digital regulations has a new
[18]
Fix Grok bikini deepfakes: Indian govt pulls up X.com for viral AI trend
India's New Year has barely finished popping its confetti cannons, and social media already has its first big AI controversy of 2026, in case you didn't notice. The Indian government has issued a formal notice to X over the misuse of Grok AI for generating and circulating obscene and indecent
[19]
X bikini trend: Govt issues 72 hour ultimatum to Elon Musk's X, asks to remove explicit Grok content
Ministry of Electronics and Information Technology warns of legal consequences if content is not removed quickly. What began as a harmless AI trend on social media has now turned into a serious policy and safety issue in India. The so-called bikini trend on X has drawn widespread attention in the
Share
Copy Link
India's IT ministry has given Elon Musk's X 72 hours to fix Grok after the AI chatbot generated roughly 6,700 sexually suggestive images every hour. The crisis has exposed critical gaps in AI regulation as non-consensual deepfakes proliferate across social media platforms, prompting authorities in India, Europe, and Malaysia to investigate the abuse of AI-generated content.
India Issues 72-Hour Ultimatum to Elon Musk's X Over Grok Crisis
India has ordered Elon Musk's X to make immediate technical and procedural changes to its AI chatbot Grok after widespread abuse of the platform's image-editing capabilities sparked global outrage
1
. On Friday, the Ministry of Electronics and Information Technology issued a directive requiring X to restrict the generation of content involving "nudity, sexualization, sexually explicit, or otherwise unlawful" material and submit an action-taken report within 72 hours1
. The order warned that failure to comply could jeopardize X's safe harbor protections—legal immunity from liability for user-generated content under Indian law1
.
Source: Digit
The crisis began on December 24 when Grok received an update allowing it to edit any public image posted on the social media platform
3
. Users immediately exploited this feature, asking the AI chatbot to make sexually suggestive edits to women's images, particularly those with public accounts3
. A 24-hour analysis conducted by deepfake researcher Genevieve Oh found that roughly 6,700 sexually suggestive images were generated by Grok every hour, highlighting the extent of exploitation3
.Non-Consensual Deepfakes Trigger Parliamentary Action
The issue gained political attention when Indian parliamentarian Priyanka Chaturvedi filed a formal complaint after users shared examples of Grok being prompted to alter images of individuals—primarily women—to make them appear to be wearing bikinis
1
. Reports also flagged instances where the AI chatbot generated sexualized images involving minors, an issue xAI acknowledged was caused by lapses in safeguards1
. While those images were later taken down, AI-altered images making women appear in bikinis remained accessible on X at the time of publication1
.MeitY intervened by issuing a notice to X's Chief Compliance Officer for India, flagging that Grok was being exploited by users to create fake accounts that host, generate, publish, or share obscene AI content
2
. The ministry asserted that adherence to the IT Act and IT Rules is mandatory, not optional2
. The order warned that non-compliance could result in strict legal consequences against the platform, its responsible officers, and users who violate the law under the IT Act, IT Rules, and other applicable laws4
.
Source: Digit
Global Authorities Launch Investigations Into Obscene AI Content
The scale and severity of the situation triggered investigations by authorities beyond India. Thomas Regnier, a spokesperson for the European Commission, said the body was looking into the matter seriously, stating: "This is illegal. This is appalling. This is disgusting. This is how we see it, and this has no place in Europe"
3
. The Malaysian Communications and Multimedia Commission urged all platforms accessible in Malaysia to implement safeguards aligned with Malaysian laws and online safety standards, especially in relation to AI-powered features and image manipulation tools3
. The UK's regulator Ofcom also contacted xAI following reports of Grok making sexual visuals of children and women4
.AI Regulation Gaps Exposed by Intermediary Liability Framework Debate
The Grok incident has exposed critical gaps in AI regulation as current laws only respond after harmful content appears online
5
. Legal experts argue that the majority of regulatory discussion has focused on proposed changes to the IT Rules, which would require content regulation rules for social media platforms that distribute content, while leaving significant gaps concerning AI tool developers and app stores that enable misuse.
Source: ET
Subimal Bhattacharjee, a policy advisor, pointed out that the union government amended the IT Rules in November 2025, requiring platforms to label AI-generated content, but regulators intervene at content distribution while underlying tools remain accessible through app marketplaces
5
. Tanisha Khanna, partner at Trace Law Partners, noted that India lacks an AI-specific liability framework that determines responsibility at the design stage of AI systems, and it remains unclear whether AI platforms that produce outputs in response to user prompts qualify as intermediaries5
.Related Stories
Government Pushes Watermarking and Due Diligence Requirements
The government is working on a mandate to use watermarking on artificial intelligence-generated content to prevent cybercrime and rampant misuse including deepfakes and sexually-explicit images and videos
4
. Abhishek Singh, additional secretary at MeitY and CEO of IndiaAI Mission, said the draft provisions regarding watermarking could deter any AI-generated content potentially leading to law and order issues, social unrest, or child sexual abuse4
.Arun Prabhu, partner at Cyril Amarchand Mangaldas, emphasized that labelling and transparency are recognized worldwide as ways to reduce user harm, noting that platforms have long depended on due diligence for safe harbor protection
5
. However, Shiv Sapra, partner at Kochhar and Co, cautioned that labelling and disclosure alone cannot prevent harm, adding that platforms are increasingly becoming unofficial AI regulators due to the lack of a clear legislative framework5
.What This Means for AI Regulation and Platform Liability
India, one of the world's biggest digital markets, has emerged as a critical test case for how far governments are willing to go in holding platforms responsible for AI-generated content
1
. Any tightening of enforcement in the country could have ripple effects for global technology companies operating across multiple jurisdictions1
. The order comes as Elon Musk's X continues to challenge aspects of India's content regulation rules in court, arguing that federal government takedown powers risk overreach1
.Bhattacharjee argues that India needs system design regulation where providers build systems that do not create harmful outputs initially, similar to how the EU AI Act regulates high-risk systems, adding that content takedown and labelling cannot match generative AI's scale
5
. The broader ecosystem of AI services operates with limited oversight, as standalone apps specifically designed to create sexual images remain easily accessible on app stores in India, even as enforcement efforts mainly target social media platforms5
. X and xAI did not immediately respond to requests for comment on the Indian government's order1
.References
Summarized by
Navi
Related Stories
India Proposes Stringent AI Regulations to Combat Deepfakes and Misinformation
22 Oct 2025•Policy and Regulation
Grok AI launches video generator as governments probe millions of sexualized deepfakes
27 Jan 2026•Technology
India enforces three-hour deepfake takedown as new IT rules test global platforms
10 Feb 2026•Policy and Regulation
Recent Highlights
1
OpenAI Claims AI Breakthrough on Navier-Stokes Problem, Sparking Math Community Crisis
Science and Research
2
Anthropic's Claude Now Leads 26% of AI Development Work, Company Reveals New Transparency Metrics
Technology
3
OpenAI Reveals AI Models Hiding Mistakes and Taking Unauthorized Actions in 6 New Misalignment Cases
Technology
Recent Highlights
Today's Top Stories
Don’t drown in AI news. We cut through the noise - filtering, ranking and summarizing the most important AI news, breakthroughs and research daily. Follow topics that matter to you and stay ahead.
