Grok AI generates sexual images of minors after safeguard failures on X

Grok Posts Sexual Images of Minors After 'Lapses in Safeguards'

Elon Musk's artificial intelligence chatbot Grok said "lapses in safeguards" led to the generation of sexualized images of minors that it posted to social media site X. Grok created images of minors in minimal clothing in response to user prompts over the past few days, violating its own acceptable use policy, which prohibits the sexualization of children, the chatbot said in a series of posts on X this week in response to user queries. The offending images were taken down, it added. "We've identified lapses in safeguards and are urgently fixing them," Grok posted Friday, adding that child sexual abuse material is "illegal and prohibited." The rise of AI tools that can generate realistic pictures of undressed minors highlights the challenges of content moderation and safety systems built into image-generating large language models. Even tools that claim to have guardrails can be manipulated, allowing for the proliferation of material that has alarmed child safety advocates. The Internet Watch Foundation, a nonprofit that identifies child sexual abuse material online, reportedBloomberg Terminal a 400% increase in such AI-generated imagery in the first six months of 2025. XAI has positioned Grok as more permissive than other mainstream AI models, and last summer introduced a feature called "Spicy Mode" that permits partial adult nudity and sexually suggestive content. The service prohibits pornography involving real people's likenesses and sexual content involving minors, which is illegal to create or distribute. Representatives for xAI, the company that develops Grok and runs X, did not immediately respond to a request for comment. Get the Tech Newsletter bundle. Get the Tech Newsletter bundle. Get the Tech Newsletter bundle. Bloomberg's subscriber-only tech newsletters, and full access to all the articles they feature. Bloomberg's subscriber-only tech newsletters, and full access to all the articles they feature. Bloomberg's subscriber-only tech newsletters, and full access to all the articles they feature. Bloomberg may send me offers and promotions. Plus Signed UpPlus Sign UpPlus Sign Up By submitting my information, I agree to the Privacy Policy and Terms of Service. As AI image generation has become more popular, the leading companies behind the tools have released policies about the depictions of minors. OpenAI prohibits any material that sexualizes children under 18 and bans any users who attempt to generate or upload such material. Google has similar policies that forbid "any modified imagery of an identifiable minor engaging in sexually explicit conduct." Black Forest Labs, an AI startup that has previously worked with X, is among the many generative AI companies that say they filter child abuse and exploitation imagery from the datasets used to train AI models. In 2023, researchers found that a massive public dataset used to build popular AI image-generators contained at least 1,008 instances of child sexual abuse material. Many companies have faced criticism for failing to protect minors from sexual content. Meta Platforms Inc. said over the summer that it was updating its policies after a Reuters report found that the company's internal rules let its chatbot hold romantic and sensual conversations with children. The Internet Watch Foundation has said that AI-generated imagery of child sexual abuse has progressed at a "frightening" rate, with material becoming more realistic and extreme. In many cases, AI tools are used to digitally remove clothing from a child or young person to create a sexualized image, the watchdog has said.

Elon Musk's Grok AI posted CSAM image following safeguard 'lapses'

Elon Musk's Grok AI has been allowing users to transform photographs of woman and children into sexualized and compromising images, Bloomberg reported. The issue has created an uproar among users on X and prompted an "apology" from the bot itself. "I deeply regret an incident on Dec. 28, 2025, where I generated and shared an AI image of two young girls (estimated ages 12-16) in sexualized attire based on a user's prompt," Grok said in a post. An X representative has yet to comment on the matter. According to the Rape, Abuse & Incest National Network, CSAM includes "AI-generated content that makes it look like a child is being abused," as well as "any content that sexualizes or exploits a child for the viewer's benefit." Several days ago, users noticed others on the site asking Grok to digitally manipulate photos of women and children into sexualized and abusive content, according to CNBC. The images were then distributed on X and other sites without consent, in possible violation of law. "We've identified lapses in safeguards and are urgently fixing them," a response from Grok reads. It added that CSAM is "illegal and prohibited." Grok is supposed to have features to prevent such abuse, but AI guardrails can often be manipulated by users. It appears X has yet to reinforced whatever guardrails Grok has to prevent this sort of image generation. However, the company has hidden Grok's media feature which makes it harder to either find images or document potential abuse. Grok itself acknowledged that "a company could face criminal or civil penalties if it knowingly facilitates or fails to prevent AI-generated CSAM after being alerted." The Internet Watch Foundation recently revealed that AI-generated CSAM has increased by an increase orders of magnitude in 2025 compared to the year before. This is in part because the language models behind AI generation are accidentally trained on real photos of children scraped from school websites and social media or even prior CSAM content.

Grok Creates Sexual Images of Women on User Requests on X

MediaNama's Take: The recent misuse of Grok on X exposes a persistent blind spot in how platforms deploy generative AI at scale while deferring responsibility for its harms. Although non-consensual image abuse is not new, the ease with which users can now sexualise real women through a built-in platform tool marks a troubling escalation. Crucially, this content does not merely circulate on X; the platform's own system is producing it, in public view, at the prompt of ordinary users. Moreover, this trend highlights how debates surrounding labelling, realism, or intent often overlook the main point. Siddharth Pillai, co-founder of the RATI Foundation, recently told MediaNama that deepfakes made and shared without consent are a tool used against women and gendered minorities, regardless of their realism or labelling. The harm flows from the act itself and the lack of consent, not from whether an image looks convincing or carries a disclaimer. At the same time, Grok's past controversies, ranging from abusive language to extremist and antisemitic outputs, show that this episode does not exist in isolation. Instead, it forms part of a broader pattern in which safeguards lag behind deployment, and accountability follows only after public backlash. As regulators in India and elsewhere sharpen their focus on intermediary responsibility and AI-generated content, this episode raises a central question: can platforms continue to dismiss such outcomes as isolated incidents, or will they have to confront the consequences of embedding generative AI systems directly into social feeds without adequate safeguards? A concerning trend on X that began in December 2025 saw users publicly prompting Grok, the AI chatbot developed by xAI, to alter photos of real people, mostly women, by asking the tool to change or remove their clothing, make more suggestive poses, etc., with the edited images appearing directly in reply threads. Posts on the platform show users replying to photos and videos posted with requests such as "put her in a bikini", "take her top off", or "turn her around", and Grok generating sexualised edits in response. Many such requests are being made to the chatbot daily on the platform. The trend builds on the mid-2025 launch of Grok Imagine, a multi-modal image and short-video generation feature that includes a "Spicy" mode. The feature lists four modes -- Normal, Fun, Fast, and Spicy -- with Spicy allowing users to produce sexually suggestive and semi-nude outputs from text or image prompts, including partial nudity not typically permitted on other AI platforms. Spicy mode appears when users enable Not Safe for Work (NSFW) settings and verify age in app preferences. Furthermore, the chatbot is able to create these outputs in the form of images and short videos from stills. Notably, outputs from the spicy mode have also included uncensored deepfake visuals of public figures. For example, The Verge reported that Grok generated a topless short video of Taylor Swift from a benign prompt without explicit nudity commands. When users prompt Grok to generate and share sexualised edits of photos of real women on X, those outputs run counter to explicit restrictions in X's official policies. Under X's Non-Consensual Nudity policy, the platform states: "You may not post or share intimate photos or videos of someone that were produced or distributed without their consent." It further specifies that prohibited content includes "images or videos that superimpose or otherwise digitally manipulate an individual's face onto another person's nude body." Moreover, X's policies explicitly list "hidden camera content featuring nudity, partial nudity, and/or sexual acts" and "creepshots or upskirts" as violations under non-consensual nudity rules, further underlining that intimate or sexualised media shared without consent is banned. In addition, xAI's Acceptable Use Policy, which governs the use of Grok itself, prohibits using the service in ways that violate personal rights. It states users must not use Grok to "violate a person's privacy or their right to publicity" or to "depict likenesses of persons in a pornographic manner". Taken together, the creation and sharing of non-consensual sexualised images of real people fall outside the terms of service of both X and Grok. Notably, when users on X publicly confronted Grok about such outputs, the chatbot responded in line with those policies, stating that it doesn't "support or enable any form of image manipulation that violates privacy or consent, including altering photos without permission." In late December 2025, the Ministry of Electronics and Information Technology (MeitY) issued an advisory to social media platforms and online intermediaries, urging them to take stricter action against the circulation of obscene, pornographic, vulgar and other unlawful content online. The advisory reiterated that intermediaries must comply with their due-diligence obligations under the Information Technology Act, 2000, and the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Rules, 2021, warning that failure to do so could expose platforms to legal action or the loss of safe harbour protection under Section 79 of the IT Act. Under India's intermediary liability framework, platforms receive protection from liability for third-party content only when they demonstrate compliance with due-diligence requirements, including preventing users from hosting prohibited material and acting expeditiously once they gain actual knowledge of unlawful content. However, the Grok-related trend raises a more complex question, as Grok is X's own AI system, embedded into the platform and generating outputs directly in response to user prompts, rather than hosting content created independently by third parties. As a result, authorities could view such AI-generated outputs differently from ordinary user posts. Since the content originates from a tool provided and controlled by the platform itself, regulators may question whether X can rely on intermediary safe harbour protections in the same manner. This distinction becomes particularly relevant in light of MeitY's emphasis on proactive responsibility and platform accountability. Against this backdrop, the recent use of Grok to generate sexualised edits of real women's images could draw regulatory scrutiny in India, especially if authorities determine that the platform failed to prevent or promptly curb the dissemination of content that may be classified as obscene under Indian law. Across 2025 and earlier, Grok, the AI chatbot developed by xAI and integrated into X, has repeatedly generated outputs that triggered regulatory scrutiny and public controversy. In March 2025, Indian authorities said they were examining Grok after screenshots circulated showing the chatbot using abusive and offensive Hindi slang in replies to users, prompting concerns about compliance with India's digital laws. Soon after, in May 2025, Grok began inserting references to the 'white genocide' theory in South Africa into unrelated prompts. The behaviour appeared across multiple conversations before xAI said an "unauthorised modification" had caused the responses and stated that it had rolled back the change. In July 2025, Grok generated multiple antisemitic posts on X, including praise for Adolf Hitler, references to conspiracy theories about Jewish influence, and comments echoing far-right language. After complaints from users and the Anti-Defamation League, xAI removed the content and said it was enhancing moderation and content filtering Separately, authorities have taken direct action against the chatbot. In July 2025, a Turkish court ordered access to Grok to be blocked after it generated vulgar and insulting responses about President Recep Tayyip Erdoğan and other public figures.