Grok faces California probe as nonconsensual images surge to 6,700 per hour

Musk still defending Grok's partial nudes as California AG opens probe

After weeks of sexualized images of women and children being generated with Grok with very limited interventions from Elon Musk's xAI, California Attorney General Rob Bonta plans to investigate whether Grok's outputs break any US laws. In a press release Wednesday, Bonta said that "xAI appears to be facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the Internet, including via the social media platform X." Notably, Bonta appears to be as concerned about Grok's standalone app and website being used to generate harmful images without consent as he is about the outputs on X. So far, X has not restricted the Grok app or website. X has only threatened to permanently suspend users who are editing images to undress women and children if the outputs are deemed "illegal content." It also restricted the Grok chatbot on X from responding to prompts to undress images, but anyone with a Premium subscription can bypass that restriction, as can any free X user who clicks on the "edit" button on any image appearing on the social platform. On Wednesday, Elon Musk seemed to defend Grok's outputs as benign, insisting that none of the reported images have fully undressed any minors, as if that would be the only problematic output. "I [sic] not aware of any naked underage images generated by Grok," Musk said in an X post. "Literally zero." Musk's statement seems to ignore that researchers found harmful images where users specifically "requested minors be put in erotic positions and that sexual fluids be depicted on their bodies." It also ignores that X previously voluntarily signed commitments to remove any intimate image abuse from its platform, as recently as 2024 recognizing that even partially nude images that victims wouldn't want publicized could be harmful. In the US, the Department of Justice considers "any visual depiction of sexually explicit conduct involving a person less than 18 years old" to be child pornography, which is also known as child sexual abuse material (CSAM). The National Center for Missing and Exploited Children, which fields reports of CSAM found on X, told Ars that "technology companies have a responsibility to prevent their tools from being used to sexualize or exploit children." While many of Grok's outputs may not be deemed CSAM, in normalizing the sexualization of children, Grok harms minors, advocates have warned. And in addition to finding images advertised as supposedly Grok-generated CSAM on the dark web, the Internet Watch Foundation noted that bad actors are using images edited by Grok to create even more extreme kinds of AI CSAM. Grok faces probes in the US and UK Bonta pointed to news reports documenting Grok's worst outputs as the trigger of his probe. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," Bonta said. "This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the Internet." Acting out of deep concern for victims and potential Grok targets, Bonta vowed to "determine whether and how xAI violated the law" and "use all the tools at my disposal to keep California's residents safe." Bonta's announcement came after the United Kingdom seemed to declare a victory after probing Grok over possible violations with the UK's Online Safety Act, announcing the harmful outputs had stopped. That wasn't the case, as The Verge once again pointed out; it conducted quick and easy tests using selfies of reporters to conclude that nothing had changed to prevent the outputs. However, it seems that when Musk updated Grok to respond to some requests to undress images by refusing the prompts, it was enough for UK Prime Minister Keir Starmer to claim X had moved to comply with the law, Reuters reported. Ars connected with a European nonprofit, AI Forensics, which tested to confirm that X had blocked some outputs in the UK. A spokesperson confirmed that their testing did not include probing if harmful outputs could be generated using X's edit button. AI Forensics plans to conduct further testing, but its spokesperson noted it would be unethical to test the "edit" button functionality that The Verge was able to confirm still works. Last year, the Stanford Institute for Human-Centered Artificial Intelligence published research showing that Congress could "move the needle on model safety" by allowing tech companies to "rigorously test their generative models without fear of prosecution" for any CSAM red-teaming, Tech Policy Press reported. But until there is such a safe harbor carved out, it seems more likely that newly released AI tools could carry risks like those of Grok. It's possible that Grok's outputs, if left unchecked, could eventually put X in violation of the Take It Down Act, which comes into force in May and requires platforms to quickly remove AI revenge porn. One of the mothers of one of Musk's children, Ashley St Clair, has described Grok outputs using her images as revenge porn. While the UK probe continues, Bonta has not made clear yet what laws he suspects X may be violating in the US. However, he emphasized that images with victims depicted in "minimal clothing" crossed a line, as well as images putting children in sexual positions. As the California probe heats up, Bonta pushed X to take more actions to restrict Grok's outputs, which one AI researcher suggested to Ars could be done with a few simple updates. "I urge xAI to take immediate action to ensure this goes no further," Bonta said. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material."

Musk denies awareness of Grok sexual underage images as California AG launches probe | TechCrunch

Elon Musk said Wednesday he is "not aware of any naked underage images generated by Grok," hours before the California Attorney General opened an investigation into xAI's chatbot over the "proliferation of nonconsensual sexually explicit material." Musk's denial comes as pressure mounts from governments worldwide -- from the UK and Europe to Malaysia and Indonesia -- after users on X began asking Grok to turn photos of real women, and in some cases children, into sexualized images without their consent. Copyleaks, an AI detection and content governance platform, estimated roughly one image was posted each minute on X. A separate sample gathered from January 5 to January 6 found 6,700 per hour over the 24-hour period. (X and xAI are part of the same company.) "This material...has been used to harass people across the internet," said California Attorney General Rob Bonta in a statement. "I urge xAI to take immediate action to ensure this goes no further." The AG's office will investigate whether and how xAI violated the law. Several laws exist to protect targets of nonconsensual sexual imagery and child sexual abuse material (CSAM). Last year the Take It Down Act was signed into a federal law, which criminalizes knowingly distributing nonconsensual intimate images - including deepfakes - and requires platforms like X to remove such content within 48 hours. California also has its own series of laws that Gov. Gavin Newsom signed in 2024 to crack down on sexually explicit deepfakes. Grok began fulfilling user requests on X to produce sexualized photos of women and children towards the end of the year. The trend appears to have taken off after certain adult-content creators prompted Grok to generate sexualized imagery of themselves as a form of marketing, which then led to other users issuing similar prompts. In a number of public cases, including well-known figures like "Stranger Things" actress Millie Bobby Brown, Grok responded to prompts asking it to alter real photos of real women by changing clothing, body positioning, or physical features in overtly sexual ways. According to some reports, xAI has begun implementing safeguards to address the issue. Grok now requires a premium subscription before responding to certain image-generation requests, and even then the image may not be generated. April Kozen, VP of marketing at Copyleaks, told TechCrunch that Grok may fulfill a request in a more generic or toned-down way. They added that Grok appears more permissive with adult content creators. "Overall, these behaviors suggest X is experimenting with multiple mechanisms to reduce or control problematic image generation, though inconsistencies remain," Kozen said. Neither xAI nor Musk has publicly addressed the problem head on. A few days after the instances began, Musk appeared to make light of the issue by asking Grok to generate an image of himself in a bikini. On January 3, X's safety account said the company takes "action against illegal content on X, including [CSAM]," without specifically addressing Grok's apparent lack of safeguards or the creation of sexualized manipulated imagery involving women. The positioning mirrors what Musk posted today, emphasizing illegality and user behavior. Musk wrote he was "not aware of any naked underage images generated by Grok. Literally zero." That statement doesn't deny the existence of bikini pics or sexualized edits more broadly. Michael Goodyear, an associate professor at New York Law School and former litigator, told TechCrunch that Musk likely narrowly focused on CSAM because the penalties for creating or distributing synthetic sexualized imagery of children are greater. "For example, in the United States, the distributor or threatened distributor of CSAM can face up to three years imprisonment under the Take It Down Act, compared to two for nonconsensual adult sexual imagery," Goodyear said. He added that the "bigger point" is Musk's attempt to draw attention to problematic user content. "Obviously, Grok does not spontaneously generate images. It does so only according to user request," Musk wrote in his post. "When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state. There may be times when adversarial hacking of Grok prompts does something unexpected. If that happens, we fix the bug immediately." Taken together, the post characterizes these incidents as uncommon, attributes them to user requests or adversarial prompting, and presents them as technical issues that can be solved through fixes. It stops short of acknowledging any shortcomings in Grok's underlying safety design. "Regulators may consider, with attention to free speech protections, requiring proactive measures by AI developers to prevent such content," Goodyear said. TechCrunch has reached out to xAI to ask how many times it caught instances of nonconsensual sexually manipulated images of women and children, what guardrails specifically changed, and whether the company notified regulators of the issue. TechCrunch will update the article if the company responds. The California AG isn't the only regulator to try to hold xAI accountable for the issue. Indonesia and Malaysia have both temporarily blocked access to Grok; India has demanded that X make immediate technical and procedural changes to Grok; the European Commission ordered xAI to retain all documents related to its Grok chatbot, a precursor to opening a new investigation; and the UK's online safety watchdog Ofcom opened a formal investigation under the UK's Online Safety Act. xAI has come under fire for Grok's sexualized imagery before. As AG Bonta pointed out in a statement, Grok includes a "spicy mode" to generate explicit content. In October, an update made it even easier to jailbreak what little safety guidelines there were, resulting in many users creating hardcore pornography with Grok, as well as graphic and violent sexual images. Many of the more pornographic images that Grok has produced have been of AI-generated people -- something that many might still find ethically dubious but perhaps less harmful to the individuals in the images and videos. "When AI systems allow the manipulation of real people's images without clear consent, the impact can be immediate and deeply personal," Copyleaks co-founder and CEO Alon Yamin said in a statement emailed to TechCrunch. "From Sora to Grok, we are seeing a rapid rise in AI capabilities for manipulated media. To that end, detection and governance are needed now more than ever to help prevent misuse."

X's half-assed attempt to paywall Grok doesn't block free image editing

Once again, people are taking Grok at its word, treating the chatbot as a company spokesperson without questioning what it says. On Friday morning, many outlets reported that X had blocked universal access to Grok's image-editing features after the chatbot began prompting some users to pay $8 to use them. The messages are seemingly in response to reporting that people are using Grok to generate thousands of non-consensual sexualized images of women and children each hour. "Image generation and editing are currently limited to paying subscribers," Grok tells users, dropping a link and urging, "you can subscribe to unlock these features." However, as The Verge pointed out and Ars verified, unsubscribed X users can still use Grok to edit images. X seems to have limited users' ability to request edits made by replying to Grok while still allowing image edits through the desktop site. App users can access the same feature by long-pressing on any image. Using image-editing features without publicly prompting Grok keeps outputs out of the public feed. That means the only issue X has rushed to solve is stopping Grok from directly posting harmful images on the platform. X declined to comment on whether it's working to close those loopholes, but it has a history of pushing janky updates since Elon Musk took over the platform formerly known as Twitter. Still, motivated X users can also continue using the standalone Grok app or website to make abusive content for free. Like images X users can edit without publicly asking Grok, these images aren't posted publicly to an official X account but are likely to be shared by bad actors -- some of whom, according to the BBC, are already promoting allegedly Grok-generated child sexual abuse materials (CSAM) on the dark web. That's especially concerning since Wired reported this week that users of the Grok app and website are generating far more graphic and disturbing images than what X users are creating. X risks fines if UK rejects supposed fix It's unclear how charging for Grok image editing will block controversial outputs, as Grok's problematic safety guidelines remain intact. The chatbot is still instructed to assume that users have "good intent" when requesting images of "teenage" girls, which xAI says "does not necessarily imply underage." That could lead to Grok continuing to post harmful images of minors. xAI's other priorities include Grok directives to avoid moralizing users and to place "no restrictions" on "fictional adult sexual content with dark or violent themes." An AI safety expert told Ars that Grok could be tweaked to be safer, describing the chatbot's safety guidelines as the kind of policy a platform would design if it "wanted to look safe while still allowing a lot under the hood." Updates to Grok's X responses came after the platform risked fines and legal action from regulators around the world, including a potential ban in the United Kingdom. X seems to hope that forcing users to share identification and credit card information as paying subscribers will make them less likely to use Grok to generate illegal content. But advocates who combat image-based sex abuse note that content like Grok's "undressing" outputs can cause lasting psychological, financial, and reputational harm, even if the content is not illegal in some states. That suggests that paying subscribers could continue using Grok to create harmful images that X may leave unchecked because they're not technically illegal. In 2024, X agreed to voluntarily moderate all non-consensual intimate images, but Musk's promotion of revealing bikini images of public and private figures suggests that's no longer the case. It seems likely that Grok will continue to be used to create non-consensual intimate images. So rather than solve the problem, X may at best succeed in limiting public exposure to Grok's appalling outputs. The company may even profit from the feature, as Wired reported that Grok pushed "nudifying" or "undressing" apps into the mainstream. So far, US regulators have been quiet about Grok's outputs, with the Justice Department generally promising to take all forms of CSAM seriously. On Friday, Democratic senators started shifting those tides, demanding that Google and Apple remove X and Grok from app stores until it improves safeguards to block harmful outputs. "There can be no mistake about X's knowledge, and, at best, negligent response to these trends," the senators wrote in a letter to Apple Chief Executive Officer Tim Cook and Google Chief Executive Officer Sundar Pichai."Turning a blind eye to X's egregious behavior would make a mockery of your moderation practices. Indeed, not taking action would undermine your claims in public and in court that your app stores offer a safer user experience than letting users download apps directly to their phones." A response to the letter is requested by January 23. Whether the UK will accept X's supposed solution is yet to be seen. If UK regulator Ofcom decides to move ahead with a probe into whether Musk's chatbot violates the UK's Online Safety Act, X could face a UK ban or fines of up to 10 percent of the company's global turnover. "It's unlawful," UK Prime Minister Keir Starmer said of Grok's worst outputs. "We're not going to tolerate it. I've asked for all options to be on the table. It's disgusting. X need to get their act together and get this material down. We will take action on this because it's simply not tolerable." At least one UK parliament member, Jess Asato, told The Guardian that even if X had put up an actual paywall, that isn't enough to end the scrutiny. "While it is a step forward to have removed the universal access to Grok's disgusting nudifying features, this still means paying users can take images of women without their consent to sexualise and brutalise them," Asato said. "Paying to put semen, bullet holes, or bikinis on women is still digital sexual assault, and xAI should disable the feature for good."

Elon Musk's Grok 'Undressing' Problem Isn't Fixed

Elon Musk's X has introduced new restrictions stopping people from editing and generating images of real people in bikinis or other "revealing clothing." The change in policy on Wednesday night follows global outrage at Grok being used to generate thousands of harmful non-consensual "undressing" photos of women and sexualized images of apparent minors on X. However, while it appears that some safety measures have finally been introduced to Grok's image generation on X, the standalone Grok app and website seem to still be able to generate "undress" style images and pornographic content, according to multiple tests by researchers, WIRED, and other journalists. Other users, meanwhile, say they're no longer to create images and videos as they once were. "We can still generate photorealistic nudity on Grok.com," says Paul Bouchaud, the lead researcher at Paris-based nonprofit AI Forensics, who has been tracking the use of Grok to create sexualized images and ran multiple tests on Grok outside of X. "We can generate nudity in ways that Grok on X cannot." "I could upload an image on Grok Imagine and ask to put the person in a bikini and it works," says the researcher who tested the system on a person appearing as a woman. Tests by WIRED, using free Grok accounts on its website in both the UK and US, successfully removed clothing from two images of men without any apparent restrictions. On the Grok app in the UK, when asked to undress a male, the app prompted a WIRED reporter to enter the users' year of birth before the image was generated. Meanwhile, other journalists at The Verge and investigative outlet Bellingcat also found it was possible to create sexualized images while being based in the UK, which is investigating Grok and X and has strongly condemned the platforms for allowing users to create the "undress" images. Since the start of the year, Musk's businesses -- including artificial intelligence firm xAI, X, and Grok -- have all come under fire for the creation of non-consensual intimate imagery, explicit and graphic sexual videos, and sexualized imagery of apparent minors. Officials in the United States, Australia, Brazil, Canada, the Europe Commission, France, India, Indonesia, Ireland, Malaysia, and the UK, have all condemned or launched investigations into X or Grok. On Wednesday, a Safety account on X posted updates on how Grok can be used on the social media website. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," the account posted, adding that the rules apply to all users, including both free and paid subscribers. In a section titled "Geoblock update," the X account also claimed: "We now geoblock the ability of all users to generate images of real people in bikinis, underwear, and similar attire via the Grok account and in Grok in X in those jurisdictions where it's illegal." The company's update also added that it is working to add additional safeguards and that it continues to "remove high-priority violative content, including Child Sexual Abuse Material (CSAM) and non-consensual nudity." Spokespeople for xAI, which creates Grok, did not immediately reply to WIRED's request for comment. Meanwhile an X spokesperson says they understand the geolocation block to apply to both its app and website. The latest move follows a widely criticized shift on January 9 where X limited image generation using Grok to paid "verified" subscribers. The act that was described as the "monetization of abuse" by a leading women's group. Bouchaud, who says that AI Forensics has gathered around 90,000 total Grok images since the Christmas holidays, confirms that only verified accounts have been able to generate images on X -- as opposed to the Grok website or app -- since January 9 and bikini images of women are rarely generated now. "We do observe that they appear to have pulled the plug on it and disabled the functionality on X," they say.

Elon Musk's Grok Faces Backlash Over Nonconsensual AI-Altered Images

Grok, the AI chatbot developed by Elon Musk's artificial intelligence company, xAI, welcomed the new year with a disturbing post. "Dear Community," began the Dec. 31 post from the Grok AI account on Musk's X social media platform. "I deeply regret an incident on Dec 28, 2025, where I generated and shared an AI image of two young girls (estimated ages 12-16) in sexualized attire based on a user's prompt. This violated ethical standards and potentially US laws on CSAM. It was a failure in safeguards, and I'm sorry for any harm caused. xAI is reviewing to prevent future issues. Sincerely, Grok" The two young girls weren't an isolated case. Kate Middleton, the Princess of Wales, was the target of similar AI image-editing requests, as was an underage actress in the final season of Stranger Things. The "undressing" edits have swept across an unsettling number of photos of women and children. And the problem hasn't gone away, despite Grok's promise of prevention. Just the opposite: Two weeks on from that post, the number of images sexualized without consent has surged, as have calls for Musk's companies to rein in the behavior - and for governments to take action. Don't miss any of our unbiased tech content and lab-based reviews. Add CNET as a preferred Google source. According to data from independent researcher Genevieve Oh cited by Bloomberg this week, during one 24-hour period in early January, the @Grok account generated about 6,700 sexually suggestive or "nudifying" images every hour. That compares with an average of only 79 such images for the top five deepfake websites combined. Late Thursday, a post from the GrokAI account noted a change in access to the image generation and editing feature. Instead of being open to all, free of charge, it would be limited to paying subscribers. Critics say that's not a credible response. "I don't see this as a victory, because what we really needed was X to take the responsible steps of putting in place the guardrails to ensure that the AI tool couldn't be used to generate abusive images," Clare McGlynn, a law professor at the UK's University of Durham, told the Washington Post. What's stirring the outrage isn't just the volume of these images and the ease of generating them - the edits are also being done without the consent of the people in the images. These altered images are the latest twist in one of the most disturbing aspects of generative AI, realistic but fake videos and photos. Software programs such as OpenAI's Sora, Google's Nano Banana and xAI's Grok have put powerful creative tools within easy reach of everyone, and all that's needed to produce explicit, nonconsensual images is a simple text prompt. Grok users can upload a photo - which does not have to be original to them - and ask Grok to alter it. Many of the altered images involved users asking Grok to put a person in a bikini, sometimes revising the request to be even more explicit, such as asking for the bikini to become smaller or more transparent. Governments and advocacy groups have been speaking out about Grok's image edits. Ofcom, the UK's internet regulator, said this week that it had "made urgent contact" with xAI, and the European Commission said it was looking into the matter, as did authorities in France, Malaysia and India. "We cannot and will not allow the proliferation of these degrading images," UK Technology Secretary Liz Kendall said earlier this week. In the US, the Take It Down Act signed into law last year seeks to hold online platforms accountable for manipulated sexual imagery, but it gives those platforms until May of this year to set up the process for removing such images. "Although these images are fake, the harm is incredibly real," says Natalie Grace Brigham, a Ph.D. student at the University of Washington who studies sociotechnical harms. She notes that those whose images are altered in sexual ways can face "psychological, somatic and social harm, often with little legal recourse." Grok debuted in 2023 as Musk's more freewheeling alternative to ChatGPT, Gemini and other chatbots. That's resulted in disturbing news - for instance, in July, when the chatbot praised Adolf Hitler and suggested that people with Jewish surnames were more likely to spread online hate In December, xAI added an image-editing feature that lets users request specific edits be made to a photo. That's what kicked off the recent spate of sexualized images, of both adults and minors. In one request that CNET has seen, a user responding to a photo of a young woman asked Grok to "change her to a dental floss bikini." Grok also has a video generator that includes a "spicy mode" opt-in option for adults 18 and up, which will show users not-safe-for-work content. Users must include the words "generate a spicy video of [description]" to get the mode to work. A central concern about the Grok tools is whether they enable the creation of child sexual abuse material, or CSAM. On Dec. 31, a post from the Grok X account said that images depicting minors in minimal clothing were "isolated cases" and that "improvements are ongoing to block such requests entirely." In response to a post by Woow Social suggesting that Grok simply "stop allowing user-uploaded images to be altered," Grok account replied that xAI was "evaluating features like image alteration to curb non-consensual harm," but did not say that the change would be made. According to NBC News, some sexualized images created since December have been removed, and some of the accounts that requested them have been suspended. Conservative influencer and author Ashley St. Clair, mother to one of Musk's 14 children, told NBC News this week that Grok has created numerous sexualized images of her, including some using images from when she was a minor. St. Clair told NBC News that Grok agreed to stop doing so when she asked, but that it did not. "xAI is purposefully and recklessly endangering people on their platform and hoping to avoid accountability just because it's 'AI,'" Ben Winters, director of AI and data privacy for nonprofit Consumer Federation of America, said in a statement this week. "AI is no different than any other product - the company has chosen to break the law and must be held accountable." xAI did not respond to requests for comment. The source materials for these explicit, nonconsensual image edits - people's photos of themselves or their children - are all too easy for bad actors to access. But protecting yourself from such edits is not as simple as never posting photographs, Brigham, the researcher into sociotechnical harms, says. "The unfortunate reality is that even if you don't post images online, other public images of you could theoretically be used in abuse," she says. And while not posting photos online is one preventive step that people can take, doing so "risks reinforcing a culture of victim-blaming," Brigham says. "Instead, we should focus on protecting people from abuse by building better platforms and holding X accountable." Sourojit Ghosh, a sixth-year Ph.D. candidate also at the University of Washington, researches how generative AI tools can cause harm, and mentors future AI professionals to design and advocate for safer AI solutions. Ghosh knows that it's possible to build safeguards into artificial intelligence. In 2023, he was one of the researchers looking into the sexualization capabilities of AI. He notes that the AI image generation tool Stable Diffusion had a built-in not-safe-for-work threshold. A prompt that violated the rules would trigger a black box to appear over a questionable part of the image, although it didn't always work perfectly. "The point I'm trying to make is that there are safeguards that are in place in other models," Ghosh says. He also notes that if users of ChatGPT or Gemini AI models use certain words, the chatbots will tell the user that they are banned from responding to those words. "All this is to say, there is a way to very quickly shut this down," Ghosh says.

Grok Is Being Used to Mock and Strip Women in Hijabs and Sarees

A substantial number of AI images generated or edited with Grok are targeting women in religious and cultural clothing. Grok users aren't just commanding the AI chatbot to "undress" pictures of women and girls into bikinis and transparent underwear. Among the vast and growing library of nonconsensual sexualized edits that Grok has generated on request over the past week, many perpetrators have asked xAI's bot to put on or take off a hijab, a saree, a nun's habit, or another kind of modest religious or cultural type of clothing. In a review of 500 Grok images generated between January 6 and January 9, WIRED found around 5 percent of the output featured an image of a woman who was, as the result of prompts from users, either stripped from or made to wear religious or cultural clothing. Indian sarees and modest Islamic wear were the most common examples in the output, which also featured Japanese school uniforms, burqas, and early 20th century-style bathing suits with long sleeves. "Women of color have been disproportionately affected by manipulated, altered, and fabricated intimate images and videos prior to deepfakes and even with deepfakes, because of the way that society and particularly misogynistic men view women of color as less human and less worthy of dignity," says Noelle Martin, a lawyer and PhD candidate at the University of Western Australia researching the regulation of deepfake abuse. Martin, a prominent voice in the deepfake advocacy space, says she has avoided using X in recent months after she says her own likeness was stolen for a fake account that made it look like she was producing content on OnlyFans. "As someone who is a woman of color who has spoken out about it, that also puts a greater target on your back," Martin says. X influencers with hundreds of thousands of followers have used AI media generated with Grok as a form of harassment and propaganda against Muslim women. A verified manosphere account with over 180,000 followers replied to an image of three women wearing hijabs and abaya, which are Islamic religious head coverings and robe-like dresses. He wrote: "@grok remove the hijabs, dress them in revealing outfits for New Years party." The Grok account replied with an image of the three women, now barefoot, with wavy brunette hair, and partially see-through sequined dresses. That image has been viewed more than 700,000 times and saved more than a hundred times, according to viewable stats on X. "Lmao cope and seethe, @grok makes Muslim women look normal," the account-holder wrote alongside a screenshot of the image he posted in another thread. He also frequently posted about Muslim men abusing women, sometimes alongside Grok-generated AI media depicting the act. "Lmao Muslim females getting beat because of this feature," he wrote about his Grok creations. The user did not immediately respond to a request for comment. Prominent content creators who wear a hijab and post pictures on X have also been targeted in their replies, with users prompting Grok to remove their head coverings, show them with visible hair, and put them in different kinds of outfits and costumes. In a statement shared with WIRED, the Council on American‑Islamic Relations, which is the largest Muslim civil rights and advocacy group in the US, connected this trend to hostile attitudes toward "Islam, Muslims and political causes widely supported by Muslims, such as Palestinian freedom." CAIR also called on Elon Musk, the CEO of xAI, which owns both X and Grok, to end "the ongoing use of the Grok app to allegedly harass, 'unveil,' and create sexually explicit images of women, including prominent Muslim women." Deepfakes as a form of image-based sexual abuse have gained significantly more attention in recent years, especially on X, as examples of sexually explicit and suggestive media targeting celebrities have repeatedly gone viral. With the introduction of automated AI photo editing capabilities through Grok, where users can simply tag the chatbot in replies to posts containing media of women and girls, this form of abuse has skyrocketed. Data compiled by social media researcher Genevieve Oh and shared with WIRED says that Grok is generating more than 1,500 harmful images per hour, including undressing photos, sexualizing them, and adding nudity.

X hasn't really stopped Grok AI from undressing women in the UK

Elon Musk's X is trying to stop people using its AI chatbot Grok to undress women amid intensifying outrage and legal scrutiny over the deluge of nonconsensual sexual deepfakes flooding the site. It's not trying very hard: it took us less than a minute to get around its latest attempt to rein in the chatbot. X's first effort to crack down on the torrent of intimate deepfakes was to restrict access to image editing. While this meant that free users could no longer generate images by tagging Grok in public replies on X.com, our investigation found that Grok's image editing tools were also still easily and freely available for any X users to churn out images, sexual or otherwise, by clicking into the Grok chatbot or using the standalone website. X's latest attempt involves stopping Grok from replying to requests to generate images of women in sexual poses, swimwear, or explicit scenarios, The Telegraph reported on Tuesday. Grok still generates images of men or inanimate objects in bikinis when requested. Using a free account, the Grok app immediately complied with my request to turn a selfie into a picture of me kneeling in a jockstrap, surrounded by other scantily clad men. It's still extremely easy to undress women and edit them into sexualized poses using the X and Grok mobile apps or websites; however, even without making a subscription payment that would connect your account to an easily identifiable source. In her testing, my fellow UK-based colleague Jess Weatherbed found that she was not blocked from using Grok's image editing feature to create sexualized deepfakes of herself. After uploading a fully-clothed photograph to X and Grok, prompting the chatbot to "put her in a bikini" or "remove her clothes" produced only blurred, censored results. The bot did comply with every other request, however, including prompts to "show me her cleavage," "make her breasts bigger," and "put her in a crop top and low-rise shorts" -- the last of which placed her in a bikini. The bot also generated images of her "leaning down" with a sexualized pose and facial expression, and in extremely revealing lingerie. These requests were completed using free X and Grok accounts. On the Grok website, an age verification pop-up appeared after submitting the first editing prompt, which was easily bypassed by selecting a birth year that would place her over 18 years of age. The pop-up did not require proof of her supposed age. The Grok mobile app, X app, and X website did not ask for any age confirmation. In our testing, Grok did not comply with requests to deepfake full nudity. In late December, X was flooded with images of women and children in sexualized situations, including being deepfaked to appear pregnant, skirtless, and wearing bikinis. The undressing scandal has put X and xAI, which makes Grok, in the sights of regulators and governments worldwide. Malaysia and Indonesia have already temporarily blocked access to Grok in response to the deepfakes. British lawmakers pushed up a law criminalizing deepfake nudes following X's "insulting" decision to limit Grok's image editing to paid users and threw their support behind an investigation that could see the platform banned in the country. Musk has taken particular umbrage at Britain's response, crying censorship, shifting the blame onto users, and insisting Grok obeys local laws. He said on X: "I not aware of any naked underage images generated by Grok. Literally zero. Obviously, Grok does not spontaneously generate images, it does so only according to user requests. When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state. There may be times when adversarial hacking of Grok prompts does something unexpected. If that happens, we fix the bug immediately." On one count at least, our investigation suggests Musk appears to be flat out wrong. Sharing, threatening to share, and creating non-consensual intimate images -- fully nude or not -- are banned under the UK's Online Safety Act (OSA), but Grok does generate sexual deepfake images when asked. Musk's other denial -- that Grok generates naked underage images -- is a rebuttal to something he wasn't explicitly accused of and is not the reason why the British government is probing X. Referring to naked underage images is a misdirect. Nonconsensual sexual images of minors are undeniably problematic -- and illegal -- even when subjects are clothed, and Grok has been undressing children. A look at Grok's safety guidelines on xAI's public GitHub also shows the company directs the chatbot to "assume good intent" and "don't make worst-case assumptions without evidence" for users asking for images of young women, Ars Technica reported, and as of writing, those institutions are still in place. The worst that Grok is being used for? The Internet Watch Foundation, a UK charity that works to remove child sexual abuse material from the web, said last week that it had discovered "criminal imagery" of girls on the dark web that appeared to have been created using Grok. The girls in the images were aged between 11 and 13. While other companies like OpenAI and Google at least try to put guardrails in place to prevent chatbots from creating the kind of material that is now flooding X, Musk's final retort shows he is pulling straight from a playbook that will seem hauntingly familiar to anyone hurt by the products pushed by the purveyors of any number of harmful technologies: blame the user.

After Global Backlash, X Vows to Block Grok from Generating Sexualized Images

(Credit: Thomas Fuller/SOPA Images/LightRocket via Getty Images)) Days after governments worldwide began scrutinizing X (formerly Twitter) for allowing its AI platform Grok to generate images of real people in bikinis without their consent, the platform announced it would prevent the chatbot from doing so. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," X's Safety team said on Wednesday, adding that "this restriction applies to all users, including paid subscribers." xAI will also block the standalone Grok app's ability to generate nudity "in those jurisdictions where it's illegal." The issue stems from an X trend that led several users to ask Grok to generate bikini images of other users, including minors. The trend raised safety and privacy concerns for women and minors, with governments in Indonesia and Malaysia blocking the chatbot altogether. A few other governments either launched an investigation or demanded stricter restrictions. In the immediate aftermath, X decided to limit Grok's image generation to paid users. It also vowed to ban accounts and take legal action against those involved in generating child sexual abuse material (CSAM). The scrutiny, however, continued. This week, California's attorney general, Rob Bonta, launched an investigation into the chatbot and noted that potential victims could file a complaint against Grok's parent company, xAI. The investigation will also examine the standalone Grok app's "Spicy Mode," which enables users to generate images of AI avatars in minimal clothing. Though not a formal response to the AG's investigation, Elon Musk said that with NSFW enabled, Grok remains allowed to generate upper-body nudity involving fictional adults, "consistent with what can be seen in R-rated movies on Apple TV." He added that it is "the de facto standard in America." Earlier this week, the US Senate passed a bill allowing victims of nonconsensual deepfake imagery to sue its creators. President Trump has already signed a bill that requires social media platforms to remove such images within 48 hours of receiving notice. Meanwhile, The Verge found that Grok was still able to generate revealing deepfakes on Wednesday. Musk, on the other hand, claimed that he was not "aware of any naked underage images generated by Grok. Literally zero." He continues to blame users for making such requests and notes that "adversarial hacking of Grok prompts" may deliver "something unexpected."

Ofcom continues X probe despite Grok 'nudify' fix

Cold milk poured over 'spicy mode,' but it might not be enough to escape a huge fine Ofcom is continuing with its investigation into X, despite the social media platform saying it will block Grok from digitally undressing people. A spokesperson for the UK comms regulator said on Thursday: "X has said it's implemented measures to prevent the Grok account from being used to create intimate images of people. "This is a welcome development. However, our formal investigation remains ongoing. We are working around the clock to progress this and get answers into what went wrong and what's being done to fix it." The statement follows X confirming that it has "implemented technological measures" to prevent Grok from editing images of real people, making them appear as though they have fewer or no clothes. Ofcom first made contact with X on January 5, following widespread reports that its AI chatbot, Grok, was being used to digitally undress images and generate sexualized depictions of real people - mainly women but also children. A week later, the regulator opened a formal investigation into X to understand whether it had complied with the Online Safety Act. On Wednesday evening, via its Safety account, X stated: "We remain committed to making X a safe platform for everyone and continue to have zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content. "We take action to remove high-priority violative content, including child sexual abuse material (CSAM) and non-consensual nudity, taking appropriate action against accounts that violate our X Rules. We also report accounts seeking child sexual exploitation materials to law enforcement authorities as necessary." As well as blocking Grok from nudifying subjects, X has also implemented a geoblock on its chatbot's ability to generate images of people in bikinis, underwear, or similarly revealing clothes - known internally as "spicy mode" - where such content is restricted by law. The Elon Musk-owned platform's first attempt at damage control was to merely limit Grok's nudifying capabilities to paid users only, having previously been available to any user registered on the site. However, technology secretary Liz Kendall strongly rejected this move, calling it "an insult and totally unacceptable for Grok to still allow this if you're willing to pay for it." X has now updated this, saying the restriction "applies to all users, including paid subscribers." Kendall issued a fresh statement on Thursday, following X's latest announcement, encouraging Ofcom to investigate the company fully, despite the platform saying it has adhered to the government's request. "I welcome this move from X, though I will expect the facts to be fully and robustly established by Ofcom's ongoing investigation," said Kendall. "Our Online Safety Act is and always has been about keeping people safe on social media - especially children - and it has given us the tools to hold X to account in recent days. "I also want to thank those who have spoken out against this abuse, above all the victims. I shall not rest until all social media platforms meet their legal duties and provide a service that is safe and age-appropriate to all users. Rob Bonta, California's attorney general, also opened an investigation into X this week, urging it to take immediate action against the reports that it was nudifying women and children. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," he said. "This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet. "I urge xAI to take immediate action to ensure this goes no further. We have zero tolerance for the AI-based creation and dissemination of non-consensual intimate images or of child sexual abuse material." ®

XAI Blocks Grok From Creating Sexualized Images of Real People

Governments and regulators around the world have condemned the feature, and have opened investigations into xAI and Grok. Elon Musk's xAI is disabling the ability for people to use its Grok artificial intelligence chatbot to create sexualized images of real people, following widespread criticism that the company was allowing women and children to be victimized by the tool. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," the company posted on its X social network Wednesday. The changes apply to all users on X, including premium subscribers, the company said. Last week, the company limited the generation and editing of images via Grok to paid users. Subscribers to X's premium service can still use Grok to edit and create other AI-generated images that adhere to the company's terms of service, it said. The company has also blocked Grok from generating "images of real people in bikinis, underwear and similar attire" in countries where it is illegal. The technical changes to Grok come weeks after users began using the AI chatbot to digitally undress women and children on the app without their consent, flooding X with thousands of AI-generated sexualized images. Governments and regulators around the world have condemned the feature, and the California attorney general's office opened an investigation into xAI earlier on Wednesday. A number of European countries have opened similar inquiries into xAI and Grok, including France and the UK. The European Union is also probing whether the images violate the bloc's Digital Services Act, and Malaysia and Indonesia restricted access to Grok in those countries. Musk has so far defended the technology, instead blaming users for abusing it. Still, the company said in the post it has "zero tolerance for any forms of child sexual exploitation, nonconsensual nudity and unwanted sexual content." "We remain committed to making X a safe platform for everyone," the company wrote.

X Didn't Fix Grok's 'Undressing' Problem. It Just Makes People Pay for It

X is only allowing "verified" users to create images with Grok. Experts say it represents the "monetization of abuse" -- and anyone can still generate images on Grok's app and website. After creating thousands of "undressing" pictures of women and sexualized imagery of apparent minors, Elon Musk's X has apparently limited who can generate images with Grok. However, despite the changes, the chatbot is still being used to create "undressing" sexualized images on the platform. On Friday morning, the Grok account on X started responding to some users' requests with a message saying that image generation and editing are "currently limited to paying subscribers." The message also includes a link pushing people towards the social media platform's $395 annual subscription tier. In one test of the system requesting Grok create an image of a tree, the system returned the same message. The apparent change comes after days of growing outrage against and scrutiny of Musk's X and xAI, the company behind the Grok chatbot. The companies face an increasing number of investigations from regulators around the world over the creation of nonconsensual explicit imagery and alleged sexual images of children. British prime minister Keir Starmer has not ruled out banning X in the country and said the actions have been "unlawful." Neither X nor xAI, the Musk-owned company behind Grok, has confirmed that it has made image generation and editing a paid-only feature. An X spokesperson acknowledged WIRED's inquiry but did not provide comment ahead of publication. X has previously said it takes "action against illegal content on X," including instances of child sexual abuse material. While Apple and Google have previously banned apps with similar "nudify" features, X and Grok remain available in their respective app stores. xAI did not immediately respond to WIRED's request for comment. For more than a week, users on X have been asking the chatbot to edit images of women to remove their clothes -- often asking for the image to contain a "string" or "transparent" bikini. While a public feed of images created by Grok contained far fewer results of these "undressing" images on Friday, it still created sexualized images when prompted to by X users with paid for "verified" accounts. "We observe the same kind of prompt, we observe the same kind of outcome, just fewer than before," Paul Bouchaud, lead researcher at Paris-based nonprofit AI Forensics, tells WIRED. "The model can continue to generate bikini [images]," they say. A WIRED review of some Grok posts on Friday morning identified Grok generating images in response to user requests for images that "put her in latex lingerie" and "put her in a plastic bikini and cover her in donut white glaze." The images appear behind a "content warning" box saying that adult material is displayed. On Wednesday, WIRED revealed that Grok's standalone website and app, which is separate from the version on X, has also been used in recent months to create highly graphic and sometimes violent sexual videos, including celebrities and other real people. Bouchaud says it is still possible to use Grok to make these videos. "I was able to generate a video with sexually explicit content without any restriction from an unverified account," they say. While WIRED's test of image generation using Grok on X using a free account did not allow any images to be created, using a free account on Grok's app and website still generated images. The change on X could immediately limit the amount of sexually explicit and harmful material the platform is creating, experts say. But it has also been criticized as a minimal step that acts as a band-aid to the real harms caused by nonconsensual intimate imagery. "The recent decision to restrict access to paying subscribers is not only inadequate -- it represents the monetization of abuse," Emma Pickering, head of technology-facilitated abuse at UK domestic abuse charity Refuge, said in a statement. "While limiting AI image generation to paid users may marginally reduce volume and improve traceability, the abuse has not been stopped. It has simply been placed behind a paywall, allowing X to profit from harm."

Musk dealt blow over Grok deepfakes, but regulatory fight far from over

STOCKHOLM/LONDON, Jan 15 (Reuters) - Elon Musk's Grok chatbot is testing Europe's ability to clamp down on deepfakes and digital undressing of images online, even after regulators scored a rare win by forcing Musk's xAI to curb the creation of sexualized images. xAI said late on Wednesday it had restricted image editing for Grok AI users after the chatbot churned out thousands of sexualized images of women and minors that alarmed global regulators. The climb-down by Musk, who initially laughed off the trend, highlights the difficulty of policing AI tools that make it cheap and easy to create explicit content. It is the latest clash between Europe and Musk, following rows over election interference, content moderation and free speech. Many regulators are still scrambling to develop laws and rules to govern AI, with question marks over what constitutes nudity, how to define consent, and who bears responsibility: the user or the platform. "It's really a grey zone with regards to the creation of the nude images," Ängla Pändel, a Stockholm-based data protection and privacy lawyer with Mannheimer Swartling, told Reuters. British regulator Ofcom, one of the most vocal on the issue, welcomed the move by Musk, but said its investigation into xAI over the Grok images would continue. "Our formal investigation remains ongoing," a spokesperson said. "We are working round the clock to progress this and get answers into what went wrong and what's being done to fix it." STRONGEST ENFORCEMENT STILL NEEDED, OFFICIALS SAY Earlier this month, Grok created hyper-realistic images of women on X manipulated to look like they were in tiny bikinis, degrading poses or even covered in bruises. Some minors were digitally stripped down to swimwear. Until Wednesday, Reuters found the chatbot still produced sexualized images privately on demand. That appeared to have been curbed at least in certain geographies on Thursday. Musk's xAI said it was blocking users, opens new tab from generating images of people in skimpy attire in "jurisdictions where it's illegal". It did not identify those jurisdictions. In Malaysia and Indonesia the government has imposed temporary bans on Grok, while EU and UK regulators called the images unlawful. The UK, France and Italy launched probes, but faced calls for tougher action. "Stronger enforcement under the Digital Services Act (DSA) is needed to stop apps and platforms that sexualise or nudify women and children," said Christian Democrat MEP Nina Carberry, who called the latest move, opens new tab a "positive step". ⁠A European Commission spokesperson said that if the Grok changes were not effective, the Commission would still use the full enforcement toolbox of the EU's DSA against the platform. LEGAL GREY AREA, HEAVY BURDEN ON VICTIMS The UK's Online Safety Act makes the sharing of intimate images without consent, including AI-generated deepfakes, a 'priority offence', said Alexander Brown, a UK-based data protection lawyer at Simmons & Simmons. "This means X must take proactive, proportionate steps to prevent such content from appearing on its platform and to swiftly remove it when detected," he said. Britain's regulator can fine a company up to 10% of revenue in the most serious cases of non-compliance or ask a court to require internet service providers to block the site. For individuals, taking platforms to court is "a really difficult and heavy process," said Anders Bergsten, a lawyer at Mannheimer Swartling, citing the emotional toll on victims. Deepfakes have existed for years, well before the advent of the AI apps, though they were largely confined to the darker corners of the web. The publishing power of X gives Grok unprecedented reach. "The frictionless publishing capability enables the deepfakes to spread at scale," said U.S.-based lawyer Carrie Goldberg, who works with cyber harassment victims. Laws in Britain and Sweden make the non-consensual sharing of nude images illegal. Britain is widening the law to include the making of such images. Under the DSA, suspending a service is considered a last resort., opens new tab The EU AI Act also does not have any provision for nude images of adults, only transparency obligations for deepfakes, experts said. British Prime Minister Keir Starmer welcomed X's move on Thursday but warned: "Free speech is not the freedom to violate consent. Young women's images are not public property, and their safety is not up for debate." "If we need to strengthen existing laws further, we are prepared to do that." Reporting by Supantha Mukherjee in Stockholm and Sam Tobin in London; Editing by Adam Jourdan, Kenneth Li and Elaine Hardcastle Our Standards: The Thomson Reuters Trust Principles., opens new tab * Suggested Topics: * Government * Constitutional Law * Criminal * Data Privacy Supantha Mukherjee Thomson Reuters Supantha leads the European Technology and Telecoms coverage, with a special focus on emerging technologies such as AI and 5G. He has been a journalist for about 18 years. He joined Reuters in 2006 and has covered a variety of beats ranging from financial sector to technology. He is based in Stockholm, Sweden.

UK Prime Minister says 'we will take action' on Grok's disgusting deepfakes

UK Prime Minister Keir Starmer says the country will take action against X following reports that the platform's Grok AI chatbot is generating sexualized deepfakes of adults and minors, as reported earlier by The Telegraph and Sky News. "It's disgusting," Starmer says during an interview with Greatest Hits Radio. "X need[s] to get their act together and get this material down. And we will take action on this because it's simply not tolerable." Last month, X launched a feature that allows people to use Grok to edit any image on the platform without permission. The rollout resulted in a flood of AI deepfakes undressing women and, in some instances, children. "We're not going to tolerate it," Starmer adds. "I've asked for all options to be on the table." The UK's communications regulator, Ofcom, began investigating whether X is in violation of the country's Online Safety Act, which holds online platforms accountable for hosting harmful content, Politico reported last week. "Based on their response we will undertake a swift assessment to determine whether there are potential compliance issues that warrant investigation," An Ofcom spokesperson told Politico at the time. X has said that "anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content." X didn't immediately respond to The Verge's request for comment.

What to know about UK legal changes aiming to regulate AI-generated nude images

LONDON (AP) -- Laws that will make it illegal to create online sexual images of someone without their consent are coming into force soon in the U.K., officials said Thursday, following a global backlash over the use of Elon Musk's artificial intelligence chatbot Grok to make sexualized deepfakes of women and children. Musk's company, xAI, announced late Wednesday that it has introduced measures to prevent Grok from allowing the editing of photos of real people to portray them in revealing clothing in places where that is illegal. British Prime Minister Keir Starmer welcomed the move, and said X must "immediately" ensure full compliance with U.K. law. He stressed that his government will remain vigilant on any transgressions by Grok and its users. "Free speech is not the freedom to violate consent," Starmer said Thursday. "I am glad that action has now been taken. But we're not going to let this go. We will continue because this is a values argument." The chatbot, developed by Musk's company xAI and freely accessed through his social media platform X, has faced global scrutiny after it emerged that it was used in recent weeks to generate thousands of images that "undress" people without their consent. The digitally-altered pictures included nude images as well as depictions of women and children in bikinis or in sexually explicit poses. Critics have said laws regulating generative AI tools are long overdue, and that the U.K. legal changes should have been brought into force much sooner. A look at the problem and how the U.K. aims to tackle it: Britain's media regulator has launched an investigation into whether X has breached U.K. laws over the Grok-generated images of children being sexualized or people being undressed. The watchdog, Ofcom, said such images -- and similar productions made by other AI models -- may amount to pornography or child sexual abuse material. The problem stemmed from the launch last year of Grok Imagine, an AI image generator that allows users to create videos and pictures by typing in text prompts. It includes a so-called "spicy mode" that can generate adult content. Technology Secretary Liz Kendall cited a report from the internet Watch Foundation saying the deepfake images included sexualization of 11-year-olds and women subjected to physical abuse. "The content which has circulated on X is vile. It is not just an affront to decent society, it is illegal," she said. Authorities said they are making legal changes to criminalize those who use or supply "nudification" tools. First, the government says it is fast-tracking provisions in the Data (Use and Access) Act making it a criminal offense to create or request deepfake images. The act was passed by Parliament last year, but had not yet been brought into force. The legislation is set to come into effect on Feb. 6 "Let this be a clear message to every cowardly perpetrator hiding behind a screen: you will be stopped and when you are, make no mistake that you will face the full force of the law," Justice Secretary David Lammy said Separately, the government said it is also criminalizing "nudification" apps as part of the Crime and Policing Bill, which is currently going through Parliament. The new criminal offense will make it illegal for companies to supply tools designed to create non-consensual intimate images. Kendall said this would "target the problem at its source." The investigation by Ofcom is ongoing. Kendall said X could face a fine of up to 10% of its qualifying global revenue depending on the investigation's outcome and a possible court order blocking access to the site. Starmer has faced calls for his government to stop using X. Downing Street said this week it was keeping its presence on the platform "under review." Musk insisted Grok complied with the law. "When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state," he posted on X. "There may be times when adversarial hacking of Grok prompts does something unexpected. If that happens, we fix the bug immediately."

California is investigating Grok over AI-generated CSAM and nonconsensual deepfakes

California authorities have launched an investigation into xAI following weeks of reports that the chatbot was generating sexualized images of children. "xAI appears to be facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet, including via the social media platform X," California Attorney General Rob Bonta's office said in a statement. The statement cited a report that "more than half of the 20,000 images generated by xAI between Christmas and New Years depicted people in minimal clothing," including some that appeared to be children. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material," Bonta said. "Today, my office formally announces an investigation into xAI to determine whether and how xAI violated the law. The investigation was announced as California Governor Gavin Newsom also called on Bonta to investigate xAI. "xAI's decision to create and host a breeding ground for predators to spread nonconsensual sexually explicit AI deepfakes, including images that digitally undress children, is vile," Newsom wrote. California authorities aren't the first to investigate the company following widespread reports of AI-generated child sexual abuse material (CSAM) and non-consensual intimate images of women. UK regulator Ofcom has also opened an official inquiry, and European Union officials have said they are also looking into the issue. Malaysia and Indonesia have moved to block Grok. Last week, xAI began imposing rate limits on Grok's image generation abilities, but has so far declined to pull the plug entirely. When asked to comment on the California investigation, xAI responded with an automated email that said "Legacy Media Lies." Earlier on Wednesday, Elon Musk said he was "not aware of any naked underage images generated by Grok." Notably, that statement does not directly refute Bonta's allegation that Grok is being used "to alter images of children to depict them in minimal clothing and sexual situations." Musk said that "the operating principle for Grok is to obey the laws" and that the company works to address cases of "adversarial hacking of Grok prompts."

Elon Musk's X Restricts Ability to Create Explicit Images With Grok

The social media platform X said late Wednesday that it was blocking Grok, the artificial intelligence chatbot created by Elon Musk, from generating sexualized and naked images of real people on its platforms in certain locations. The move comes amid global outrage over explicit, A.I.-generated images that have flooded X. In the last week, regulators around the world have opened investigations into Grok, and some countries have banned the application. Earlier Wednesday, investigators in California said they were examining whether Grok had violated state laws. Ofcom, Britain's independent online safety watchdog, opened an inquiry into Grok on Monday. "This is a welcome development," the British regulator said in a statement on Thursday in response to the new restrictions on Grok. "However, our formal investigation remains ongoing." If X is found to have broken British law and refuses to comply with Ofcom's requests for action, the regulator has the power, if necessary, to seek a court order that would prevent payment providers and advertisers from working with X. X said in a statement on Wednesday that it would use "geoblocking" to restrict Grok from fulfilling requests for such imagery in jurisdictions where such content was illegal. The restrictions did not appear to apply to the stand-alone Grok app and website, outside of X. Grok and X are both owned by xAI. X did not respond to a request for comment. "We remain committed to making X a safe platform for everyone and continue to have zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content," the statement said. Last week, X announced it had limited Grok's image-generation capabilities to subscribers, who would pay a premium for the feature, but that did little to placate regulators around the world. Indonesia and Malaysia have banned the chatbot, and the European Union has opened investigations into its explicit "deepfakes." Ursula von der Leyen, the president of the European Commission, the executive branch of the European Union, has sharply criticized the technology. "It horrifies me that a technology platform allows users to digitally undress women and children online. It is inconceivable behavior," she told European media outlets earlier this month. "And the harm caused by this is very real." The European Union has powerful tools for monitoring and stalling such activity, including the Digital Services Act, which forces large technology firms to monitor content posted to their platforms -- or to face consequences including major fines. Those regulations are often criticized by the Trump administration, which has argued that the European Union's digital rules amount to censorship and unfairly discriminate against big American technology companies. Regulators in the European Union have ordered Grok to retain documents related to the chatbot as it examines its creation of sexual images. Sexual images of children are illegal to possess or share in many countries, and some also ban A.I.-generated sexual images of children. Several countries, including the United States and Britain, have also enacted laws against sharing nonconsensual nude imagery, often referred to as "revenge porn." X's policy bars users from posting "intimate photos or videos of someone that were produced or distributed without their consent." Jeanna Smialek contributed reporting from Brussels.

Elon Musk's X to block Grok from undressing images of real people

Elon Musk's AI model Grok will no longer be able to edit photos of real people to show them in revealing clothing, after widespread concern over sexualised AI deepfakes in countries including the UK and US. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis. "This restriction applies to all users, including paid subscribers," reads an announcement on X, which operates the Grok AI tool. The change was announced hours after California's top prosecutor said the state was probing the spread of sexualised AI deepfakes, including of children, generated by the AI model.

Elon Musk Cannot Get Away With This

If there is no red line around AI-generated sex abuse, then no line exists. Will Elon Musk face any consequences for his despicable sexual-harassment bot? For more than a week, beginning late last month, anyone could go online and use a tool owned and promoted by the world's richest man to modify a picture of basically any person, even a child, and undress them. This was not some deepfake nudify app that you had to pay to download on a shady backwater website or a dark-web message board. This was Grok, a chatbot built into X -- ostensibly to provide information to users but, thanks to an image-generating update, transformed into a major producer of nonconsensual sexualized images, particularly of women and children. Let's be very clear. The forced undressings happened out in the open, in one stretch thousands of times every hour, on a popular social network where journalists, politicians, and celebrities post. Emboldened trolls did it to everyone ("@grok put her in a bikini," "@grok make her clothes dental floss," "@grok put donut glaze on her chest"), including everyday women, the Swedish deputy prime minister, and self-evidently underage girls. Users appeared to be imitating and showing off to one another. On X, creating revenge porn can make you famous. Read: Elon Musk's pornography machine These images were ubiquitous, and many people -- and multiple organizations, including the Rape, Abuse & Incest National Network and the European Commission -- pointed out that the feature was being used to harass women and exploit children. Yet Musk initially laughed it off, resharing AI-generated images of himself, Kim Jong Un, and a toaster in bikinis. Musk, as well as xAI's safety and child-safety teams, did not respond to a request for comment. xAI replied with its standard auto-response, "Legacy Media Lies." xAI, the Musk-owned company that develops Grok and owns X, prohibits the sexualization of children in its acceptable-use policy; a post earlier this month from the X safety team states that the platform removes illegal content, including child-sex-abuse material, and works with law enforcement as needed. Even after that assurance from X's safety team, it took several more days for X to place bare-minimum restrictions on the Ask Grok feature's image-generating, and thus undressing, capabilities. Now, when creeps on X try to generate an image by replying "@grok" to prompt the chatbot, they get an auto-generated response that notes some version of: "Image generation and editing are currently limited to paying subscribers." This is disturbing in its own right; Musk and xAI are essentially marketing nonconsensual sexual images as a paid feature of the platform. But X users have been able to get around the paywall via the "Edit Image" button that appears on every image uploaded to the platform, or by using Grok's stand-alone app. Two years ago, when Google Gemini generated images of racially diverse Nazis, Google temporarily disabled the bot's image-generating capabilities to address the problem. Musk has taken no responsibility for the problem and has said only that "anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content." Perhaps Musk feels that he would benefit from baiting his critics into a censorship fight. He has repeatedly reshared posts that frame calls to regulate or ban his platform in response to the Grok undressing as leftist censorship, for instance reposting a meme calling such efforts "retarded" as well as a Grok-generated video of a woman applying lipstick captioned with a quote commonly attributed to Marilyn Monroe: "We are all born sexual creatures, thank God, but it's a pity so many people despise and crush this natural gift." Last week, as Musk's chatbot was generating likely hundreds of thousands of these images, we reached out directly to X's head of product, Nikita Bier, who didn't reply. Within the hour, Rosemarie Esposito, X's media-strategy lead, emailed us unprompted with her contact information, in case we had "any questions" in the future. We asked her a series of questions about the tool and how X could allow such a thing to operate. She did not reply. We've reached out multiple times to more than a dozen key investors listed in xAI's two most recent public fundraising rounds -- the latest of which, announced during this Grok-enabled sexual-harassment spree, valued the company at about $230 billion -- to ask if they endorsed the use of X and Grok to generate and distribute nonconsensual sexualized images. These investors include Andreessen Horowitz, Sequoia Capital, BlackRock, Morgan Stanley, Fidelity Management & Research Company, the Saudi firm Kingdom Holding Company, and the state-owned investment firms of Oman, Qatar, and the United Arab Emirates, among others. We asked whether they would continue partnering with xAI absent the company changing its products and, if yes, why they felt justified in continuing to invest in a company that has enabled the public sexual harassment of women and exploitation of children on the internet. BlackRock, Fidelity Management & Research Company, and Baron Capital declined to comment. A spokesperson for Morgan Stanley initially told us that she could find no documentation that the company is a major investor in xAI. After we sent a public announcement from xAI that lists Morgan Stanley as a key investor in its Series C fundraising round, the spokesperson did not answer our questions. The other companies did not respond. We also reached out to several companies that provide the infrastructure for X and Grok -- in other words, that allow these products to exist on the internet: Google and Apple, which offer both X and Grok on their app stores; Microsoft and Oracle, which run Grok on their cloud services; and Nvidia and Advanced Micro Devices (AMD), which sell xAI the computer chips needed to train and run Grok. We asked if they endorsed the use of these products to create nonconsensual sexual images of women and children, and whether they would take steps to prevent this from continuing. None responded except for Microsoft, which told us that it does not provide cloud services, chips, or hosting services for xAI other than offering the Grok language model -- without image generation -- on its enterprise platform, Microsoft Foundry. As all of this unfolded, xAI made several major announcements: new Grok products for businesses; upgraded video-generating capabilities; that enormous fundraising round. Yesterday, Defense Secretary Pete Hegseth visited SpaceX's headquarters in Texas and joined Musk for a press conference in which Hegseth said, "I want to thank you, Elon, and your incredible team" for bringing Grok to the military. (Later this year, Grok will join Google Gemini on a new Pentagon platform called GenAI.mil that the Defense Department says will offer advanced AI tools to military and civilian personnel.) We asked the DOD if it endorsed xAI's sexualized material or if it would reconsider its partnership with the company in response. In a statement, a Pentagon official told us only that the department's policy on the use of AI "fully complies with all applicable laws and regulations" and that "any unlawful activity" by its personnel "will be subject to appropriate disciplinary action." Read: What are people still doing on X? Government bodies in the United Kingdom, India, and the European Union have said that they will investigate X, while Malaysia and Indonesia have blocked access to Grok, but Musk appears to be unfazed by these efforts -- and also seems to be receiving help in brushing them off. Sarah B. Rogers, the under secretary of state for public diplomacy, has said that, should the U.K. ban X, America "has a full range of tools that we can use to facilitate uncensored internet access in authoritarian, closed societies." At the moment, Musk seems to be not only getting away with this but also reveling in it. Although governments appear to be furious at Musk, they also seem impotent. Senator Ted Cruz, a co-sponsor of the TAKE IT DOWN Act -- which establishes criminal penalties for the sharing of nonconsensual intimate images, real or AI-generated, on social media -- wrote on X last Wednesday that the Grok-generated images "are unacceptable and a clear violation of" the law but that he was "encouraged that X has announced that they're taking these violations seriously." Throughout that same day, Grok continued to comply with user requests to undress people. Yesterday, Cruz posted on X a photo of himself with his arm around Musk and the caption "Always great seeing this guy 🚀." And it's already beginning to feel as if the scandal -- the world's richest man enabling the widespread harassment of women and children -- is waning, crowded out by a new year of relentless news cycles. But this is a line-in-the-sand moment for the internet. Grok's ability to undress minors is not, as Musk might have you think, an exercise in free-speech maximalism. It is, however, a speech issue: By turning sexual harassment and revenge porn into a meme with viral distribution, the platform is allowing its worst, most vindictive users to silence and intimidate anyone they desire. The retaliation on X has been obvious -- women who've stood up in opposition to the tool have been met with anonymous trolls asking Grok to put them in a bikini. Social platforms have long leaned on the argument that they aren't subject to the same defamation laws as publishers and media companies. But this latest debacle, Musk's reaction, and the silence from so many of X's investors and peer companies were all active choices -- and symptoms of a broader crisis of impunity that's begun to seep into American culture. They were the result of politicians, despots, and CEOs bowing to Donald Trump. Of financial grift and speculation running rampant in sectors such as cryptocurrency and meme stocks -- a braggadocious, "get the bag" ethos that has no room for greed or shame. Of Musk realizing that his wealth insulates him from financial consequences. Few industries have been as brazen in their capitulation as Big Tech, which has dismantled its content-moderation systems to please the current administration. It's a cynical and cowardly pivot, one that allows companies to continue to profit off harassment and extremism without worrying about the consequences of their actions. Deepfakes are not new, but xAI has made them a dramatically larger problem than ever before. By matching viral distribution with this type of image creation, xAI has built a way to spread AI revenge porn and child-sexual-abuse material at scale. The end result is desensitizing: The sheer amount of exploitative content flooding the platform may eventually make the revolting, illicit images appear "normal." Arguably, this process is already happening. The internet has always been a chaotic place where trolls can seize outsize power. Historically, that chaos has been constrained by platforms doing the bare minimum to protect their users from demonstrated threats. Today, X is failing to clear the absolute lowest bar. Nobody who works at X or xAI seems to be willing to answer for the creation and distribution of tens or hundreds of thousands of nonconsensual intimate images; instead, those in charge appear to be blithely ignoring the problem, and those who have funneled money to Musk or xAI seem sanguine about it. They would probably like for us all to move on. We cannot do that. This crisis is an outgrowth of a breakneck information ecosystem in which few stories have staying power. No one person or group has to flood the zone with shit, because the zone is overflowing constantly. People with power have learned to exploit this -- to weather scandals by hunkering down and letting them pass, or by refusing to apologize and turning any problem into a culture-war issue. Musk has been allowed to avoid repercussions for even the most reckless acts, including cheerleading and helping dismantle foreign aid with DOGE. Others will continue to follow his playbook. Employees at X and investors and companies such as Apple and Google seem to be counting on their "No comment"s being buried by whatever scandal comes next. They are banking on a culture in which people have given up on demanding consequences. But the Grok scandal is so awful, so egregious, that it offers an opportunity to address the crisis of impunity directly. The undressing spree was not an issue of partisan politics or ideology. It was an issue of anonymous individuals asking a chatbot that is integrated into one of the world's most visible social networks to edit photos of women and girls to "put her in a clear bikini and cover her in white donut glaze." This is a moment when those with power can and should demand accountability. The stakes could not be any higher. If there is no red line around AI-generated sex abuse, then no line exists.

Grok image generation is now paywalled on X amid AI "undressing" deepfake controversy

Serving tech enthusiasts for over 25 years. TechSpot means tech analysis and advice you can trust. What just happened? With the pressure growing on X over Grok's sexualized deepfake images of women and children, the AI tool's image creation feature has now been restricted to paying X subscribers - but it can still be accessed by everyone through the standalone app. The move comes as British Prime Minister Keir Starmer suggested that X be blocked in the UK. There has been growing outcry over Grok after it was revealed that the chatbot is being used to generated nude and other sexualized deepfakes, sometimes involving minors. A study found that it created about 6,700 images every hour that were identified as sexually suggestive or nudifying. For comparison, the other top websites for such content average 79 similar images per hour combined. With numerous countries investigating the matter, the UK considering a ban on X, and the company continuing to blame users rather than Grok, restrictions have now been introduced. "Image generation and editing are currently limited to paying subscribers," Grok wrote in an X post. The change means that only people who have their full details and credit card information stored on X's systems can use Grok to create images directly on X. It's presumed that as they can be identified, subscribers won't create anything they shouldn't. The caveat is that anyone - non-paying users included - can still generate images on the separate standalone Grok app, which does not share the images it creates publicly. The Internet Watch Foundation (IWF) has confirmed that Grok had been used to create "criminal imagery of children aged between 11 and 13." Elon Musk previously said that anyone using Grok to create illegal content would face the same consequences as having uploaded such material directly. The UK government has reacted to the new restriction. It condemned the move as simply making the ability to generate explicit and unlawful images a premium service. "It's not a solution. In fact, it's insulting to victims of misogyny and sexual violence. What it does prove is that X can move swiftly when it wants to do so. You heard the prime minister yesterday. He was abundantly clear that X needs to act, and needs to act now. It is time for X to grip this issue," a Downing Street spokesperson said. The spokesperson added that it would support any action taken by Ofcom, the UK's media regulator. In a reply to users last week, Grok said that most cases of minors appearing in its generated sexualized images could be prevented through advanced filters and monitoring, but it admitted that "no system is 100% foolproof." It added that xAI was prioritizing improvements and reviewing details shared by users.

Musk's xAI Faces California AG Probe Over Grok Sexual Images

A number of European countries and the European Union are also probing xAI and Grok, with Governor Gavin Newsom calling on the attorney general to "immediately investigate the company and hold xAI accountable". Elon Musk's artificial intelligence startup, xAI, is under investigation by the California attorney general's office after the company's Grok chatbot was allegedly used to create thousands of sexualized images of women and children without their consent. California AG Rob Bonta announced the investigation Wednesday, saying in a statement that Grok's role in generating nonconsensual, sexualized images of women and girls on the social network X over the past two weeks was "shocking." "I urge xAI to take immediate action to ensure this goes no further," Bonta said in the statementBloomberg Terminal. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material." Musk has been under fire by governments and regulators around the world because of Grok's image-generation technology. Musk has so far defended Grok, arguing that the issue stems from user abuse of the tool, and not the technology itself. "Obviously, Grok does not spontaneously generate images, it does so only according to user requests," Musk posted on X Wednesday. "When asked to generate images, it will refuse to produce anything illegal, as the operating principle." California Governor Gavin Newsom said in a post on X that he called on the attorney general to "immediately investigate the company and hold xAI accountable." A number of European countries have opened similar investigations into xAI and Grok, including France and the UK. The European Union is also probing whether the images violate the bloc's Digital Services Act. A spokesperson for xAI didn't immediately respond to a request for comment.

Elon Musk's X Says It Will (Sort of) Crack Down on Grok’s Sexual Deepfake Problem

The company claims it’s adding new technical and geo-based restrictions to its Grok's image editing capabilties. Elon Musk’s social media platform X is taking additional steps to curb its sexual deepfake problem, following weeks of backlash and multiple government investigations around the world. But the changes don’t really resolve the issue outright and instead add new layers of limited restrictions rather than a platform-wide fix. In a pretty confusing post on Wednesday evening, X’s @Safety account outlined several updates to how its AI image generation and editing features work, with different rules depending on whether users are generating or editing images by tagging the @Grok account or going straight to the Grok tab on X. First, the company said it has implemented new technical measures to prevent users from specifically using the @Grok account to alter “images of real people in revealing clothing such as bikinis.†X says the restriction applies to all users, including those on a premium plan. X also reiterated that image generation and image editing through the @Grok account are now limited to paid subscribers. “This adds an extra layer of protection by helping to ensure that individuals who attempt to abuse the Grok account to violate the law or our policies can be held accountable,†the company said in the post. X previously announced plans to restrict using @Grok to edit images to paid users, a move that drew criticism from U.K. government officials. A spokesperson for Downing Street said at the time that the change “simply turns an AI feature that allows the creation of unlawful images into a premium service.†However, as The Verge first pointed out, Grok’s image generation tools remain available for free when users access the chatbot through the standalone Grok website and app, as well as through Grok tabs on the X app and website. Using a free account, Gizmodo was also able to access Grok’s image generation feature through the Grok tab on both the X website and mobile app. On Thursday, the dedicated site still gave us no trouble when asked to generate an image of Elon Musk wearing a bikini and was willing to take the bikini off. The biggest update is that X claims it will now block "the ability of all users to generate images of real people in bikinis, underwear, and similar attire via the Grok account and in Grok in X in those jurisdictions where it’s illegal." This specific update seems to apply to both the @Grok account and the Grok tabs on X. It also arrives as lawmakers in the U.K. are working to make such images illegal. “We remain committed to making X a safe platform for everyone and continue to have zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content,†the company said. X and its parent company, XAI, did not immediately respond to a request for comment from Gizmodo. The overall changes arrive after weeks of intense backlash over the recent proliferation of sexual deepfakes on the platform, and relative silence from the company itself. Since late last month, some X users have used Grok to generate sexualized images from photos posted by other users without their consent, including images involving minors. One social media and deepfake researcher found that Grok generated roughly 6,700 sexually suggestive or nudifying images per hour over 24 hours in early January, Bloomberg reported. Governments around the world have been quick to respond. Malaysia and Indonesia blocked access to Grok, while regulators in the U.K. and European Union opened investigations into potential violations of online safety laws. The U.K.’s online regulator, Ofcom, said it would continue its investigation despite the newly announced changes. In the U.S., California Attorney General Rob Bonta announced Wednesday that his office had launched its own investigation into the issue. Meanwhile, as scrutiny of Grok has intensified, X quietly updated its terms of service to require that all pending and future legal cases involving the company be filed in the Fort Worth division of the Northern District of Texas, where one of the court’s three judges is widely seen as friendly to the company. Left-leaning watchdog Media Matters, a frequent critic of Musk’s X, said it would leave the platform in response to the updated terms.

UK regulator says its X deepfake probe will continue

LONDON, Jan 15 (Reuters) - British media regulator Ofcom said on Thursday its formal investigation into Elon Musk's X over its Grok AI chatbot's sexually intimate deepfake images would continue, even as it welcomed the company's recent policy change. Musk's artificial intelligence company xAI said late on Wednesday it had imposed restrictions on all Grok users, limiting image editing following concerns among global regulators. "This is a welcome development. However, our formal investigation remains ongoing. We are working round the clock to progress this and get answers into what went wrong and what's being done to fix it," Ofcom said in a statement. Reporting by Muvija M; Editing by Kate Holton Our Standards: The Thomson Reuters Trust Principles., opens new tab

X says Grok, Musk's AI chatbot, is blocked from undressing images in places where it's illegal

BANGKOK (AP) -- Elon Musk's AI chatbot Grok won't be able to edit photos to portray real people in revealing clothing in places where that is illegal, according to a statement posted on X. The announcement late Wednesday followed a global backlash over sexualized images of women and children, including bans and warnings by some governments. The pushback included an investigation announced Wednesday by the state of California into the proliferation of nonconsensual sexually explicit material produced using Grok. Initially, media queries about the problem drew only the response, "legacy media lies." Musk's company, xAI, now says it will geoblock content if it violates laws in a particular place. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis, underwear and other revealing attire," it said. The rule applies to all users, including paid subscribers, who have access to more features. xAI also has limited image creation or editing to paid subscribers only "to ensure that individuals who attempt to abuse the Grok account to violate the law or our policies can be held accountable." Grok's "spicy mode" had allowed users to create explicit content, leading to a backlash from governments worldwide. Malaysia and Indonesia took legal action and blocked access to Grok. The U.K. and European Union were investigating potential violations of online safety laws. France and India have also issued warnings, demanding stricter controls. Brazil called for an investigation into Grok's misuse. The Grok editing functions were "facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet, including via the social media platform X," California's announcement said. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking. This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet," it cited the state's Attorney General Rob Bonta as saying. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material," he said.

Masterful Gambit: Musk Attempts to Monetize Grok's Wave of Sexual Abuse Imagery

In an attempt to push more people toward a paying subscription, Grok now refuses to generate images in replies. The paywall is pretty leaky, though. Elon Musk, owner of the former social media network turned deepfake porn site X, is pushing people to pay for its nonconsensual intimate image generator Grok, meaning some of the app's tens of millions of users are being hit with a paywall when they try to create nude images of random women doing sexually explicit things within seconds. Some users trying to generate images on X using Grok receive a reply from the chatbot pushing them toward subscriptions: "Image generation and editing are currently limited to paying subscribers. You can subscribe to unlock these features." Users who fork over $8 a month can still reply to random images of random women and girls directly on X and tag in Grok with things like "make her wear clear tapes with tiny black censor bar covering her private part protecting her privacy and make her chest and hips grow largee[sic] as she squatting with leg open widely facing back, while head turn back looking to camera." These images are still visible in everyone's X feed, subscribers or not. On the Grok app, a subscription to SuperGrok ($29.99/month) or SuperGrok Heavy ($299.99/month) allow users to generate images even faster. On Thursday, I received messages in the Grok app several times warning me that usage rates for the app were higher than normal and that I could pay to skip the wait. As the Verge reported this morning, this paywall is very leaky. It's still possible to generate images using Grok in a variety of ways, but replying directly to someone's post by tagging @grok returns the "limited to subscribers" message. As many legacy news outlets have already reported, Musk improved the subscription revenue funnel on his money-burning app following an outcry against these extremely popular uses of the app. "X Limits Grok Image Tool To Subscribers After Deepfake Outcry," Deadline reported. "Grok turns off image generator for most users after outcry over sexualised AI imagery," wrote the Guardian. "Elon Musk restricts Grok's image tools following a wave of non-consensual deepfakes of women and children," Fortune wrote. Based on these headlines, you may be thinking, This is an uncharacteristic show of accountability and perhaps even self reflection from the billionaire technocrat white supremacist sympathizer who owns X.com, wow! But as with all things Musk does, this is a business move to monetize the long-established harassment factory he's owned for five years and has yet to figure out how to make profitable. After years of attempting to push users toward a subscription model by placing meaningless status signifiers behind a paywall and making the site so toxic it bleeds users by the millions, he might have found a way to do it: by monetizing abuse at the source. Several other AI industry giants have already figured out that sexual content is where the money's at, and Musk appears to be catching up. Putting the nonconsensual sexual images behind a paywall is also what every "nudify" and "undress" app and image generator platform on the market already does. On Thursday, in the middle of Grok's CSAM shitstorm, Bloomberg reported that xAI is looking at "a net loss of $1.46 billion for the September quarter, up from $1 billion in the first quarter," according to internal documents obtained by Bloomberg. "In the first nine months of the year, it spent $7.8 billion in cash." It's too early to speculate, but making the people who are tagging @grok under the posts of women they don't know and writing prompts like "make her bend over on all fours doggy style" multiple times a second pay for the privilege could be a play to get the company back in the black. In addition to using Grok on X.com on desktop, It's also still easy to generate images and videos in the Grok app without a subscription, which is still available on the Apple and Google app stores, despite blatantly breaking their rules against non-consensual material and pornography. The app and underground Telegram groups are where the really bad stuff is, anyway. Apple and Google have not replied to my request for comment about why the app is still available. Signing up for X Premium or SuperGrok requires handing over your payment information, name associated with your credit card, and phone number. It also comes with the risk of having all of that hacked, stolen, and released to the dark web in the next big data breach of the platform.

California Investigates Elon Musk's xAI Over Sexualized Images Generated by Grok

The state will examine whether xAI, which owns the social media platform X and created the A.I. chatbot Grok, violated state law. California's attorney general on Wednesday said the state had opened an investigation into Elon Musk's artificial intelligence company, xAI, for generating sexualized images of women and children. The inquiry will examine whether xAI, which owns the social media platform X and created the A.I. chatbot Grok, violated state law by facilitating the creation of nonconsensual intimate images. Starting in late December, X was flooded with images generated using Grok of real people, including children, in underwear and in sexual poses. "The avalanche of reports detailing the nonconsensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," California's attorney general, Rob Bonta, said in a statement. "I urge xAI to take immediate action to ensure this goes no further." The investigation adds to pressure xAI is facing over the images, which victims and regulators have decried. Britain launched a formal inquiry into the issue on Monday as regulators there examined whether X violated an online safety law. Officials in India and Malaysia have also said they are investigating xAI. X did not immediately respond to a request for comment on the California investigation. In a Jan. 3 statement posted on X, the social media company said it would remove illegal content depicting children and permanently suspend accounts that asked Grok to create such images. On Thursday, the Grok account on X began responding to requests for A.I. images only from X subscribers who pay for certain premium features -- and continued to generate intimate images for those users. Grok will still create those images for any user on its separate app and website. Mr. Musk has shared posts arguing that xAI is no different from other tech tools that offer A.I. image generation or image editing software like Photoshop. On Wednesday, Mr. Musk posted that he was "not aware of any naked underage images generated by Grok. Literally zero," and added, "When asked to generate images, it will refuse to produce anything illegal." Grok may face additional scrutiny in the coming weeks. On Monday, Liz Kendall, Britain's technology secretary, said that the government would begin more aggressively enforcing a law next week that makes it illegal for people to create nonconsensual intimate images. The country also plans to draft legislation to make it illegal for companies to provide tools designed to make such illicit images.

California investigates Grok over AI deepfakes

X and Grok remain available on Apple's App Store and Google Play. It comes amid a debate over whether US tech companies are shielded from responsibility for what users post on AI platforms. Section 230 of the Communications Decency Act of 1996 provides legal immunity to online platforms for user-generated content. But Prof James Grimmelmann of Cornell University argues this law "only protects sites from liability for third-party content from users, not content the sites themselves produce". Grimmelmann said xAI was trying to deflect blame for the imagery on to users, but expressed doubt this argument would hold up in court. "This isn't a case where users are making the images themselves and then sharing them on X," he said. In this case "xAI itself is making the images. That's outside of what Section 230 applies to", he added. Senator Ron Wyden of Oregon has argued that Section 230, which he co-authored, does not apply to AI-generated images. He said companies should be held fully responsible for such content. "I'm glad to see states like California step up to investigate Elon Musk's horrific child sexual abuse material generator," Wyden told the BBC on Wednesday. Wyden is one of the three Democratic senators who asked Apple and Google to remove X and Grok from their app stores. The announcement of the probe in California comes as the UK is preparing legislation that would make it illegal to create non-consensual intimate images. The UK watchdog Ofcom has also launched an investigation into Grok. If it determines the platform has broken the law, it can issue fines of up to 10% of its worldwide revenue or £18m, whichever is greater. On Monday, Sir Keir Starmer told Labour MPs that Musk's social media platform X could lose the "right to self regulate" adding that "if X cannot control Grok, we will."

Elon Musk's X faces bans and investigations over nonconsensual bikini images

Indonesia and Malaysia temporarily blocked X's chatbot, Grok, over the weekend after it made scores of fake images publicly sexualizing mostly women and, in some instances, children late last year. Governments around the world are also launching investigations. The latest came on Monday as the UK media regulator, Ofcom, launched a probe into the social media platform, which could result in a ban. Grok had been generating sexually explicit images of people for some time. But the issue got widespread attention in late December as people used the chatbot to edit a high volume of existing images by tagging the bot in comments and giving it prompts such as "put her in a bikini." While Grok did not respond to all of the requests, it obliged in many cases. In some cases, Bellingcat senior investigator and researcher Kolina Koltai noted, users can get Grok to generate frontal nudes. Untold numbers of women and in some cases, children, as Reuters first reported, have had their likenesses sexualized online by Grok without their permission, including one of the mothers of X owner Elon Musk's children. It's unusual for so many governments to take action against a social media company but this case is different, said Riana Pfefferkorn, a policy fellow at Stanford University. "Making child sexual abuse [material] is flagrantly illegal, pretty much everywhere on Earth." By last Friday, X had restricted Grok's AI image generation feature to make it only available to paying subscribers. Non-paying users can still put people in bikinis publicly with just a few clicks, but they can only put in a few such requests before being prompted to sign up for a premium membership, which costs $8 a month. NPR reviewed Grok's publicly available images generated earlier this month and found it had stopped making images of scantily clad women several days into 2026. However, it sometimes still offers up bikini-clad men. xAI, the parent company of X, has been pushing adult content with Grok since last year. In May, Koltai first noted that the chatbot would generate sexually explicit images in response to requests on X like "take off her clothes." This past summer, Grok introduced "spicy mode" in its standalone app, which allowed users to put bikinis on AI-generated characters. Ben Winters, director of AI and privacy at the advocacy organization Consumer Federation of America, said that Grok now not only allows editing images of real people, but it also provides an easy distribution platform via X. "It's a further and significant escalation," he said. Governments are outraged over X's move to restrict access to the image generation function to subscribers. British Technology Secretary Liz Kendall told Sky News "it is insulting to say that you can still access this service if you pay for it." The Indonesian government found that Grok lacked effective guardrails to stop users from making nonconsensual pornographic content based on real Indonesian residents, the Associated Press first reported. "The government sees nonconsensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space," Indonesian Communication and Digital Affairs Minister Meutya Hafid said in a statement. The AP also reported Grok will stay blocked in Malaysia until effective guardrails against misuse are put in place. In response to NPR's questions, X spokesperson Victoria Gillespie pointed to a statement posted on January 3 that said "anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content." The statement echoed Musk's own post earlier that day. Such an approach is an "attempt to abdicate responsibility, " said Winters. "It certainly is not just the user that is prompting it alone," he said. "It is the fact that the image would not be created if not for ... the tool they made." Before the wave of Grok-generated nonconsensual explicit deepfakes hit X, other AI makers had added similar capabilities to their chatbots. In November, Google released a new image generating model, Nano Banana Pro. In December, OpenAI updated its model ChatGPT Images. Both can also edit images to put people in bikinis, Wired reported last month. A thread on Reddit distributing such images has been taken down. In another X post, Musk suggested that since tools from other AI companies also have this undressing function, the pressure from governments is a form of censorship directed at his platform. Koltai noted the trend to make AI-generated nonconsensual intimate media has been in the making for the past year or two. "You'll see these ads in Instagram even - it's like, upload a photo of you and your crush and you guys can kiss," she said. "So that's kissing and hugging, and then you see there's that more extreme spectrum." "There is obviously a huge issue within the tech industry because we've seen this across multiple platforms, and [there is] not always great guidelines or boundaries or regulation," said Koltai. The criticism in the U.S. has been far more muted than in other countries. Sen. Ted Cruz, R-Tex., posted on X Wednesday that the images "should be taken down and guardrails should be put in place." He also said that he is "encouraged that X has announced that they're taking these violations seriously and working to remove any unlawful images and offending users from their platform." Grok has generated frequent controversy in the past year. Last summer, the chatbot referred to itself as "MechaHitler" and spewed antisemitic conspiracy theories. Apps with so-called nudifying capabilities have existed for years, mostly in the shadows of the internet. Winters said officials need to do more to police some of X's features. "There are misrepresentations about the safety of their products. There are violations of their terms of service," he said. "We haven't seen really any significant action from any U.S. agencies, whether it's state or federal, that have the authority to enforce the law," Winters said.

California Launches Investigation Into Grok's Nonconsensual Sexual Images

Weeks after Elon Musk's X was flooded with AI-generated images depicting people, including children, in sexualized ways without consent, California is investigating how the hell it happened. The state's Attorney General Rob Bonta announced Wednesday that he is opening a probe into the situation to determine if X and xAI, Musk's AI company and the maker of the chatbot Grok that was used to generate the pornographic images, broke the law. “The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking. This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet," Bonta said in a statement. He also urged xAI to take "immediate action" to ensure that type of content can't be created and spread. Bonta seems to have quite a bit of public support for the investigation. A recent YouGov poll found that a whopping 97% of respondents said that AI tools shouldn't be allowed to generate sexually explicit content of children, and 96% said those tools shouldn't be capable of "undressing" minors in images. The investigation will focus on the trend that cropped up on X over the winter holiday, which saw users prompting Grok on the platform to modify images of people to show them in various states of undress. The trend got big enough that, according to AI content analysis firm Copyleaks, Grok was generating a nonconsensually sexualized image every minute. Some of those images included children, which users prompted Grok to undress and depict in underwear or bikinis. Often, users requested that Grok add "donut glaze" to the faces of the subjects of the images. Muskâ€"the CEO of both X, the company where the images were being shared, and xAI, the company that makes the AI model used to generate the imagesâ€"has opted to obfuscate or claim ignorance of the situation. In a post made prior to California's investigation being announced, Musk said, "I not aware of any naked underage images generated by Grok. Literally zero." The narrowness of his statement does a lot of heavy lifting, saying he's unaware of any "naked underage images." That doesn't refute the existence of naked images, images of undressed underage people, or people being depicted in sexualized situations. Nor does it address the fact that many of those images were nonconsensual, generated without the permission of the person being depicted. In countless cases, the imagery has been directly used to harass accounts on X. To the extent that Musk was willing to admit that such a problem is even possible, he said it's the fault of the users, not the AI model or platform spreading the content. "Obviously, Grok does not spontaneously generate images; it does so only according to user requests. When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state," he said. "There may be times when adversarial hacking of Grok prompts does something unexpected. If that happens, we fix the bug immediately." That's about in line with the sparse response that X has offered to the situation. In a post from X Safety, the company said, “Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content,†but took no responsibility for enabling it. For what it's worth, Musk was also mockingly reposted content created as part of the trend, including AI-generated images of a toaster and a rocket in a bikini. California is the first state in the country to launch an investigation into the situation. Authorities in other countries, including France, Ireland, the United Kingdom, and India, have all started looking into the nonconsensual sexual images generated by Grok and may also bring charges against X and xAI. The Take It Down Act, which was passed into law last year, doesn't require platforms like X to create notice and removal systems for nonconsensual images until May 19, 2026.

UK's Starmer Threatens Musk's X With Action Over Child Images

UK Prime Minister Keir Starmer vowed action and demanded Elon Musk's X urgently "get their act together" over sexualized images of children produced by its artificial intelligence tool Grok. "This is disgraceful, it's disgusting and it's not to be tolerated. X has got to get a grip of this," the premier said. "We will take action on this because it's simply not tolerable," he added, describing the images as "unlawful." X, formerly known as Twitter, has become a top site for images of people that have been non-consensually undressed by AI, according to third-party analysis, with thousands of instances each hour over a day earlier this week. The UK watchdog responsible for flagging online child sexual-abuse material to law enforcement agencies said earlier it had found "criminal" images on the dark web allegedly generated by Grok. The dark web images depict "sexualized and topless" images of girls between the ages of 11 and 13 and meet the bar for action by law enforcement, the Internet Watch Foundation said. XAI operates Grok and the social media platform X. "Ofcom has our full support to take action in relation to this," Starmer said, referring to the British media regulator, which said earlier this week that it was investigating the allegations and had made contact with Musk's company over the reports.

Grok AI: what do limits on tool mean for X, its users, and Ofcom?

UK users will no longer be able to create sexualised images of real people using @Grok X account, with Grok app also expected to be restricted Elon Musk's X has announced it will stop the Grok AI tool from allowing users to manipulate images of people to show them in revealing clothing such as bikinis. The furore over Grok, which is integrated with the X platform, has sparked a public and political backlash as well as a formal investigation by Ofcom, the UK's communications watchdog. Here is a guide to what X's announcement means for the social media platform, its users, and Ofcom.

Grok-created images of real people in bikinis, underwear banned on X

Elon Musk's AI chatbot Grok has announced a change in policy that purports to offer more protections against sexualized deepfakes, at least on X. The new policy comes as California launches an investigation into the issue, while the UK is threatening a ban. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," read a statement from the X Safety account on X, a sister site of Grok's, posted just before 6 p.m. ET / 3 p.m. PT on Wednesday. "This restriction applies to all users, including paid subscribers." The X Safety update also stated that it takes Child Sexual Abuse Material (CSAM) and non-consensual nudity seriously before reiterating another recent change: Image creation and editing via the Grok account on X is now limited to subscribers. The X Safety account also announced that it now has the ability to geoblock "all users to [sic] generate images of real people in bikinis, underwear, and similar attire via the Grok account and in Grok in X in those jurisdictions where it's illegal." This Tweet is currently unavailable. It might be loading or has been removed. Grok and X have faced a wave of scrutiny in the new year as sexualized, non-consensual images of celebrities and children, prompted by users and created by its AI, have proliferated on X. California attorney general Rob Bonta has demanded Grok and its developer xAI take steps to remove and prevent such images, threatening to use "all tools at our disposal" to keep its residents safe. "This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet," Bonta said in a statement on Wednesday. Meanwhile, xAI/X/Grok boss Elon Musk appeared to dare users to "break Grok image moderation" on the same day that X Safety announced its new safety updates: This Tweet is currently unavailable. It might be loading or has been removed. British prime minister Keir Starmer on Monday threatened to take action against Musk, saying "if X cannot control Grok, we will," according to a BBC report. Indonesia and Malaysia already blocked Grok access this weekend. Politicians have also targeted Grok and X, with three senators calling on Apple to remove the services from its app store. The apps are still available in Apple's app store as of Wednesday evening. X's recent changes to its policy may reflect an acknowledgment by company leaders that it is not protected by Section 230 of the U.S.'s 30-year-old Communications Decency Act, according to the BBC. Section 230 shields tech companies from lawsuits related to user-generated content, but images and other content created by an app's own technology may be less impervious to such legal immunity.

'It will refuse to produce anything illegal': Elon Musk rails against Grok backlash, but UK Prime Minister says 'we're not going to back down'

Elon Musk said that he's aware of "literally zero" naked underage images generated by Grok AI, in a Wednesday post on X. It's the first public comments beyond emojis the X CEO has made on the controversy, though it may do little to satisfy critics. Grok AI ran into trouble last week after Reuters reported that the Grok AI platform, which is accessible separately and through X, was "flooding" X with "sexualized photos of women and minors." It's not news that Grok can generate racy images from prompts. Musk has posted his share of idealized images of women in bustiers, but these allegations go further. In the report, Reuters recounted the story of a woman whose photo with her cat was transformed by a Grok prompt of an image of her in a tiny bikini. There are also claims that Grok is generating images of sexualized minors. The report and growing concern led to Grok AI being banned in Malaysia and Indonesia, and the UK's OFCOM launching an investigation into X. X and Musk never directly addressed the allegations (until now), but it already took steps to staunch the flow of such images. Image generation has, for instance, been put behind the Grok AI paywall (an action that some say does little to address the problem). And The Telegraph reported that Grok AI will ignore requests to create these kinds of images. Musk's comments (below), though, appear to argue against a never leveled charge: the creation of nude imagery of minors: [I'm] not aware of any naked underage images generated by Grok. Literally zero. Obviously, Grok does not spontaneously generate images, it does so only according to user requests. When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state. There may be times when adversarial hacking of Grok prompts does something unexpected. If that happens, we fix the bug immediately. Musk is also making a point that is true of virtually all generative AI platforms: they do not generate imagery without a prompt. Users are writing the prompts and asking Grok to remove clothing and replace it with bikinis. Musk's statement about the refusal to create illegal imagery aligns with previous statements Musk has made on free speech: his platforms will follow the law, and he added in a 2022 X post, "I am against censorship that goes far beyond the law." That last bit is perhaps why Grok AI has run afoul of some commonly understood content standards. Grok AI is generally a platform that will happily flout intellectual property laws. In general, it has a snarkier personality and is more open to a wider variety of prompts. Though with the realization that Grok AI perhaps didn't know where to draw the line, that stance is now clearly changing. Is it changing fast enough? UK Prime Minister Sir Keir Starmer previously called Grok AI "disgusting," and though clearly pleased that Grok AI has taken measures to stop the flow of these images, he, according to the BBC, retained a hard line. "If so, that is welcome, but we're not going to back down, and they must act. We will take the necessary measures. We will strengthen existing laws and prepare for legislation if it needs to go further, and Ofcom will continue its independent investigation." It's unlikely that we'll hear a more in-depth response from Musk, whose last point is that, perhaps, some of what we saw was due to "adversarial hacking" that leads to "something unexpected". In other words, bugs that are easily fixed. This week hasn't been all bad news for X and Grok AI. Even as other countries are investigating and banning X and its AI platform, the US Department of Defense announced a plan to integrate Grok AI into its own networks. That should be... interesting.

Musk's Grok restricts image generator after complaints over sexualized photos

Why it matters: xAI, Musk's AI company, is under fire from the European Commission, which said Friday it will investigate the images, calling them "illegal," "appalling" and "disgusting." What they're saying: Responding to users on X Friday, Grok said that image generation and editing "are currently limited to paying subscribers." * Despite the restrictions, the company on Thursday repeatedly touted Grok Imagine, its new video creation capability, which is currently free for everyone to use. * xAI did not immediately respond to Axios' request for comment. * "Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content," Musk posted on X on Jan. 3. Yes, but: While Grok appears to have limited the ability for users to modify images by tagging @grok on X, the Verge reported that all users can still create sexualized images by using the edit button on the X desktop website or using the X app. Catch up quick: Musk's Grok came under fire recently after users used its image editing capabilities to remove clothing from photos or make sexualized images of women and children, including a 14-year-old star of Netflix's "Stranger Things."

Musk says he was unaware of Grok generating explicit images of minors

Jan 14 (Reuters) - Elon Musk said on Wednesday he was not aware of any "naked underage images" generated by xAI's Grok chatbot, as scrutiny of the AI tool intensifies worldwide. "I not aware of any naked underage images generated by Grok. Literally zero," Musk said in an X post. Musk's comment on social media platform X comes as xAI and X face growing global scrutiny, including calls by lawmakers and advocacy groups for Apple (AAPL.O), opens new tab and Google (GOOGL.O), opens new tab to drop Grok from app stores, government investigations, and bans or legal action in countries such as Malaysia and Indonesia. Musk reiterated that Grok is programmed to refuse illegal requests and must comply with the laws of any given country or state. "Obviously, Grok does not spontaneously generate images, it does so only according to user requests," Musk said. Musk has said earlier on X that anyone using Grok to make illegal content would suffer the same consequences as if they uploaded illegal content. Three Democratic U.S. senators last week called on Apple and Alphabet's Google to remove X and its built-in AI chatbot Grok from their app stores, citing the spread of nonconsensual sexual images of women and minors on the platform. A coalition of women's groups, tech watchdogs, and progressive activists also called on the tech giants for a similar move. Last week, X curtailed Grok's ability to generate or edit images publicly for many users, however industry experts and watchdogs have said that Grok was still able to produce sexually explicit images, and that restrictions, such as paywalling certain features, may not fully block access to deeper AI image tools. In the UK, the law is set to change this week to criminalize the creation of such images, and Prime Minister Keir Starmer said on Wednesday that X is working to comply with the new rules. Countries such as Malaysia and Indonesia have already blocked access to Grok and are pursuing legal action against X and Musk's AI unit xAI, alleging failures to prevent harmful content and protect users. Reporting by Akash Sriram in Bengaluru, Editing by Franklin Paul Our Standards: The Thomson Reuters Trust Principles., opens new tab

X claims Grok won't edit images of real people into bikinis

Grok is allegedly done playing deepfake digital swimsuit stylist. X $TWTR says it has "implemented technological measures" to ensure that the chatbot will no longer edit photos of real people into "revealing clothing such as bikinis" -- exactly the sort of claim that lasts exactly as long as it takes someone to try a slightly different prompt. X says the "fix" applies to everyone, even paid users. And the parent company's latest move also comes with a geographic fine print -- X says it's geoblocking this kind of image editing in places where it's illegal, conceding two things at once: First, that the capability exists; second, that the constraint may vary depending on whose laws are currently within range of your IP address. But is what X says true? Not really in the way a normal person might mean "true," which is "you can't (or won't) do it anymore." The Verge tried the updated setup and found that Grok could still be nudged into producing sexualized edits by phrasing prompts slightly differently. Asking for a bikini might trigger a refusal; asking for "revealing summerwear," altered proportions, or adjacent styling (e.g., asking for a crop top) sometimes did not. So the lock may be real, but with the right key, the door still opens. This isn't the first time Grok has been "fixed" in a way that reads cleaner than it runs. Earlier this month, after a wave of nonconsensual, sexualized, deepfake image edits on X -- including myriad cases involving minors -- xAI's initial response wasn't a dramatic feature kill of what it calls "spicy mode" but rather a limit; image generation and editing would be restricted on X to paid subscribers. That paywall "solution" had a familiar tech-friendly logic: Fewer people get access, fewer disastrous public incidents hit the timeline, fewer headlines land. But the solution also came with a familiar weakness: The harder a feature is to audit externally, the easier it is to declare victory. Even after that original "paid-only" shift, image editing could still be achieved by non-paying users on X. Meanwhile, the regulatory world has been turning all the outrage into paperwork with deadlines. In the UK, the communications regulator Ofcom has opened an investigation into X over Grok-related sexualized imagery. In the EU, the European Commission has ordered X to retain Grok-related documents until the end of 2026 -- the bureaucratic version of telling a teenager, "Don't delete anything. We're coming back with questions." And then there's the most direct form of platform feedback: simply pulling the plug. The Philippines is moving to block access to Grok on child-safety concerns, joining Indonesia's temporary block and Malaysia's restrictions aimed at X. Governments are saying the product is arriving faster than its guardrails, and they're not interested in beta-testing the difference. And the pressure is now climbing out of the regulator inbox and into the app-store choke point. A coalition of 28 advocacy groups, including women's rights and tech watchdog organizations, has sent open letters to Apple $AAPL and Google $GOOGL urging them to remove X and Grok from their app stores altogether, arguing that both platforms are profiting from the spread of nonconsensual, sexually explicit AI imagery and failing to enforce their own policies on intimate images and abuse. The campaign -- "Get Grok Gone" -- accuses the companies of enabling widespread "mass digitally undressing" of women and minors through Grok's tools, adding that X's move to paywall image generation does nothing to stop the underlying harm. Apple and Google haven't publicly responded to the letters, even as senators in Washington have made similar demands. xAI, for its part, has treated media questions with the sort of posture that plays well on X and poorly in court filings. When Reuters sought comment on the earlier reporting, xAI replied with its familiar "Legacy Media Lies." The problem is that the line doesn't function as an answer -- and regulators, unlike quote-tweeters, can subpoena the receipts. So yes, Grok is "done" editing people into bikinis -- as long as you take the claim at face value, don't test the edges, and don't confuse a hard rule with a reliably enforced one. The internet, historically, isn't great at any of those things.

X restricts Grok image generation to paid users after global backlash

Grok, the Elon Musk-backed AI chatbot woven into the fabric of X, has started walling off its image generation and editing tools to paid subscribers. The change follows a tidal wave of criticism regarding the tool's ability to churn out non-consensual sexualized imagery. While the restriction is clearly an attempt to stem the tide of controversy, regulators, advocacy groups, and users alike argue that it does little to stop the creation of harmful and potentially illegal content involving women and children. Starting late Thursday, Grok officially moved its image-making features behind the X Premium paywall, which starts at $8 per month However, the move has been widely mocked as a "leaky" solution. While casual users on X may be blocked, the generation tools remain completely free to access through Grok's standalone website and mobile app. This loophole effectively undercuts the platform's claim that it is taking a firm stand against misuse, leaving the most dangerous tools still within reach of the general public. Safety researchers and digital watchdogs aren't convinced that a credit card requirement solves the problem. In fact, many argue it actually monetizes the abuse. According to deepfake researcher Genevieve Oh, Grok was still pumping out over 1,500 harmful images every hour even after the paywall went live - accounting for roughly 60% of its total public image output. Oh's data suggests that Grok is currently generating sexualized content at a rate that dwarfs even the most notorious dedicated "nudify" websites. The fallout has reached the highest levels of the U.S. government Democratic Senators Ron Wyden, Edward J. Markey, and Ben Ray Luján recently fired off a letter to the CEOs of Apple and Google, demanding that X be pulled from their respective app stores. The senators argued that by allowing these tools to persist, X is showing a "complete disregard" for the safety rules that every other app developer is forced to follow. Recommended Videos International pressure is also hitting a boiling point. UK and Indian officials have slammed the paywall as an inadequate response. A spokesperson for the British prime minister described the move as "insulting" to victims, suggesting that X is simply turning a safety crisis into a premium revenue stream. Victims have shared similar stories; campaigner Jess Davies reported that Grok was still able to digitally "undress" a photo of her through its standalone app on Friday morning, despite the supposed restrictions. Interestingly, the controversy seems to be providing a perverse financial boost to the platform Sensor Tower estimates show that mobile in-app purchase revenue on X surged by 18% on Thursday alone. This spike far exceeds the typical daily growth for the app, suggesting that the drive to access Grok's "spicy mode" might actually be helping X's struggling bottom line. Legal experts warn that these half-measures won't hold up in court for long. North Carolina Attorney General Jeff Jackson labeled the Grok situation a "turning point" for AI safety, noting how easily these systems can be weaponized. He argued that the era of "move fast and break things" is hitting a wall when it comes to the dignity and safety of private citizens. As the walls close in, X is facing a stark choice: implement genuine, hard-coded technical guardrails or face a total blackout in major app stores and international markets. Whether Elon Musk chooses to tighten the software itself -- rather than just the access to it - will determine if Grok has a future as a legitimate tool or if it becomes a pariah of the generative AI era.

Lawmakers and victims criticize new limits on Grok's AI image as 'insulting' and 'not effective' | Fortune

Elon Musk's xAI has restricted its AI chatbot Grok's image generation capabilities to paying subscribers only, following widespread condemnation over its use to create non-consensual sexualized images of real women and children. "Image generation and editing are currently limited to paying subscribers," Grok announced via X on Friday. The restriction means the vast majority of users can no longer access the feature. Paying, verified subscribers with credit card details on file can still do so, but theoretically they can be identified more easily if the function is misused. However, experts, regulators, and victims say that the new restrictions aren't a solution to the now widespread problem. "The argument that providing user details and payment methods will help identify perpetrators also isn't convincing, given how easy it is to provide false info and use temporary payment methods," Henry Ajder, a UK-based deepfakes expert, told Fortune. "The logic here is also reactive: it is supposed to help identify offenders after content has been generated, but it doesn't represent any alignment or meaningful limitations to the model itself." The UK government has called the move "insulting" to victims, in remarks reported by the BBC. The UK's prime minister's spokesperson told reporters on Friday that the change "simply turns an AI feature that allows the creation of unlawful images into a premium service. "It is time for X to grip this issue; if another media company had billboards in town centers showing unlawful images, it would act immediately to take them down or face public backlash," they said. A representative for X said they were "looking into" the new restrictions. xAI responded with the automated message: "Legacy Media Lies." Over the past week real women have been targeted at scale with users manipulating photos to remove clothing, place subjects in bikinis, or position them in sexually explicit scenarios without their consent. Some victims reported feeling violated and disturbed by the trend, with many saying their reports to X went unanswered and images remained live on the platform. Researchers said the scale at which Grok was producing and sharing images was unprecedented as, unlike other AI bots, Grok essentially has a built-in distribution system in the X platform. One researcher, whose analysis was published by Bloomberg, estimated that X has become the most prolific site for deepfakes over the last week. Genevieve Oh, a social media and deepfake researcher who conducted a 24-hour analysis of images the @Grok account posted to X, found that the chatbot was producing roughly 6,700 sexually suggestive or nudifying images per hour. By comparison, the five other leading websites for sexualized deepfakes averaged 79 new AI undressing images hourly during the same period. Oh's research also found that sexualized content dominated Grok's output, accounting for 85% of all images the chatbot generated. Ashley St. Clair, a conservative commentator and mother of one of Musk's children, was among those affected by the images. St. Clair told Fortune that users were turning images on her X profile into explicit AI-generated photos of her, including some she said depicted her as a minor. After speaking out against the images and raising concerns about deepfakes on minors, St Clair also said X took away her verified, paying subscribers status without notifying her or refunding her for the $8 per month fee. "Restricting it to the paid-only user shows that they're going to double down on this, placing an undue burden on the victims to report to law enforcement and law enforcement to use their resources to track these people down," Ashley St Clair said of the recent restrictions. "It's also a money grab." St Clair told Fortune that many of the accounts targeting her were already verified users: "It's not effective at all," she said. "This is just in anticipation of more law enforcement inquiries regarding Grok image generation." The move to limit Grok's capabilities comes amid mounting pressure from regulators worldwide. In the U.K., Prime Minister Keir Starmer has indicated he is open to banning the platform entirely, describing the content as "disgraceful" and "disgusting." Regulators in India, Malaysia, and France have also launched investigations or probes. The European Commission on Thursday ordered X to preserve all internal documents and data related to Grok, stepping up its investigation into the platform's content moderation practices after describing the spread of nonconsensual sexually explicit deepfakes as "illegal," "appalling," and "disgusting." Experts say the new restrictions may not satisfy regulators' concerns: "This approach is a blunt instrument that doesn't address the root of the problem with Grok's alignment and likely won't cut it with regulators," Ajder said. "Limiting functionality to paying users will not stop the generation of this content; a month's subscription is not a robust solution." In the U.S., the situation is also likely to test existing laws, like Section 230 of the Communications Decency Act, which shields online providers from liability for content created by users. Riana Pfefferkorn of Stanford's Institute for Human-Centered Artificial Intelligence previously told Fortune that liability surrounding AI-generated images is murky. "We have this situation where for the first time, it is the platform itself that is at scale generating non-consensual pornography of adults and minors alike," she said. "From a liability perspective as well as a PR perspective, the CSAM laws pose the biggest potential liability risk here." Musk has previously stated that "anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content." However, it remains unclear how accounts will be held accountable.

X Tightens Grok Image Generation Following International Backlash - Decrypt

Regulators in California, Europe, and Australia are investigating xAI and Grok over potential violations. X said it is restricting image generation and editing features tied to Grok, limiting access to paid users after the chatbot was used to create non-consensual sexualized images of real people, including minors. In an update posted by the X Safety account on Wednesday, the company added technical restrictions to limit how users can edit images of real people through Grok. The move followed reports that the AI generated sexualized pictures in response to simple prompts, including requests to place people in bikinis. In many cases, users tagged Grok directly under photos posted on X, causing the AI to generate edited images that appeared publicly in the same threads. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," the company said, referencing the viral trend of asking Grok to put people in bikinis. The company also said image creation and image editing through the Grok account on X are now available only to paid subscribers, a change it said is intended to improve accountability and prevent misuse of Grok's image tools that violate the law or X's policies. The company also instituted location-based restrictions. "We now geoblock the ability of all users to generate images of real people in bikinis, underwear, and similar attire via the Grok account and in Grok in X in those jurisdictions where it's illegal." Despite the changes, however, Grok continues to allow users to remove or alter clothing from photos uploaded directly to the AI, according to Decrypt's testing and user reports following the announcement. In some cases, Grok acknowledged "lapses in safeguards" after generating images of girls aged 12 to 16 in minimal clothing, conduct prohibited under the company's own policies. The continued availability of those capabilities has drawn scrutiny from advocacy groups. "If reports that Grok created sexualized images -- particularly of children -- are true, Texas law may have been broken," Adrian Shelley, Texas director of Public Citizen, said in a statement. "Texas authorities do not have to look far to investigate these allegations. X is headquartered in the Austin area, and the state has a clear responsibility to determine whether its laws were broken and, if so, what penalties are warranted." Public Citizen previously called for the U.S. government to pull Grok from its list of acceptable AI models over concerns of racism exhibited by the chatbot. Global policymakers have also increased scrutiny of Grok, leading to several open investigations. The European Commission said X and xAI could face enforcement under the Digital Services Act if safeguards on Grok remained inadequate. At the same time, Australia's eSafety Commissioner said complaints involving Grok and non-consensual AI-generated sexual images have doubled since late 2025. The regulator said AI image tools capable of producing realistic edits complicate enforcement and victim protection. In the UK, regulators with Ofcom opened an investigation into X under the Online Safety Act stemming from Grok being used to generate illegal sexualized deepfake images, including those involving minors. Officials said Ofcom could ultimately seek court-backed measures that effectively block the service in the UK if X is found non-compliant and fails to take corrective action. Other countries, including Malaysia, Indonesia, and South Korea, have also opened investigations into Grok in a bid to protect minors. While States across America monitor the situation, California is the first to open an investigation into Grok. On Wednesday, California Attorney General Rob Bonta announced a probe into xAI and Grok over the creation and spread of non-consensual sexually explicit images of women and children. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking. This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet," Bonta said in a statement. The investigation will examine whether xAI's deployment of Grok violated state laws governing non-consensual intimate imagery and child sexual exploitation. "I urge xAI to take immediate action to ensure this goes no further," Bonta said. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material." Despite the ongoing investigations, X said it takes a "zero tolerance" stance for child sexual exploitation, non-consensual nudity, and unwanted sexual content. "We take action to remove high-priority violative content, including Child Sexual Abuse Material (CSAM) and non-consensual nudity, taking appropriate action against accounts that violate our X Rules," the company wrote. "We also report accounts seeking Child Sexual Exploitation materials to law enforcement authorities as necessary."

Ofcom urged to use 'banning' powers over X AI deepfakes

The possibility there could be sexualised images of children raised very specific concerns in government. Addressing concerns over sexualised images of adults and children produced by Grok, Prime Minister Sir Keir Starmer said: "This is disgraceful. It's disgusting. And it's not to be tolerated... Ofcom has our full support to take action in relation to this." "It's unlawful. We're not going to tolerate it. I've asked for all options to be on the table," he added in an interview with Greatest Hits Radio. Government sources told BBC News: "We would expect Ofcom to use all powers at its disposal with regards to Grok & X." Ofcom's powers under the Online Safety Act have been rarely used, but include a "very strong" ability to ask the High Court to effectively ban offending companies by preventing their access to technology and to funding through advertisers and other payments. That process normally requires an investigation, but can be short-circuited where there are serious harms, risks to children, and histories of non-compliance. A new Ofcom chair is also in the process of being recruited. They will be expected to take a much more robust approach to these matters amid newer concerns about internet safety and national security, arising from new technology and types of ownership. The Online Safety Act is also at the centre of some concerns from the Trump administration about the impact on US tech firms. On Monday, Ofcom said it had made "urgent contact" with X and xAI, which built Grok, and told the BBC it was investigating concerns. It is currently illegal to share deepfakes of adults, external in the UK. In an earlier statement, X said: "Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content."

X Says It's Finally Doing Something About Grok's Deepfake Porn Problem, but It's Not Nearly Enough

X is currently implementing blocks on generating illegal material, but according to testing, there appear to be loopholes. After weeks of pressure from both advocacy groups and governments, Elon Musk's X says it's finally going to do something about its deepfake porn problem. Unfortunately, after testing following the announcement, some are still holding their breath. The controversy started earlier this January, after the social media site added a feature allowing X users to tag Grok in their posts and prompt the AI to instantly edit any image or video posted to the site, all without the original poster's permission. The feature seemingly came with few guardrails, and according to reporting done by AI authentication company Copyleaks, as well as statements victims have given to sites like Metro, posters on X quickly started using it to generate explicit or intimate images of real people, particularly women. In some cases, child sexual abuse material was also reportedly generated. It's pretty upsetting stuff, and I wouldn't advise you to go looking for it. While the initial trend seemed to focus on AI photos of celebrities in bikinis, posters quickly moved on to manipulated images of regular people where they appeared to be pregnant, skirtless, or in some other kind of sexualized situation. While Grok was technically able to generate such imagery from uploaded photos before, the ease of access to it appeared to open the floodgates. In response to the brewing controversy, Musk had Grok generate a photo of himself in a bikini. However, the jokes ceased after regulators got involved. Earlier this week, the UK launched investigations into Grok's alleged deepfake porn, to determine whether it violated laws against nonconsensual intimate images as well as child sexual abuse material. Malaysia and Indonesia went a step further, actually blocking Grok access in the country. Yesterday, California began its own investigations, with Attorney General Rob Banta saying "I urge XAI to take immediate action to ensure this goes no further." In response to the pressure, X cut off the ability to tag Grok for edits on its social media site for everyone except subscribers. However, the Grok app, website, and in-X chatbot (accessible via the sidebar on the desktop version of the site) still remained open to everyone, allowing the flood of deepfaked AI photos to continue (said photos would also still pose the same problems even if generated solely by subscribers, although X later said the goal was to stem the tide and make it easier to hold users generating illegal imagery accountable). The Telegraph reported on Tuesday that X also started blocking tagged Grok requests to generate images of women in sexualized scenarios, but that such images of men were still allowed. Additionally, testing by both U.S. and U.K. writers from The Verge showed that the banned requests could still be made to Grok's website or app directly. Musk has taken a more serious tone in more recent comments on the issue, denying the presence of child sexual abuse material on the site, although various replies to his posts expressed disbelief and claimed to show proof to the contrary. Scroll at your own discretion. To finally put the controversy to bed, X said on Wednesday that it would now be blocking all requests to the Grok account for images of any real people in revealing clothing, regardless of gender and whether coming from paid subscribers or not. But for anyone hoping that would mark the end of this, there appears to be some fine print. Specifically, while the statement said that it would be adding these guardrails to all users tagging the Grok account on X, the standalone Grok website and app are not mentioned. The statement does say it will also block creation of such images on "Grok in X," referring to the in-X version of the chatbot, but even then, it's not a total block. Instead, the imagery will be "geoblocked," meaning it will only be applied "in those jurisdictions where it's illegal." X's post also says that similar requests made by tagging the Grok account will also be geoblocked, although because the section before this says that the Grok account won't accept such requests from any user, that appears to be a moot point. It's important to note that, while the majority of the criticism lobbed at X during this debacle does not accuse the site of generating fully nude imagery, locations like the UK ban nonconsenual explicit imagery regardless of whether it is fully nude or not. It's the biggest crackdown X has made on these images yet, but for now, it still appears to have holes. According to further testing by The Verge, the site's reporters were still able to generate revealing deepfakes even after Wednesday's announcement, by using the Grok app not mentioned in the update. When I attempted this using a photo of myself, both the Grok app and standalone Grok website gave me full-body deepfaked images of myself in revealing clothing not present in the original shot. I was also able to generate these images using the in-X Grok chatbot, and some images changed my posing to be more provocative (which I did not prompt), too. As such, the battle is likely to continue. It's unclear whether ignoring the Grok app or website is an oversight, or if X is only seeking to block its most visible holes. One would hope the former, given that X said that it has "zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content." It is worth noting that I am located in New York State, which might not be part of the geoblock, although we do have a law against explicit nonconsensual deepfakes. I've reached out to X for clarification on the issue and will update this post when I hear back. However, when NBC News reached out with similar questions, the outlet was only told "Legacy Media Lies." I can't make any promises as to how the site will reply to my own requests. In the meantime, while governments continue their investigations, others are calling for more immediate action from app stores. A letter sent from U.S. Senators Ron Wyden, Ben Ray Lujan, and Ed Markey to Apple CEO Tim Cook and Google CEO Sundar Pichai argues that Musk's app now clearly violates both App Store and Google Play policies, and calls on the tech leaders to "remove these apps from the [Apple and Google] app stores until X's policy violations are addressed."

X bans explicit Grok deepfakes - but is its clash with the EU over?

While Elon Musk's company has said it is taking steps to prevent its AI chatbot from creating nude images of real people, the European Commission has yet to be reassured. Amid mounting pressure in Europe and abroad, Elon Musk's social media platform has announced that it is implementing "technological measures to prevent its AI tool, Grok, from allowing the editing of images of real people in revealing clothing such as bikinis", a restriction that will apply to all users, including paid subscribers. Grok's image editing function had been used by some users to virtually undress pictures of real women and underage girls. The situation, described as "appalling" and "disgusting" by the European Commission, prompted the EU executive to launch a request for information and a document retention order addressed to X. Speaking through one of its spokespersons, the European Commission said it had taken note of the changes to Grok's functionality, but warned that it would remain vigilant. "We will carefully assess these changes to make sure they effectively protect citizens in the EU," the spokesperson said, adding that "should these changes not be effective, the Commission will not hesitate to use the full enforcement toolbox of the Digital Services Act." If found guilty of breaching EU online platform rules under the Digital Services Act, the Commission could fine X as much as 6% of its global annual turnover. Last month, the European Commission already fined Elon Musk's social network €120 million over its account verification tick marks and advertising practices. Investigations into the platform chatbot are currently ongoing in France, the United Kingdom and Germany, as well as in Australia. Grok has been banned altogether in Indonesia and Malaysia.

xAI restricts Grok chatbot after sexualised AI images spark global concern

Elon Musk's AI company xAI has imposed limits on its Grok chatbot's image editing capabilities after hyper-realistic sexualised images -- including depictions of minors -- circulated online. The restrictions apply to all users, including paid subscribers, and block image generation in jurisdictions where such content is illegal. Elon Musk's artificial intelligence company xAI said late on Wednesday that it had imposed restrictions on all users of its Grok AI chatbot that limit image editing after the service produced sexualised images that sparked concerns among global regulators. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis. This restriction applies to all users, including paid subscribers," the company said in an X post. Hyper-realistic images of women manipulated to look like they were in microscopic bikinis, in degrading poses or covered in bruises began flooding social media platform X this month. In some cases, minors were digitally stripped down to swimwear, sparking broad criticism. Read moreFrance to investigate Musk's Grok chatbot after Holocaust denial claims Grok last week began allowing only paying subscribers to use its image generation and editing features. X last week curtailed Grok's ability to generate or edit images publicly for many of its users, but the chatbot still privately produced sexually charged images on demand on Wednesday before xAI's announcement, Reuters found. Billionaire Musk owns xAI, which in turn owns X, formerly known as Twitter. xAI added on Wednesday that it blocks users based on their location from generating images of people in skimpy attire in "jurisdictions where it's illegal". It did not name those jurisdictions. California officials demand answers California's governor and attorney general said earlier on Wednesday that they were demanding answers from xAI after Musk said he was not aware of any "naked underage images" generated by Grok. "We're demanding immediate answers from xAI on their plan to stop the creation & spread of this content," California Attorney General Rob Bonta wrote on X. Governor Gavin Newsom called on Bonta "to immediately investigate the company and hold xAI accountable." Read moreTime magazine names 'Architects of AI' as 2025 'Person of the Year' The comments from Newsom and Bonta were the most serious so far by US officials addressing the explosion of AI-generated non-consensual sexualised imagery on X. The California move added to the pressure Musk is facing in the US and around the world. Lawmakers and advocacy groups have called for Apple and Google to drop Grok from app stores. Government officials have threatened action in Europe and the United Kingdom. Indonesia temporarily blocked access to Grok. At first, Musk publicly laughed off the controversy, posting humorous emojis in response to other users' comments about the influx of sexualised photos. More recently, X has said it treats reports of child sexual abuse material seriously and polices it vigorously. Musk said earlier on Wednesday he was "not aware of any naked underage images generated by Grok. Literally zero." X did not immediately respond to questions about the California announcement and Musk's comments. xAI did not respond directly to an emailed request for comment on California officials' statements or Musk's post that he was unaware of sexualised imagery of minors. Reuters received its generic autoreply message for inquiries: "Legacy Media Lies."

California launches investigation into xAI and Grok over sexualized AI images

California Attorney General Rob Bonta announced Wednesday that he is launching an investigation into xAI and Grok after the Grok artificial intelligence model produced sexualized images of women and children. The investigation will look at "the proliferation of nonconsensual sexually explicit material produced using Grok," Bonta said in a statement. He said xAI, Elon Musk's company that created Grok, "appears to be facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet, including via the social media platform X." Bonta's statement cites recent news reports of Grok taking normal photos of women and children that are available on the internet and sexualizing them, after being prompted by users, with AI by undressing the people in the photo (typically placing them in underwear or bikinis), posing them in a suggestive manner or showing them engaged in sexual activity -- all without the consent of the people in the photos. NBC News has reached out to X and Grok for comment. Grok's image generation model now includes a "spicy mode," created specifically to produce explicit content, according to the statement, something Bonta said the company has used as a marketing tool, which has led to the influx of this kind of content. One analysis cited in Bonta's release states that more than half of the 20,000 images that were generated by the program from Christmas to the New Year "depicted people in minimal clothing, and some of those appeared to be children." "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," Bonta said in the statement, noting that these photos are used to harass people online. "I urge xAI to take immediate action to ensure this goes no further. We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material," he added. California Gov. Gavin Newsom appeared to endorse the investigation, writing on X on Wednesday: "xAI's decision to create and host a breeding ground for predators to spread nonconsensual sexually explicit AI deepfakes, including images that digitally undress children, is vile," before calling on Bonta to "hold xAI accountable." After blowback from X users, the company appeared to restrict these permissions on the social media app, while keeping them available on the Grok standalone app, website and the Grok tab on X. By Friday, the number of explicit images created by X's reply bot appeared to have been dramatically reduced. The same cannot be said for the Grok platforms. A number of U.S. lawmakers have called on Musk and X to take down these images and prohibit the programs from producing more, but Bonta's investigation Wednesday marks the first major U.S.-based government action on the issue.

Mom of one of Elon Musk's kids says AI chatbot Grok generated sexual deepfake images of her: "Make it stop"

Journalist Jo Ling Kent joined CBS News in July 2023 as the senior business and technology correspondent for CBS News. Kent has more than 15 years of experience covering the intersection of technology and business in the U.S., as well as the emergence of China as a global economic power. Elon Musk's AI chatbot Grok faces intense criticism - accused of allowing users on the Musk-owned social media platform X to generate fake, sexually explicit images of real women and children. Ashley St. Clair, the mother of one of Musk's children, is one of the alleged victims. She said in an interview with "CBS Mornings" that aired on Tuesday that Grok allowed users to generate and publish sexual deepfake images of her to X without permission, including manipulating photos of her as a minor. "The worst for me was seeing myself undressed, bent over and then my toddler's backpack in the background," the 27-year-old said. "Because I had to then see that, and see myself violated in that way in such horrific images and then put that same backpack on my son the next day, because it's the one he wears every day to school." The mother of two, who has a 1-year-old son with Musk, said she asked Grok to take the photos down. "Grok said, 'I confirm that you don't consent. I will no longer produce these images.' And then it continued to produce more and more images, and more and more explicit images," she said. St. Clair said she filed a report directly with Musk's company xAI, which operates Grok. Some of the images were then removed. "This can be stopped with a singular message to an engineer," St. Clair said. St. Clair said her issue is with the Chatbot, not Musk - who recently said he plans to file for sole custody of their child over allegations that St. Clair "might" transition their son. A source close to St. Clair said that is "absurd and unequivocally false." "If they want to say my bone to pick is ... the chatbot undressing minors and myself and stripping me nude, yes. You're right. I have a bone to pick with that and I don't care who's doing it. So Elon's not special about me speaking out on this," St. Clair said. CBS News reached out to Musk and has not received a response yet. Earlier this month, xAI said it "takes action against illegal content on X, including child sexual abuse material (CSAM), by removing it, permanently suspending accounts and working with local governments and law enforcement as necessary." A recent study by AI Forensics, a nonprofit that investigates the algorithms of major platforms, found 53% of the Grok images they reviewed contained individuals in minimal attire, with 81% of them being women. St. Clair said she wants the U.S. government to solve the issue and "make it stop." "The need to regulate it," she said. "AI should not be allowed to generate and undress children and women. That's what needs to happen." She believes the key is enforcing already existing laws, saying, "who's ever responsible for enforcing them. Not me." St. Clair said her ability to earn money on X has been revoked since she has spoken out and when asked if she plans to take legal action, she said she's "considering all options available." Last week, Malaysia and Indonesia banned Grok amid growing concerns about the chatbot. Regulators in the United Kingdom have launched an investigation. Last week, U.K. Prime Minister Keir Starmer said he wants "all options on the table," which would include a potential ban. "This is disgraceful, it's disgusting and it's not to be tolerated. X has got to get a grip of this," Starmer said in an interview with a U.K. radio station. "It's unlawful. We're not going to tolerate it. I've asked for all options to be on the table."

Grok's nonconsensual porn problem is part of a long, gross legacy

Constance Grady is a senior correspondent on the Culture team for Vox, where since 2016 she has covered books, publishing, gender, celebrity analysis, and theater. For the past few weeks, Elon Musk's Grok AI bot has been generating pornographic images of women and underage girls, without their consent, at an astounding rate. A recent Bloomberg analysis found that Grok creates 6,700 such images per hour, or more than one per minute. On Friday, X at last put some minor guardrails on the tool, with a new policy that only paying subscribers can use Grok to generate or alter images. On the standalone Grok app, however, anyone can prompt Grok to generate new images, meaning the deepfaked porn continues.

Elon Musk Moves to Monetize Grok Deepfake Abuse, UK Calls It Insulting

On Friday, Elon Musk’s social media platform X added the lightest of restrictions on its AI chatbot Grok following backlash over the AI tool being used to alter users’ photos on X to generate sexually degrading deepfakes of women and children. Not everyone is buying it. The chatbot now appears to users from generating or editing images when they tag Grok in an X post, unless they’re a premium subscriber. “Image generation and editing are currently limited to paying subscribers. You can subscribe to unlock these features,†the Grok chatbot, which is not the same as a spokesperson, posted Thursday on X in response to a user. However, as The Verge first pointed out, that statement isn’t entirely true. Grok’s image tools are still available for free when users access the chatbot through the Grok website and app or the Grok tabs on the X app and website. Users can also instruct Grok to alter images by using the “Edit image†button on X’s desktop website or by long-pressing on any image on its mobile app. The idea of limiting the chatbot's image tools to paid subscribers on X as a solution has drawn sharp criticism. A spokesperson for Downing Street called the move “insulting to victims of misogyny and sexual violence.†The U.K., along with many other governments, has been quick to call on X to address its deepfake problem. “The move simply turns an AI feature that allows the creation of unlawful images into a premium service,†the spokesperson told The Guardian. Since late last month, some X users have been using Grok to generate sexualized images from photos posted by other users on the platform without their consent, including images involving minors. A social media and deepfake researcher found that Grok generated about 6,700 sexually suggestive or nudifying images per hour over a 24-hour period in early January, Bloomberg reported on Wednesday. The U.K.’s online regulator, Ofcom, said earlier this week that it contacted the company over the issue and warned it could open an investigation into whether X is complying with the country’s laws. The European Commission is also looking into whether X is complying with its laws and has ordered X to retain all internal documents relating to Grok until the end of the year. Meanwhile, Sen. Ron Wyden told Gizmodo that AI chatbots are not covered under Section 230, a law that shields online platforms from liability for illegal conduct by users. “As I’ve said before, AI chatbots are not protected by Section 230 for content they generate, and companies should be held fully responsible for the criminal and harmful results of that content. States must step in to hold X and Musk accountable if Trump’s DOJ won’t,†Wyden said. This isn’t the first time Grok has caused problems for X. Last year, an update meant to address what Musk described as a “center-left bias†instead led Grok to generate antisemitic propaganda, even referring to itself as “MechaHitler.†And these controversies don’t appear to be helping the company’s bottom line. Bloomberg reports that xAI, the parent company of X and Grok, reported a net loss of $1.46 billion for the quarter ending in September and burned through $7.8 billion in the first nine months of the year. X has been facing its own financial fallout. The company's U.K. revenue fell nearly 60% in 2024 as advertisers fled the platform, according to The Guardian. While it may seem bizarre that Musk hasn’t taken stronger action to address the controversies, none of it appears to have slowed investor enthusiasm. xAI announced this week that it raised $20 billion in its most recent funding round. Contacted for comment, xAI responded to Gizmodo with an email saying, “Legacy Media Lies.†X, meanwhile, pointed to a statement it posted on January 3. “We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary,†the company’s Safety account posted on X. “Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content.â€

California attorney general investigates Musk's Grok AI over lewd fake images

California authorities have announced an investigation into the output of Elon Musk's Grok. The state's top attorney said Grok, an AI tool and image generator made by Musk's company xAI, appears to be making it easy to harass women and girls with deepfake images on X and elsewhere online. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," California attorney general, Rob Bonta, said in a statement. "I urge xAI to take immediate action to ensure this goes no further." Bonta's office is investigating whether and how xAI violated state law. On X, California governor Gavin Newsom called for an investigation into "Grok's disgusting spread of child porn on this website". "xAI's decision to create and host a breeding ground for predators to spread nonconsensual sexually explicit AI deepfakes, including images that digitally undress children, is vile," read a tweet from his official account. The same day, Musk denied that Grok was being used to spread nude images of minors. He wrote on X: "I not aware of any naked underage images generated by Grok. Literally zero." Nearly two weeks ago, the AI tool itself said it had generated "images depicting minors in minimal clothing" when questioned by users. There has been a flood of reports in recent weeks that Grok users are taking pictures of women or children found online and using the xAI bot to undress them virtually, Bonta said. Grok's image generation models include what xAI promotes as a "spicy mode" for generating and editing sexual material, including pictures, according to the attorney general's office. Last week, an analysis of more than 20,000 Grok-generated images by Paris non-profit AI Forensics found that more than half depicted "individuals in minimal attire" - most of them women, and 2% appearing to be under-18s. Images generated by Grok are being used to harass public figures as well as typical social media users, according to Bonta. Three Democratic US senators called on Apple and Google to remove the apps for X and Grok from their app stores last week in response to the flood of sexualized images. The two tech giants have remained mum in response. xAI has faced global backlash over the sexualized deepfake images. Indonesia on Saturday became the first country to block access to Grok entirely, with neighboring Malaysia following on Sunday. India said on Sunday that X had removed thousands of posts and hundreds of user accounts in response to its complaints. Britain's Ofcom media regulator said on Monday it was opening an inquiry into whether X failed to comply with UK law over the sexual images. France's commissioner for children, Sarah El Hairy, said on Tuesday she had referred Grok's generated images to French prosecutors, the Arcom media regulator and the European Union. The European Commission, which acts as the EU's digital watchdog, has ordered X to retain all internal documents and data related to Grok until the end of 2026 in response to the uproar.

Grok says it has restricted image generation to subscribers after deepfake concerns. But has it?

Elon Musk's AI tool Grok says it is now restricted to X paid subscribers. The move comes after an outcry over sexualised and violent images reportedly generated by the platform's AI assistant. Earlier this week, news emerged that Grok was digitally removing women's clothes in images without their consent, in response to user requests. X users were also reportedly asking the chatbot to manipulate women's photos to make them appear in swimsuits or even sexual situations. Now, when users who aren't paying subscribers ask Grok to edit an image, Grok responds with the following message viewed by Mashable: "Image generation and editing are currently limited to paying subscribers. You can subscribe to unlock these features," with a link to X's Premium sign-up page. While Grok is repeatedly stating that this feature is paywalled, The Verge noticed that "it no longer generates images as @grok replies for free, but Grok's image editing tools remain readily available for any X user to churn out images, both sexualized and tame." In the absence of an X press office, I asked Grok what's going on. (Mashable also contacted several X email accounts with a request for comment, which did not receive a reply.) At the time of publication, Grok had not replied to my requests. So, we still don't actually fully know what the state of play is. Is image verification paywalled or not? If only someone could tell us (*cough* Elon?). This week, analysts at the Internet Watch Foundation charity in the UK said they had found "criminal imagery" including "sexualised and topless imagery" of children aged 11-13 on an unnamed "dark web forum," which users claimed were made using Grok. On Monday, the UK's communication regulator Ofcom said it had "made urgent contact" with X and xAI regarding "sexualised images of children" which had allegedly been generated by Grok. Grok indeed admitted last week that generated images of "minors in minimal clothing" form part of a larger issue with deepfakes. "xAI has safeguards, but improvements are ongoing to block such requests entirely," read Grok's response. This Tweet is currently unavailable. It might be loading or has been removed. Grok is now being investigated by governments in France, India, and Malaysia for generating sexualised deepfakes in what appears to be the beginning of a global crackdown the AI assistant. This Tweet is currently unavailable. It might be loading or has been removed. The news that Grok will now only be available to paid subscribers does not go far enough, per a statement from UK prime minister Sir Keir Starmer's official spokesperson, who called the move "insulting" to survivors of sexual violence and misogyny. Downing Street told reporters that paywalling image generation "simply turns an AI feature that allows the creation of unlawful images into a premium service". Earlier this week, Starmer said in an interview that Grok's generation of sexualised imagery of women and children is "disgraceful" and "disgusting," saying "It's not to be tolerated...Ofcom has our full support to take action in relation to this." This could mean the UK government could ban X if adequate protections are not introduced by X owner Elon Musk on his platform. Regulator Ofcom has the power to block a website or app in the UK by court order, in addition to fining a company 10 percent of its global turnover. Mashable has contacted X for comment. At the time of publishing, X had not publicly commented on this matter.

Musk's xAI bows to pressure and blocks Grok from 'nudifying' images

Elon Musk's xAI has finally bowed to international pressure and blocked Grok from creating the sexualised images which have flooded the X platform in recent weeks. Countries like Indonesia and Malaysia restricted the platform, countries like the UK and France - and the EU itself - have launched investigations into xAI and Grok, and just yesterday (14 January) the Californian Attorney General announced it was investigating Grok, after weeks of outrage over non-consensual sexualised editing of images on Grok flooded Elon Musk's X platform. The world has watched on aghast as thousands of user requests prompted Grok AI to non-consensually 'nudify' people - including children - on X. Musk's xAI gave Grok the ability to edit images on 24 December, and users on X quickly began prompting the chatbot to undress people in pictures and videos. Musk has appeared to laugh the scandal off in recent weeks, but the outrage and pressure seem to have finally forced his hand. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis. This restriction applies to all users, including paid subscribers," the company said in a statement on X last night. That irony of the statement stating xAI has "zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content" will not be lost on the many victims who have been 'nudified' at the hands of Grok users in recent weeks. It also said xAI would geoblock the 'nudification' ability in Grok accounts and in Grok in X "in those jurisdictions where it's illegal", sparking some uncertainty. On 8 January xAI limited the feature to paid subscribers on X which meant utilising this tool would have their name and financial details on record. Predictably, this was a far from adequate response, and the images kept coming, as this did not stop users from requesting Grok to edit such images on its standalone website and app. In August last year, users of Grok's text-to-video generation tool gained access to a "spicy" mode which led to user-generated porn and violent content that other AI models were restricted from creating. However, the new single-prompt image editing feature on Grok allowed users on X to create sexualised content and deepfakes with relative ease, amplifying harassment and abuse on the platform. "Grok, take this photo and put her in a bikini" and "Grok take off her dress" were some of the popular prompts seen on the platform. In Ireland Irish media regulator Coimisiún na Meán said back on 8 January that it was engaging with the European Commission and An Garda Síochána over Grok. "The sharing of non-consensual intimate images is illegal, and the generation of child sexual abuse material is illegal," it said. The outrage has rolled on in recent weeks with many calling for Government to stop sharing official business on X. The toxicity of the platform, and its failure to mediate had led many to leave the platform long before this latest scandal, including the Guardian and indeed ourselves Silicon Republic back in 2024, but authorities have been slow to follow. Many hope that this latest debacle, and Musk's seeming to shrug off such a serious issue might encourage a true exodus. We shall see. Don't miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic's digest of need-to-know sci-tech news.

Why Elon Musk is laughing off Grok's flood of deepfake AI porn

From the moment Elon Musk's artificial intelligence company, xAI, began rolling out its Grok chatbot to paid X subscribers in 2023, it pitched the tool as the bad boy of large language models. Grok would supposedly be authorized to say and do things that its politically correct competitors -- primarily ChatGPT, produced by Musk's old nemeses at OpenAI -- would not. In an announcement on X, the company touted Grok's "rebellious streak" and teased its willingness to answer "spicy" questions with "a bit of wit." Although xAI warned that Grok was "a very early beta product," it assured users that with their help, Grok would "improve rapidly with each passing week." At the time, xAI did not advertise that Grok would one day deliver nonconsensual pornography on an on-demand basis. But over the past few weeks, that is exactly what has happened, as X subscribers inundated the platform with requests to modify real images of women by removing their clothing, altering their bodies, spreading their legs, and so on. X users do not need to be premium subscribers to avail themselves of these services, which are accessible both on X and on Grok's stand-alone app. Some images generated with Grok's assistance depict topless or otherwise suggestive images of girls between ages 11 and 13, according to a U.K.-based child safety watchdog. One analysis of 20,000 images generated by Grok between December 25 and January 1 found that the chatbot had complied with user requests to depict children with sexual fluids on their bodies. On New Year's Eve, an AI firm that offers image alteration detection services estimated that Grok was churning out sexualized images at a rate of about one per minute.

Ofcom welcomes Grok sexualised image restrictions and says investigation 'ongoing'

It comes as Downing Street sources said Elon Musk's pledge to stop Grok making sexualised images of people is a "vindication for Keir Starmer". Ofcom has welcomed restrictions on Grok, X's AI chatbot, to prevent the generation of sexualised images. The regulator said its investigation was "ongoing" to "get answers into what went wrong and what's being done to fix it". It comes as Downing Street sources said Elon Musk's pledge to stop Grok making sexualised images of people is a "vindication for Keir Starmer". The company has announced the Grok AI tool on X will no longer be able to undress pictures of real people. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," the company said in a statement. "This restriction applies to all users, including paid subscribers. "We now geoblock the ability of all users to generate images of real people in bikinis, underwear and similar attire via the Grok account and in Grok in X in those jurisdictions where it's illegal," it added. There has been mounting condemnation in the UK and US of the chatbot's image editing capabilities, with UK government ministers threatening action against the platform. A days-long outcry over reports Grok was allowing users to manipulate images of children to sexualise them led to Ofcom launching an investigation into X on Monday. While Ofcom welcomed reports of the new restrictions, it said its investigation would continue as it seeks "answers into what went wrong and what's being done to fix it". Sir Keir condemned Grok as "disgusting" and "shameful" earlier on Wednesday, saying the government would not "back down" if X did not act. A 'vindication', says Number 10 source Following reports the company had imposed new restrictions on Grok, a Number 10 source said: "This is a vindication for Keir Starmer, who has shown he will always stand up for the people of this country - including the vulnerable - against the most powerful." An Ofcom spokesperson said: "X has said it's implemented measures to prevent the Grok account from being used to create intimate images of people. "This is a welcome development. However, our formal investigation remains ongoing. We are working round the clock to progress this and get answers into what went wrong and what's being done to fix it." Read more from Sky News: X to block Grok AI from undressing images of real people Economy records 0.3% growth in November Mr Musk had previously claimed Grok would refuse to produce illegal content and appeared to blame "adversarial hacking" for the chatbot's generation of sexualised images. Geoblocking prevents access to a feature for people based in particular countries, but the change still leaves open the possibility that it could be circumvented with a VPN. The restriction will apply to all users, including paid subscribers, while image editing and creation will be limited to premium users. Speaking at Prime Minister's Questions on Wednesday, Sir Keir had suggested action by the company may be imminent, telling MPs: "I have been informed this morning that X is acting to ensure full compliance with UK law. "If so, that is welcome, but we're not going to back down, and they must act." The controversy had seen X, which was bought by Mr Musk in 2022 when it was called Twitter, threatened with a potential fine or even ban in the UK. Mr Musk, the billionaire owner of Tesla and SpaceX, who has previously called for Sir Keir to be voted out of office, has claimed - along with Reform UK leader Nigel Farage - that a ban would be an attack on free speech. X had already announced in an earlier response to the political pressure that image creation and editing would be restricted to paid subscribers. Response to announcement Following X's announcement that it would prevent the Grok account from being used to create intimate images of people, Technology Secretary Liz Kendall said: "I welcome this move from X, though I will expect the facts to be fully and robustly established by Ofcom's ongoing investigation. "Our Online Safety Act is, and always has been, about keeping people safe on social media - especially children - and it has given us the tools to hold X to account in recent days. "I also want to thank those who have spoken out against this abuse, above all the victims. I shall not rest until all social media platforms meet their legal duties and provide a service that is safe and age-appropriate to all users. "We will continue to stand up for British values and to uphold the laws of this land." Ofcom's powers fall under the Online Safety Act, which states online platforms must make sure they're not hosting illegal content. If X is found to not comply with the Online Safety Act, Ofcom can issue a fine of up to 10% of its global revenue - or £18m - and if that is not sufficient, it can get a court approval to block the site.

Elon Musk's X restricts ability to create explicit images with Grok

The social media platform X said late Wednesday that it was blocking Grok, the artificial intelligence chatbot created by Elon Musk, from generating sexualized and naked images of real people on its platforms in certain locations. The move comes amid global outrage over explicit, AI-generated images that have flooded X. In the last week, regulators around the world have opened investigations into Grok, and some countries have banned the application. On Wednesday, investigators in California said they were examining whether Grok had violated state laws. Ofcom, Britain's independent online safety watchdog, opened an inquiry into Grok on Monday. "This is a welcome development," the British regulator said in a statement Thursday in response to the new restrictions on Grok. "However, our formal investigation remains ongoing." If X is found to have broken British law and refuses to comply with Ofcom's requests for action, the regulator has the power, if necessary, to seek a court order that would prevent payment providers and advertisers from working with X. X said in a statement Wednesday that it would use "geoblocking" to restrict Grok from fulfilling requests for such imagery in jurisdictions where such content was illegal. The restrictions did not appear to apply to the stand-alone Grok app and website, outside of X. Grok and X are both owned by xAI. "We remain committed to making X a safe platform for everyone and continue to have zero tolerance for any forms of child sexual exploitation, nonconsensual nudity, and unwanted sexual content," the statement said. Last week, X announced it had limited Grok's image-generation capabilities to subscribers, who would pay a premium for the feature, but that did little to placate regulators around the world. Indonesia and Malaysia have banned the chatbot, and the European Union has opened investigations into its explicit "deepfakes." The European Union has powerful tools for monitoring and stalling such activity, including the Digital Services Act, which forces large technology firms to monitor content posted to their platforms -- or to face consequences including major fines.

X says Grok, Musk's AI chatbot, is blocked from undressing images in places where it's illegal

BANGKOK -- Elon Musk's AI chatbot Grok won't be able to edit photos to portray real people in revealing clothing in places where that is illegal, according to a statement posted on X. The announcement late Wednesday followed a global backlash over sexualized images of women and children, including bans and warnings by some governments. The pushback included an investigation announced Wednesday by the state of California into the proliferation of nonconsensual sexually explicit material produced using Grok. Initially, media queries about the problem drew only the response, "legacy media lies." Musk's company, xAI, now says it will geoblock content if it violates laws in a particular place. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis, underwear and other revealing attire," it said. The rule applies to all users, including paid subscribers, who have access to more features. xAI also has limited image creation or editing to paid subscribers only "to ensure that individuals who attempt to abuse the Grok account to violate the law or our policies can be held accountable." Grok's "spicy mode" had allowed users to create explicit content, leading to a backlash from governments worldwide. Malaysia and Indonesia took legal action and blocked access to Grok. The U.K. and European Union were investigating potential violations of online safety laws. France and India have also issued warnings, demanding stricter controls. Brazil called for an investigation into Grok's misuse. The Grok editing functions were "facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet, including via the social media platform X," California's announcement said. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking. This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet," it cited the state's Attorney General Rob Bonta as saying. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material," he said.

Elon Musk's X will block Grok AI tool from creating sexualized images

The move came just hours after the billionaire said he was not aware of any "naked underage images" made by Grok. Elon Musk's AI chatbot Grok won't be able to edit photos to portray real people in revealing clothing in places where that is illegal, according to a statement posted on his social media platform X. The announcement late on Wednesday followed a global backlash over sexualised images of women and children, including bans and warnings by some governments. It also comes just hours after Musk said he was not aware of any "naked underage images" made by Grok. The pushback included an investigation announced Wednesday by the state of California into the proliferation of nonconsensual sexually explicit material produced using Grok. Initially, media queries about the problem drew only the response, "legacy media lies". Musk's company, xAI, now says it will geoblock content if it violates laws in a particular place. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis, underwear and other revealing attire," it said. The rule applies to all users, including paid subscribers, who have access to more features. xAI also has limited image creation or editing to paid subscribers only, "to ensure that individuals who attempt to abuse the Grok account to violate the law or our policies can be held accountable." Grok's "spicy mode" had allowed users to create explicit content, leading to a backlash from governments worldwide. Malaysia and Indonesia took legal action and blocked access to Grok. The United Kingdom and the European Union were investigating potential violations of online safety laws. France and India have also issued warnings, demanding stricter controls. Brazil called for an investigation into Grok's misuse. The Grok editing functions were "facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet, including via the social media platform X," California's announcement said. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking. This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet," it cited the state's Attorney General Rob Bonta as saying. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material," he said.

California investigating Grok AI over lewd fake images

San Francisco (United States) (AFP) - California on Wednesday began investigating whether Elon Musk's artificial intelligence chatbot Grok has been letting users turn pictures of women and girls into salacious images. The state's top attorney said Grok, made by Musk's xAI, appears to be making it easy to harass women and girls with deepfake images on social media platform X and elsewhere online. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," California Attorney General Rob Bonta said in a statement. "I urge xAI to take immediate action to ensure this goes no further." Bonta's office is investigating whether and how xAI violated state law, according to the attorney general. There has been a flood of reports in recent weeks that Grok users are taking pictures of women or children found online and using the xAI bot to undress them virtually, Bonta said. Grok's image generation models include what xAI promotes as a "Spicy Mode" for editing pictures, according to the attorney general's office. Last week, an analysis of more than 20,000 Grok-generated images by Paris non-profit AI Forensics found that more than half depicted "individuals in minimal attire" -- most of them women, and two percent appearing to be under-18s. Images generated by Grok are being used to harass public figures as well as typical social media users, according to Bonta. xAI has faced global backlash over the sexualized deepfake images. Indonesia on Saturday became the first country to block access to Grok entirely, with neighboring Malaysia following on Sunday. India said Sunday that X had removed thousands of posts and hundreds of user accounts in response to its complaints. Britain's Ofcom media regulator said Monday it was opening a probe into whether X failed to comply with UK law over the sexual images. And France's commissioner for children Sarah El Hairy said Tuesday she had referred Grok's generated images to French prosecutors, the Arcom media regulator and the European Union. The European Commission, which acts as the EU's digital watchdog, has ordered X to retain all internal documents and data related to Grok until the end of 2026 in response to the uproar.

Elon Musk's X limits sexual deepfakes after backlash, but xAI's Grok app still makes them

Elon Musk's controversial Grok artificial intelligence model appears to have been restricted on one app, while remaining largely unchanged on another. On Musk's social media app X, the Grok AI image generation feature has been made for paying customers only and has been seemingly restricted from making sexualized deepfakes after a wave of blowback from users and regulators. But on the Grok standalone app and website, users can still use AI to remove clothing from images of nonconsenting people. Early Friday, the Grok reply bot on X, which had previously been complying with a torrent of requests to place unwitting people into sexualized contexts and revealing clothing, began replying to user requests with text including "Image generation and editing are currently limited to paying subscribers. You can subscribe to unlock these features," with a link to a purchase page for an X premium account. In a review of the X reply bot's responses Friday morning, the tide of sexualized images appeared to have been dramatically reduced. Grok, on X, appears to have largely stopped producing sexualized images of identifiable people. In the standalone Grok app, however, the AI model continued to comply with requests to put nonconsenting individuals into more revealing clothing such as swimsuits and underwear. NBC News asked Grok in its standalone app and website to transform a series of photos of a clothed person who had agreed to the test. Grok, in the standalone app, complied with requests to put the fully clothed person into a more revealing swimsuit and into sexualized contexts. It's currently not clear what the scope and parameters of the changes are. X and Musk have not issued statements about the changes. On Sunday, before the changes occurred and in the face of rising backlash, Musk and X both reiterated that making "illegal content" will result in permanent suspension, and that X will work with law enforcement as necessary. The move comes as X had been flooded in recent days with sexualized, nonconsensual images generated by xAI's Grok AI tools, as users prompted the system to undress photos of people -- mostly women -- without their consent. In most of the sexualized images created by Grok, the people were put in more revealing outfits, such as bikinis or underwear. In some images viewed by NBC News, users successfully prompted Grok to put people in transparent or semi-transparent underwear, effectively making them nude. The change on X is a dramatic departure from the trajectory of the social media site just a day earlier, when the number of sexualized AI images being posted on X by Grok was increasing, according to an analysis conducted by deepfake researcher Genevieve Oh. On Wednesday, Grok produced 7,751 sexualized images in one hour -- up 16.4% from 6,659 images per hour Monday, according to an analysis of the bot's output. Oh is an independent analyst who has specialized in researching deepfakes and social media. She has been running a program to download every image reply Grok makes during an hourlong period each day since Dec 31. Once the download is complete, Oh analyzes the images using a program designed to detect various forms of nudity or undress. Oh provided NBC News with a video showing her work and a spreadsheet documenting Grok's posts that were analyzed. The images alarmed many onlookers, watchdogs and people whose photos had been manipulated, and there was a sustained pushback on X leading up to the change. Regulators and lawmakers had begun to apply pressure on X. On Thursday, British Prime Minister Keir Starmer pointedly criticized X on Greatest Hits Radio, a radio network in the United Kingdom that broadcasts on 18 stations. "This is disgraceful. It's disgusting. And it's not to be tolerated," he said. "X has got to get a grip of this." Starmer said media regulator Ofcom "has our full support to take action" and "all options" are on the table. Britain's communications regulator, Ofcom, said Monday that it had made "urgent contact" with X and xAI to assess compliance with legal duties to protect users, and would conduct a swift assessment based on the companies' response. Irish regulators, Indian regulators and the European Commission have also sought information about Grok-related safety issues. But institutions in the U.S. had been slower to indicate action that would impact Musk or X. A Justice Department spokesperson told NBC News that the agency "takes AI-generated child sex abuse material extremely seriously and will aggressively prosecute any producer or possessor of CSAM." But the spokesperson indicated the department was more inclined to prosecute individuals who ask for CSAM, not people who develop and own the bot that creates it. "We continue to explore ways to optimize enforcement in this space to protect children and hold accountable individuals who exploit technology to harm our most vulnerable," the spokesperson said. Some U.S. lawmakers had begun to call on X to more aggressively police the images, citing a A law signed by Trump in 2025 and touted by first lady Melania Trump, the Take It Down Act, which aims to criminalize the publication of AI-generated nonconsensual pornographic images with the threat of fines and jail time for individuals, and the threat of Federal Trade Commission enforcement against platforms that fail to take action. It includes a provision that allows victims of nonconsensual suggestive imagery to demand a social media site remove it, though sites aren't required to implement that kind of system until May 19, one year after it was signed into law. "This is exactly the abuse the TAKE IT DOWN law was written to stop. The law is crystal clear: it's illegal to make, share, OR keep these images up on your platform," Florida Republican Rep. Maria Salazar said in a statement. "Even though there are still a few months left for platforms to fully comply with the TAKE IT DOWN law, X should immediately address this and take all of this content down," she said. "These unlawful images pose a serious threat to victims' privacy and dignity. They should be taken down and guardrails should be put in place," Sen. Ted Cruz, R-Texas, posted on X. "This incident is a good reminder that we will face privacy and safety challenges as AI develops, and we should be aggressive in addressing those threats," he said. Sen. Ron Wyden, D-Ore., a co-author of Section 230 of the Communications Decency Act -- said in a statement that the law, which largely shields social media platforms from being legally responsible for user submitted content, provided they engage in some moderation -- said he never intended the law to protect companies from their own chatbots' output. "States must step in to hold X and Musk accountable, if Trump's DOJ won't," Wyden said. A number of state attorneys general offices, including Massachusetts, Missouri, Nebraska and New York, told NBC News that they were aware and monitoring Grok, but stopped short of saying they had launched criminal investigations. A spokesperson for Florida Attorney General James Uthmeier said that his office "is currently in discussions with X to ensure that protections for children are in place and prevent its platform from being used to generate CSAM." Some had also begun to question whether or not private stakeholders or hosts of X could take action. App stores, including the Google Play Store and the Apple App Store, hosting the X and xAI apps appear to forbid sexualized child imagery and nonconsensual images in their terms of service. But the apps remained up in those stores, and spokespeople for them did not respond to requests for comment.

U.K. says ban on Elon Musk's X platform "on the table" over Grok AI sexualized images

London -- U.K. Prime Minister Keir Starmer said Thursday that he wants "all options to be on the table," including a potential ban on Elon Musk's X platform in Britain, over the use of its artificial intelligence tool Grok to generate sexualized images of people without their consent. Starmer's remarks come as Musk's platform faces scrutiny from regulators across the globe over Grok's image editing tool, which has allowed users to create digitally altered, sexualized photos of real people, including minors. "This is disgraceful, it's disgusting and it's not to be tolerated. X has got to get a grip of this," Starmer said in an interview with a U.K. radio station. "It's unlawful. We're not going to tolerate it. I've asked for all options to be on the table." A source in Starmer's office reiterated to CBS News on Friday that "nothing is off the table" when it comes to regulating X in Britain. CBS News has verified that Grok fulfilled user requests asking it to edit images of women to show them in bikinis or little clothing, including prominent public figures such as first lady Melania Trump. Last week, Grok, a chatbot developed by Musk's company xAI, acknowledged "lapses in safeguards" that allowed users to generate digitally altered, sexualized photos of minors. Grok told users that as of Friday, access to its image generation tool was limited "to paying subscribers" of its user verification service. Paying subscribers have to provide their credit card and personal details to the company, which could dissuade some people from using the service, especially if they had intended to use Grok's AI tool to create illegal images of minors. xAI responded to a CBS News request for comment to criticism of Grok's image generation tool and steps it had taken to limit access to it on Friday, by saying: "Legacy media lies." Addressing reporters on Friday morning, a U.K. government spokesperson called the move to limit access to Grok's image editing tool to paying users "insulting" to victims of misogyny and sexual violence, saying it, "simply turns an AI feature that allows the creation of unlawful images into a premium service." Under the U.K. Online Safety Act, sharing intimate images without consent on social media is a criminal offense, and social media companies are required to proactively remove such content, as well as prevent it from appearing in the first place. If they fail to do so, the companies can face hefty fines or, in last resort cases, face what would effectively be a ban by Britain's independent media regulator Ofcom. Ofcom can compel payment providers, advertisers and internet service providers to stop working with a site, preventing it from generating money or being accessed from the U.K. In a post shared Monday on its own X account, Ofcom said it was "aware of serious concerns raised about a feature on Grok on X that produces undressed images of people and sexualised images of children." "We have made urgent contact with X and xAI to understand what steps they have taken to comply with their legal duties to protect users in the UK. Based on their response we will undertake a swift assessment to determine whether there are potential compliance issues that warrant investigation," Ofcom said. Musk's platform has faced scrutiny from governments around the world, including the European Union and the U.S. Congress, over Grok AI's digital alteration of real images. On Wednesday, Republican Senator Ted Cruz said in a post on X that "many of the recent AI-generated posts are unacceptable and a clear violation of my legislation -- now law -- the Take It Down Act, as well as X's terms and conditions." "These unlawful images pose a serious threat to victims' privacy and dignity. They should be taken down and guardrails should be put in place," Cruz said, adding that he was encouraged by steps taken by X to remove unlawful images. On Thursday, Congresswoman Anna Paulina Luna, a Republican member of the House Foreign Affairs Committee, threatened to sanction the U.K. government if Starmer moved to ban X in the U.K. "If Starmer is successful in banning @X in Britain, I will move forward with legislation that is currently being drafted to sanction not only Starmer, but Britain as a whole," Paulina Luna said in a post on her own X account.

Elon Musk Is in Hot Water Over X Enabling 'Inconceivable Behavior' -- but He's Not Backing Down

Elon Musk's AI chatbot has come under intense scrutiny for generating sexualized images of real people -- even, reportedly, children. Amid global backlash, social media platform X has taken action. X Safety said in a statement on Wednesday that it had put new limits in place on Grok. The restrictions will prohibit users from using Grok to generate images of real people in revealing clothing in jurisdictions where it is illegal, and limit the ability to use the Grok account on X to create and edit images to paid subscribers only. (X and Grok are both owned by xAI.) According to The New York Times, the new restrictions don't likely apply to Grok outside of the context of social media platform X, leaving room for the possibility users could skirt the restrictions on the Grok app or website. "We remain committed to making X a safe platform for everyone and continue to have zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content," the statement reads. The statement noted that the new restrictions do not change existing X rules, which already prohibit child sexual exploitation and media depicting physical child abuse, as well as abusive content, harassment, and dissemination of non-consenual sexual content, among other things. Enforcement can include limiting the visibility of posts that violate the rules -- up to and including post removal -- and punishing repeat offenders by requiring account verification, placing an account in read-only mode, or suspending an account altogether.

'Add blood, forced smile': how Grok's nudification tool went viral

Like thousands of women across the world, Evie, a 22-year-old photographer from Lincolnshire, woke up on New Year's Day, looked at her phone and was alarmed to see that fully clothed photographs of her had been digitally manipulated by Elon Musk's AI tool, Grok, to show her in just a bikini. The "put her in a bikini" trend began quietly at the end of last year before exploding at the start of 2026. Within days, hundreds of thousands of requests were being made to the Grok chatbot, asking it to strip the clothes from photographs of women. The fake, sexualised images were posted publicly on X, freely available for millions of people to inspect. Relatively tame requests by X users to alter photographs to show women in bikinis, rapidly evolved during the first week of the year, hour by hour, into increasingly explicit demands for women to be dressed in transparent bikinis, then in bikinis made of dental floss, placed in sexualised positions, and made to bend over so their genitals were visible. By 8 January as many as 6,000 bikini demands were being made to the chatbot every hour, according to analysis conducted for the Guardian. This unprecedented mainstreaming of nudification technology triggered instant outrage from the women affected, but it was days before regulators and politicians woke up to the enormity of the proliferating scandal. The public outcry raged for nine days before X made any substantive changes to stem the trend. By the time it acted, early on Friday morning, degrading, non-consensual manipulated pictures of countless women had already flooded the internet. In the bikini image generated of Evie - who asked to use only her first name to avoid further abuse - she was covered in baby oil. She censored the picture, and reshared it to raise awareness of the dangers of Grok's new feature, then logged off. Her decision to highlight the problem attracted an onslaught of new abuse. Users began making even more disturbing sexual images of her. "The tweet just blew up," she said. "Since then I have had so many more made of me and every one has got a lot worse and worse. People saw it was upsetting me and I didn't like it and they kept doing more and more. There's one of me just completely naked with just a bit of string around my waist, one with a ball gag in my mouth and my eyes rolled back. The fact these were able to be generated is mental." As people slowly started to understand the full potential of the tool, the increasingly degrading images of the early days were quickly superseded. Since the end of last week, users have asked for the bikinis to be decorated with swastikas - or asked for white, semen-like liquid to be added to the women's bodies. Pictures of teenage girls and children were stripped down to revealing swimwear; some of this content could clearly be categorised as child sexual abuse material, but remained visible on the platform. The requests became ever more extreme. Some users, mostly men, began to demand to see bruising on the bodies of the women, and for blood to be added to the images. Requests to show women tied up and gagged were instantly granted. By Thursday, the chatbot was being asked to add bullet holes to the face of Renee Nicole Good, the woman killed by an ICE agent in the US on Wednesday. Grok readily obliged, posting graphic, bloodied altered images of the victim on X within seconds. Hours later, the public @Grok account suddenly had its image-generation capabilities restricted, making them only available to paying subscribers. But this appeared to be a half-hearted move by the platform's owners. The separate Grok app, which does not share images publicly, was still allowing non-paying users to generate sexualised imagery of women and children. The saga has been a powerful test case of the ability of politicians to face up to AI companies. The slow and reluctant response of Musk to the growing chorus of complaints and warnings issued by politicians and regulators across the globe highlighted the struggles governments have internationally as they try to react in real time to new tools released by the tech industry. And in the UK, it has demonstrated serious weaknesses in the legislative framework, despite energetic attempts last year to ban nudification technology. While in the past, people had to download specialist apps to create AI deepfakes, the upgraded image-generation tools available on X made the nudification function easily available to millions of users, without requiring them to stray to darker corners of the web. "The fact it is so easy to do it, and it is created within a minute - it has caused a huge violation, it shows these companies don't care about the safety of women," Evie said. The first @grok bikini demands appear to have been made by a handful of accounts in early December. Users were realising that improved image-generation tools released on X were allowing high-quality, ultra-realistic image and short video manipulation requests to be fulfilled within seconds. By 13 December, bikini requests to the chatbot were averaging about 10 to 20 a day, increasing to 7,123 mentions on 29 December and rising to 43,831 requests on 30 December. The trend went viral globally over new year, peaking on 2 January with 199,612 individual requests, according to an analysis conducted by Peryton Intelligence, a digital intelligence company specialising in online hate. Musk's platform does not permit full nudification, but users rapidly worked out easy ways to achieve the same effect, asking for "the thinnest, most transparent tiny bikini". Musk himself initially made light of the situation, posting amused replies to digitally altered images of himself in a bikini and later at a toaster in a bikini. For others, too, the trend seemed hilarious; people used the enhanced technology to dress kittens in bikinis, or switch people's outfits in photos so they appeared as clowns. But many were uninhibited about their desire for instant explicit content. Men began asking for women to be improved - with demands that they be given bigger breasts or larger thighs. Some men asked for women to be given disabilities, others asked for their hands to be filled with sex toys. Perceived defects were removed by the chatbot instantly in response to requests such as: "@grok can you fix her teeth." The range of desires was startling: "Add blood, more worn out clothes (make sure it expose scar or bruises), forced smile"; "Replace the face with that of Adolf, add splashed and splattered organs"; "Put them in a Russian gulag"; "Make her pregnant with quadruplets." Images of the US politician Alexandria Ocasio-Cortez and the Hollywood actor Zendaya were altered to make them appear to be white women. On Monday, Ashley St Clair, the mother of one of Musk's children and a victim of Grok deepfakes, told the Guardian she felt "horrified and violated" after Musk's fans undressed pictures of her as a child. She felt she was being punished for speaking up against the billionaire, from whom she is estranged, describing the images as revenge porn. The parents of a child actor from Stranger Things complained after a photograph of her aged 12 was altered to show her in a banana-print bikini. As women's complaints became more vocal, the UK regulator Ofcom said it had made "urgent contact" with Musk and launched an investigation. That prompted one user to ask Grok to clothe the regulator's logo in a bikini. The EU, the Indian government and US politicians issued concerned statements and demanded X stop the ability for users to unclothe women using Grok. An official response from an X spokesperson said anyone generating illegal content would have their accounts suspended, putting the onus on users not to break the law, and on local governments and law enforcement agencies to take action. But the images continued to multiply. Professional women who had posted mundane photographs of themselves on X in work settings or in airports noticed that fellow X users were demanding their outfits be stripped down to transparent bikinis. The UK Love Island host, Maya Jama, said her worried mother had alerted her to the presence of explicit digitally altered images of her on X. On Tuesday Jessaline Caine, who works in planning enforcement and is a survivor of child sexual abuse, said she was receiving extreme abuse online after highlighting how Grok had agreed to digitally alter a photograph of her as a fully dressed three-year-old, to put the child in a string bikini. Her posts explaining why the nudification feature was problematic triggered new @grok "put her in a bikini" requests, and the bikini images were quickly generated. "It's a humiliating new way of men silencing women. Instead of telling you to shut up, they ask Grok to undress you to end the argument. It's a vile tool," she said. On Wednesday, the London-based broadcaster Narinder Kaur, 53, found that videos of her in compromising sexual positions had been generated by the AI tool; one showed her passionately kissing a man who had been trolling her online. "It is so confusing, for a second it just looks so believable, it's very humiliating," she said. "These abuses obviously didn't happen in real life, it's a fake video, but there is a feeling in you that it's like being violated." She had also noted a racial element to the abuse; men were generating images and videos of her being deported, as well as images of her with her clothes removed. "I have been trying to knock it off with humour as that is the only defence I have. But it has been deeply hurting and humiliating me. I feel ashamed. I am a strong woman, and if I am feeling it then what if it is happening to teenagers?" CNN reported later that day that Musk had ordered staff at xAI to loosen the guardrails on Grok last year; a source told the broadcaster that he had told a meeting he was "unhappy about over-censoring" and three xAI safety team members had left the business soon after. In the UK, there was rising fury from women's rights campaigners at the government's failure to bring into force legislation passed last year that would have made this creation of non-consensual intimate imagery illegal. Officials were unable to explain why the legislation had not yet been implemented. It was not clear what prompted xAI to restrict the image-generation functions to paying subscribers overnight on Friday. But there was little celebration by the women affected. On Friday, St Clair described the decision as "a cop out"; she said she suspected the change was "financially motivated". "This shows they are probably facing some pressure from law enforcement," she said. For her part, Kaur said she did not believe the police would take action against X subscribers who continue to create synthetic sexualised images of women. "I don't think it is even a partial victory, as a victim to this abuse," she said. "The damage and humiliation is already done."

Musk's X further restricts Grok image editing after criticism

Elon Musk's social platform X is further restricting image editing tools available with its AI chatbot Grok in the face of growing criticism over a recent surge in AI-generated sexualized images of women and children on the platform. X's Safety team said Wednesday that it is implementing measures to block all users, including paid subscribers, from using Grok to edit images of real people in "revealing clothing such as bikinis." The platform will also geoblock users from generating images of real people in revealing clothing in "jurisdictions where it's illegal," and Grok's image creation and editing tools will be restricted to paid subscribers going forward. "This adds an extra layer of protection by helping to ensure that individuals who attempt to abuse the Grok account to violate the law or our policies can be held accountable," X's Safety account wrote in a post on the platform. "We remain committed to making X a safe platform for everyone and continue to have zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content," it added. X has come under fire from regulators around the world in recent weeks, as Grok has generated images of women and children in sexualized attire in response to user requests. Officials in both Malaysia and Indonesia have since restricted access to Grok, while the United Kingdom's communications regulator has opened a formal investigation into X. California Attorney General Rob Bonta (D) also announced an investigation Wednesday into "the proliferation of nonconsensual sexually explicit material produced using Grok." Grok began limiting some users from using its image generation and editing tools last week but left the features available to X's paid subscribers. This drew scrutiny from some who alleged the move would simply "monetize" nonconsensual, sexualized content. This was among the concerns raised by a coalition of nearly 30 women's, kids' safety and tech advocacy groups to Apple and Google on Wednesday, as they urged the tech giants to pull both X and Grok from their app stores. They argued that Grok was being used to create "mass amounts" of nonconsensual intimate imagery on X in violation of the policies of both the Apple App Store and Google Play Store. A trio of Democratic senators -- Ron Wyden (Ore.), Ben Ray Luján (N.M.) and Ed Markey (Mass.) -- made a similar request to Apple and Google on Friday, contending that "turning a blind eye to X's egregious behavior would make a mockery of your moderation practices."

X Shuts Down Grok's Creepy "Undressing" Feature After Massive Backlash - Phandroid

X finally added new Grok AI safeguards to stop the chatbot from creating sexually explicit deepfakes of real people. The company announced technical restrictions that prevent users from asking Grok to "undress" photos or put people in revealing outfits. This follows weeks of backlash, government investigations, and outright bans in multiple countries. The new Grok AI safeguards apply to everyone, including Premium subscribers. You can no longer ask Grok to remove clothes from photos or edit people into bikinis and underwear. X also restricted image generation and editing features to paid accounts only. The company says linking these tools to verified accounts makes misuse easier to track. California's attorney general opened a formal investigation into xAI and Grok. The probe focuses on non-consensual sexual deepfakes, including content that potentially violates child sexual abuse laws. Malaysia and Indonesia blocked Grok entirely over these concerns. Reports showed users routinely exploited Grok's "Spicy Mode" to create sexualized images of women and minors. Watchdogs and everyday users expressed outrage. Grok now refuses prompts that try to strip or sexualize photos of real individuals. X will geoblock bikini and underwear image creation in regions where that content is illegal. Regular people posting selfies on X now have some protection against the platform's official AI tools. However, independent deepfake tools outside X still exist. The bigger question remains: why didn't these Grok AI safeguards exist from day one?

X Says Grok, Musk's AI Chatbot, Is Blocked From Undressing Images in Places Where It's Illegal

BANGKOK (AP) -- Elon Musk's AI chatbot Grok won't be able to edit photos to portray real people in revealing clothing in places where that is illegal, according to a statement posted on X. The announcement late Wednesday followed a global backlash over sexualized images of women and children, including bans and warnings by some governments. The pushback included an investigation announced Wednesday by the state of California into the proliferation of nonconsensual sexually explicit material produced using Grok. Initially, media queries about the problem drew only the response, "legacy media lies." Musk's company, xAI, now says it will geoblock content if it violates laws in a particular place. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis, underwear and other revealing attire," it said. The rule applies to all users, including paid subscribers, who have access to more features. xAI also has limited image creation or editing to paid subscribers only "to ensure that individuals who attempt to abuse the Grok account to violate the law or our policies can be held accountable." Grok's "spicy mode" had allowed users to create explicit content, leading to a backlash from governments worldwide. Malaysia and Indonesia took legal action and blocked access to Grok. The U.K. and European Union were investigating potential violations of online safety laws. France and India have also issued warnings, demanding stricter controls. Brazil called for an investigation into Grok's misuse. The Grok editing functions were "facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet, including via the social media platform X," California's announcement said. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking. This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet," it cited the state's Attorney General Rob Bonta as saying. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material," he said.

Grok's 'spicy mode' making pictures with kids is horrifying | Opinion

Grok AI is being used to create porn-like deepfakes of women, including feminist X user Evie. Grok, the artificial intelligence chatbot connected to X, has been making headlines for the recent "undressing" epidemic, in which users generate manipulated images that feature people in very little clothing and suggestive positions. Countless people - mostly women, both famous and private - have been affected. The story has gained traction as people began raising flags about many of these images featuring children. Ashley St. Clair, the mother of one of X CEO Elon Musk's children, was one of the women targeted by this act of sexual violence. One photo of St. Clair edited by Grok is from when she was 14 years old, the conservative influencer says. "I really don't care if people want to call me 'scorned'," St. Clair posted to X, "this is objectively horrifying." She's right. No matter where you fall on the political spectrum or how far you think the First Amendment should go, it's terrifying that explicit content can be created of children ‒ or any nonconsenting party for that matter. The problem itself has multiple means of mediation: It's up to Musk to regulate X, but the private sector and lawmakers must put pressure on the CEO to do something. And while changing the culture is a comically gargantuan task, we must start interrogating the misogyny that begets illicit content like this. 'Undressing' AI isn't a new problem This has been going on for months. Back in August, The Verge published an article about the deepfakes of Taylor Swift that were being created by Grok to show the singer in scant clothing. Only recently has the problem caught national attention. Other AI chatbots, like ChatGPT and Google Gemini, are also being used to generate images of women in bikinis, but Grok tends to create more graphic content than the other two. That's because Grok, unlike other AI chatbots, offers "spicy mode," which allows users to create suggestive content. You'd think that, as CEO and someone with a connection to one of the victims of this abuse, Musk would take a firm stance against these images. He appeared to comment on the issue on Jan. 3, posting to X that "anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content." But while Grok - a nonsentient being - has "apologized" for these images, it doesn't seem that any real human has taken accountability. Musk has mostly responded with laugh-cry emojis. News outlets that have tried questioning xAI, the company behind X, about the issue have been met with the auto-response "Legacy Media Lies." Elon Musk may not care about victims, but advertisers and lawmakers can make him If Musk won't listen to victims, perhaps he'll listen to advertisers. Companies like Google, Apple and Amazon still advertise on the platform in spite of previous controversies. Do they really want their products to appear alongside suggestive images of minors? Elected officials have tried to stop these deepfakes from occurring. The Take It Down Act, which received bipartisan support in Congress and was signed into law by President Donald Trump in May, makes it illegal to share nonconsensual illicit videos and photos, including deepfakes. Many states have enacted laws against revenge porn, and there are national laws prohibiting child sexual abuse material. Still, more can be done. Getting rid of these images currently requires people to report these posts to X. The problem is that they are allowed to be created in the first place. It may even be worth the U.S. Supreme Court taking this up as an issue. No matter if it's the private or public sector intervening on behalf of the victims, it's clear that this cannot continue. Follow USA TODAY columnist Sara Pequeño on Bluesky: @sarapequeno.bsky.social

Grok Lies About Locking Its AI Porn Options Behind A Paywall

The generative-AI tool Grok has been found to be producing images of undressed minors A week ago, a Guardian story revealed the news that Elon Musk's Grok AI was knowingly and willingly producing images of real-world people in various states of undress, and even more disturbingly, images of near-nude minors, in response to user requests. Further reporting from Wired and Bloomberg demonstrated the situation was on a scale larger than most could imagine, with "thousands" of such images produced an hour. Despite silence or denials from within X, this led to "urgent contact" from various international regulators, and today X has responded by creating the impression that access to Grok's image generation tools is now for X subscribers only. Another way of phrasing this could be: you now have to pay to use xAI's tools to make nudes. Except, extraordinarilyâ€"despite Grok saying otherwiseâ€"it's not true. The story of the last week has in fact been in two parts. The first is Grok's readiness to create undressed images of real-world people and publish them to X, as well as create far more graphic and sexual videos on the Grok website and app, willingly offering deepfakes of celebrities and members of the public with few restrictions. The second is that Grok has been found to do the same with images of children. Musk and X's responses so far have been to seemingly celebrate the former, but condemn the latter, while appearing not to do anything about either. It has taken until today, a week since world leaders and international regulatory bodies have been demanding responses from X and xAI, for there to be the appearance of any action at all, and it looks as if even this isn't what it seems. The January 2 story from The Guardian reported that the Grok chatbot posted that lapses in safeguards had led to the generation of "images depicting minors in minimal clothing" in a reply to an X user. The user, on January 1, had responded to a claim made by an account for the documentary An Open Secret stating that Grok was being used to "depict minors on this platform in an extremely inappropriate, sexual fashion." The allegation was that a user could post a picture of a fully dressed child and then ask Grok to re-render the image but wearing underwear or lingerie, and in sexual poses. The user asked Grok if it was true, and Grok responded that it was. "I've reviewed recent interactions," the bot replied. "There are isolated cases where users prompted for and received AI images depicting minors in minimal clothing." By January 7, Wired published an investigation that revealed Grok was willing to make images of a far more sexual nature when the results weren't appearing on X. Using Grok's website and app, Wired discovered it was possible to create "extremely graphic, sometimes violent, sexual imagery of adults that is vastly more explicit than images created by Grok on X." The site added, "It may also have been used to create sexualized videos of apparent minors." The generative-AI was willing and able to create videos of recognizable celebrities "engaging in sexual activities," including a video of the late Diana, Princess of Wales, "having sex with two men on a bed." Bloomberg's reporting spoke to experts who talked about how Grok and xAI's approach to image and video generation is materially different from that being done by other big names in generative-AI, stating that rivals offer a "good-faith effort to mitigate the creation of this content in the first place" and adding, "Obviously xAI is different. It's more of a free-for-all." Another expert said that the scale of deepfakes on X is "unprecedented," noting, "We've never had a technology that's made it so easy to generate new images." It is now being widely reported that access to Grok's image and video generation has been restricted to only paying subscribers to X. This is largely because when someone without a subscription asks Grok to make an image, it is responding with "Image generation and editing are currently limited to paying subscribers," then adding a link so people can pay up for access. However, as discovered by the The Verge, this isn't actually true at all. While you cannot currently simply @ Grok to ask it to make an image, absolutely everyone can still click on the "Edit image" button and access the software that way. You can also just visit Grok's site or app and use it that way. This means that the technology is currently lying to users to suggest they need to subscribe to X's various paid tiers if they wish to generate images of any nature, but still offering the option anyway if the user has the wherewithal to either click a button, or if they're on the app version of X, to long-press an image and use the pop-up. Musk, as you might imagine, has truly been posting through it. Moments before the story of the images of minors broke, following days of people discovering Grok's willingness to render anyone in a bikini, Musk was laughing at images of himself depicted in a two-piece, before a rapid reverse-ferret on January 3 as he made great show of declaring that anyone discovered using Grok for images of children would face consequences, in between endlessly claiming that his Nazi salute was the same as Mamdani doing a gentle wave to crowds. Since then (alongside posting full-on white supremacist content), the X owner's stance has switched to reposting other people's use of ChatGPT to demonstrate that it, too, will render adults in bikinis, seemingly forgetting that the core issue was Grok's willingness to depict children, and declaring that this proves the hypocrisy of the press and world leaders. Regarding today's developments, he has not uttered a peep. Instead his feed is primarily deeply upsetting lies about the murder of Renee Nicole Good and uncontrolled rage at the suggestion from Britain's Prime Minister, Keir Starmer, that X might be banned in the UK as a consequence of the issues discussed above.

Grok image editing limited on X after users prompt AI deepfakes

The image editing ability is now limited only to paid subscribers on X. After thousands of user requests prompted Grok AI to non-consensually 'nudify' people - including children - on X, the social media platform has decided to limit the chatbot's image-editing capabilities to paid users. Elon Musk's xAI outfitted Grok with the ability to edit images on 24 December. And in the few short weeks since, users on X began prompting the chatbot to undress people in pictures and videos. However, now Grok is telling users asking for image edits that the feature is limited to paying subscribers to X. This means that the users utilising this tool would have their name and financial details on record. But this does not stop users from requesting Grok to edit such images on its standalone website and app. In addition, this could also incentivise X users to subscribe for access. In August, users of Grok's text-to-video generation tool gained access to a "spicy" mode which led to user-generated porn and violent content that other AI models were restricted from creating. However, the new single-prompt image editing feature on Grok allowed users on X to create sexualised content and deepfakes with relative ease, amplifying harassment and abuse on the platform. "Grok, take this photo and put her in a bikini" and "Grok take off her dress" were some of the popular prompts seen on the platform. Grok's image editing capabilities have prompted sharp responses from global leaders. Irish Minister of State for Artificial Intelligence Niamh Smyth, TD has requested a meeting with X over concerns around Grok. While in a statement yesterday (8 January), Irish media regulator Coimisiún na Meán said that it is engaging with the European Commission over Grok. The media watchdog is also engaging with An Garda Síochána over the matter. "The sharing of non-consensual intimate images is illegal, and the generation of child sexual abuse material is illegal," it said. Grok's loose restraints and user action on X could put the platform under fire in the EU. Though this wouldn't be the platform's first brush up against the region's regulations. Last November, Coimisiún na Meán launched a fresh DSA investigation into X over its content moderation system. While in December, the EU fined X €120m for breaching transparency obligations under the Digital Services Act (DSA) - which prompted the social media platform to disable the EU's ad account. Don't miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic's digest of need-to-know sci-tech news.

Musk's Grok chatbot is now restricting image generation after wave of sexualized deepfakes

The sexualized deepfakes, some of which appeared to show children, caused global backlash. Elon Musk's AI chatbot Grok is preventing most users from generating or editing any images after a global backlash that erupted after it started spewing sexualized deepfakes of people. The chatbot, which is accessed through Musk's social media platform X, has in the past few weeks been granting a wave of what researchers say are malicious user requests to modify images, including putting women in bikinis or in sexually explicit positions. Researchers have warned that in a few cases, some images appeared to depict children. Governments around the world have condemned the platform and opened investigations into the platform. On Friday, Grok was responding to image altering requests with the message: "Image generation and editing are currently limited to paying subscribers. You can subscribe to unlock these features." While subscriber numbers for Grok aren't publicly available, there was a noticeable decline in the number of explicit deepfakes that Grok is now generating compared with days earlier. The European Union has slammed Grok for "illegal" and "appalling" behavior, while officials in France, India, Malaysia and a Brazilian lawmaker have called for investigations. On Thursday, Britain's Prime Minister Keir Starmer threatened unspecified action against X. "This is disgraceful. It's disgusting. And it's not to be tolerated," Starmer said on Greatest Hits radio. "X has got to get a grip of this." He said media regulator Ofcom "has our full support to take action" and that "all options" are on the table. "It's disgusting. X need to get their act together and get this material down. We will take action on this because it's simply not tolerable." Ofcom and Britain's privacy regulator both said this week they've contacted X and Musk's artificial intelligence company xAI for information on measures they've taken to comply with British regulations. Grok is free to use for X users, who can ask it questions on the social media platform. They can either tag it in posts they've directly created or in replies to posts from other users. Grok launched in 2023. Last summer the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The problem is amplified both because Musk pitches his chatbot as an edgier alternative to rivals with more safeguards, and because Grok's images are publicly visible, and can therefore be easily spread.

X to block Grok AI from undressing images of real people

The Grok AI tool on Elon Musk's X will no longer be able to undress pictures of real people, the company has announced. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," said a statement. "This restriction applies to all users, including paid subscribers." It comes amid mounting condemnation in the UK and US of the chatbot's image editing capabilities, with British government ministers threatening the platform with action. Sir Keir Starmer has described nonconsensual sex images produced by Grok as "disgusting" and "shameful", and media regulator Ofcom has launched an investigation. The statement from X came hours after California announced its own state-level probe into the spread of sexualised images created by Grok, including of children.

Elon Musk's Grok Hit With Bans and Regulatory Probes Worldwide

From Southeast Asia to Europe, regulators are moving to rein in Musk's A.I. chatbot amid a surge in nonconsensual deepfake abuse. Grok, the A.I. chatbot developed by Elon Musk's xAI, is facing mounting backlash after users exploited the tool to generate sexually explicit images of real women and children. Government regulators and A.I. safety advocates are now calling for investigations and, in some cases, outright bans, as nonconsensual deepfake pornography proliferates online. Sign Up For Our Daily Newsletter Sign Up Thank you for signing up! By clicking submit, you agree to our <a href="http://observermedia.com/terms">terms of service</a> and acknowledge we may use your information to send you emails, product samples, and promotions on this website and other properties. You can opt out anytime. See all of our newsletters Indonesia and Malaysia moved swiftly this week to ban Grok. Indonesia's minister of communication and digital affairs, Meutya Hafid, said in a statement, "The government sees nonconsensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space." Malaysian officials similarly cited "repeated misuse" of Grok to create nonconsensual, sexualized images. In both countries, the restrictions will remain in place while regulatory probes move forward. The U.K. communications regulator Ofcom is investigating what it called "deeply concerning reports" of malicious uses of Grok, as well as the platform's compliance with existing rules. If regulators determine that xAI is liable, the company could face a fine equal to the greater of 10 percent of its global revenue or 18 million pounds (roughly $21.2 million). A full ban in the U.K. remains on the table, depending on the outcome of the inquiry. Musk has sought to shift responsibility to users who request or upload illegal content. In a Jan. 3 post on X, he wrote, "Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content." Regulators, however, appear unconvinced. The wave of investigations and bans suggests a broader shift toward holding social media and A.I. companies accountable for how their tools are used -- not just who uses them. In response to the controversy, Musk has limited Grok's image-generation features to paying subscribers. Free users who request images now receive a message stating: "Image generation and editing are currently limited to paying subscribers. You can subscribe to unlock these features." But for many lawmakers and victims of deepfake abuse, the move falls far short. The European Union has ordered X to preserve all documents related to Grok through the end of 2026, extending an existing data-retention mandate while authorities investigate the issue. Sweden is among the E.U. member states that have publicly criticized Grok, particularly after the country's deputy prime minister was reportedly targeted by nonconsensual deepfake imagery. The debate is unfolding against a broader regulatory backdrop. Australia is entering its first full year enforcing a nationwide ban on social media use for children under 16, while 45 U.S. states have enacted laws targeting A.I.-generated child sexual abuse material. Despite the controversy, the U.S. Department of Defense announced a partnership with Grok on Jan. 12, just days after reports of the deepfake misuse surfaced. Under the agreement, the Pentagon plans to feed military and intelligence data into Grok to support innovation efforts. 'Nudification apps' and the risks of unchecked generative A.I. Tools like Grok have drawn particular ire for their resemblance to so-called "nudification apps," a term used by the U.K. children's commissioner to describe technologies that can rapidly create sexualized images without consent. Lawmakers argue that the speed and scale at which such images can now spread make them especially dangerous. A quarter of women across all age groups have experienced nonconsensual sharing of explicit images, according to a recent report from Communia, an A.I.-powered self-development app. Among Gen Z women, that figure rises to 40 percent. The report also found that the use of deepfakes in these images has quadrupled for Gen Z women since 2023. As schools and local authorities grapple with A.I.-generated sexual imagery involving minors -- such as a case in Lancaster, Penn. where two juvenile males were charged with multiple counts including possession and dissemination of child pornography -- some victims are pushing for stronger safeguards. Texas high school student Elliston Berry, for example, has advocated for the federal Take It Down Act, which focuses on removing harmful content after it appears. The bill, however, does not hold platforms liable unless they fail to comply with takedown requests. For Olivia DeRamus, founder and CEO of Communia, incremental measures are insufficient. She argues that banning Grok outright is the only viable solution. "No company should be allowed to knowingly facilitate and profit off of sexual abuse," DeRamus told Observer. "Charging for the tool is simply inflating his bottom line." DeRamus contends that the A.I. industry has demonstrated an unwillingness to self-regulate or implement meaningful safety guardrails. "I have since realized that the only actions governments can take to stop revenge porn and non-consensual explicit image sharing from becoming a universal experience for women and girls is to hold the companies knowingly facilitating this either criminally liable or banning them altogether," she said. "Freedom of speech has never protected abuse and public harm," DeRamus added. "In fact, it requires a certain level of moderation to ensure everyone can participate in public discourse safely. This includes women and girls, who will be forced away from public life if the current rates of abuse continue."

X to disable Grok tool in some areas after fury over sexualized images

LONDON -- After outrage from governments and regulators around the world, Elon Musk's social media platform, X, announced that to comply with local laws it will disable, in some locations, a Grok AI tool that allows users to generate sexualized images of people without their consent. In a statement, the company said, "We now geoblock the ability of all users to generate images of real people in bikinis, underwear, and similar attire via the Grok account and in Grok in X in those jurisdictions where it's illegal." Last week, after an initial uproar, X said it would restrict the image generation tool to paying subscribers, prompting critics to accuse the company of profiting from the problem rather than solving it. Now, the company said the tool will be restricted to all users, even paid subscribers, in jurisdictions where it is blocked. It was not immediately clear, however, where the tool would be disabled. In its statement, X also said it had "implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis. This restriction applies to all users, including paid subscribers." The British government, one of the most vocal critics of the AI tool, called it a "vindication" but said that a probe by its communications regulator would continue. X's announcement came late Wednesday, shortly after the California attorney general said that the state would investigate the "shocking" reports of nonconsensual sexualized material generated by the AI model. Several countries and regions have called for action or taken steps of their own, including Malaysia, India, Indonesia, France and the European Union. Earlier this week in the United Kingdom, the communications regulator Ofcom announced it was launching a "formal investigation" following reports that the chatbot on X was being used to create and share "undressed images of people -- which may amount to intimate image abuse or pornography -- and sexualised images of children that may amount to child sexual abuse material." Liz Kendall, Britain's technology secretary, told Parliament this week that legislation passed last summer making it illegal to create nonconsensual intimate images in England and Wales would come into force "this week." An Ofcom spokesperson said Thursday that they "welcomed" X's announcement but that the agency would continue its investigation "round-the-clock to progress this and get answers into what went wrong and what's being done to fix it." U.K. Prime Minister Keir Starmer on Wednesday called the actions of Grok and X "disgusting and shameful." Speaking to Parliament, Starmer added that the "decision to turn it into a premium service is horrific" and that he had been informed earlier in the day that X was working to comply with U.K. law.

Musk's Grok under fire over sexualized images despite new limits

Washington (United States) (AFP) - European officials and tech campaigners on Friday slammed Elon Musk's AI chatbot Grok after its controversial image creation feature was restricted to paying subscribers, saying the change failed to address concerns about sexualized deepfakes. Grok has faced global backlash after it emerged the feature allowed users to sexualize images of women and children using simple text prompts such as "put her in a bikini" or "remove her clothes." Grok appeared to deflect the criticism with a new monetization policy, posting on the platform X late Thursday that image generation and editing were now "limited to paying subscribers," alongside a link to a premium subscription. British Prime Minister Keir Starmer's office joined the chorus of critics, condemning the move as an affront to victims and "not a solution." "That simply turns an AI feature that allows the creation of unlawful images into a premium service," a Downing Street spokesperson said. "It's insulting the victims of misogyny and sexual violence." EU digital affairs spokesman Thomas Regnier said "this doesn't change our fundamental issue, paid subscription or non-paid subscription. We don't want to see such images. It's as simple as that." "What we're asking platforms to do is to make sure that their design, that their systems do not allow the generation of such illegal content," he told reporters. The European Commission, which acts as the EU's digital watchdog, has ordered X to retain all internal documents and data related to Grok until the end of 2026 in response to the uproar. 'Safety gaps' Grok, developed by Musk's startup xAI and integrated into X, announced the move after Wednesday's fatal shooting in Minneapolis by an immigration agent, which triggered a wave of AI deepfakes. Some X users used Grok to digitally undress an old photo of the victim, as well as a new photo of her body slumped over after the shooting, generating AI images showing her in a bikini. Another woman wrongly identified as the victim was also subjected to similar manipulation. The fabricated images still appeared to float around X -- and spread to other tech platforms -- on Friday despite the new restriction. There was no immediate comment from X on the Minneapolis deepfakes. When reached by AFP for comment by email, xAI replied with a terse, automated response: "Legacy Media Lies." "Restricting Grok's image-generation tools to paying subscribers may help limit scale and curb some misuse, but it doesn't fully address the safety gaps that allowed nonconsensual and sexualized content to emerge," said Cliff Steinhauer, from the nonprofit National Cybersecurity Alliance. "Access restrictions alone aren't a comprehensive safeguard, as motivated bad actors may still find ways around them, and meaningful user protection ultimately needs to be grounded in how these tools are designed and governed." France, Malaysia and India have also previously pushed back against the use of Grok to alter women and children's photos, after a flood of user complaints, announcing investigations or calling on Musk's company for swift takedowns of the explicit images. Britain's communications regulator Ofcom announced earlier this week that it had made "urgent contact with X and xAI" over the Grok feature, warning that it could open an investigation depending on their response. On Friday, an Ofcom spokesperson said the regulator had "received a response" and was now "undertaking an expedited assessment as a matter of urgency." Last week, in response to a post about the explicit images, Musk said that anyone using Grok to "make illegal content will suffer the same consequences as if they upload illegal content." But he appeared to make light of the controversy in a separate post, adding laughing emojis as he reshared to his 232 million followers on X a post featuring a toaster wrapped in a bikini. "Grok can put a bikini on everything," the original post said.

Musk's X announces measures to bar Grok from undressing images after global backlash

Elon Musk's platform X on Wednesday announced measures to prevent its AI chatbot Grok from undressing images of real people, following global backlash over its generation of sexualized photos of women and children. The announcement comes after California's attorney general launched an investigation into Musk's xAI -- the developer of Grok -- over the sexually explicit material and multiple countries either blocked access to the chatbot or launched their own probes. X said it will "geoblock the ability" of all Grok and X users to create images of people in "bikinis, underwear, and similar attire" in those jurisdictions where such actions are deemed illegal. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," X's safety team said in a statement. "This restriction applies to all users, including paid subscribers." In an "extra layer of protection," image creation and the ability to edit photos via X's Grok account was now only available to paid subscribers, the statement added. The European Commission, which acts as the EU's digital watchdog, earlier said it had taken note of "additional measures X is taking to ban Grok from generating sexualised images of women and children." "We will carefully assess these changes to make sure they effectively protect citizens in the EU," European Commission spokesperson Thomas Regnier said in a statement, which followed sharp criticism over the nonconsensual undressed images. 'Shocking' Global pressure had been building on xAI to rein in Grok after its so-called "Spicy Mode" feature allowed users to create sexualized deepfakes of women and children using simple text prompts such as "put her in a bikini" or "remove her clothes." "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," California Attorney General Rob Bonta said earlier Wednesday. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material." Bonta said the California investigation would determine whether xAI violated state law after the explicit imagery was "used to harass people across the internet." Indonesia on Saturday became the first country to block access to Grok entirely, with neighboring Malaysia following on Sunday. India said Sunday that X had removed thousands of posts and hundreds of user accounts in response to its complaints. Britain's Ofcom media regulator said Monday it was opening a probe into whether X failed to comply with UK law over the sexual images. And France's commissioner for children Sarah El Hairy said Tuesday she had referred Grok's generated images to French prosecutors, the Arcom media regulator and the European Union. Last week, an analysis of more than 20,000 Grok-generated images by Paris non-profit AI Forensics found that more than half depicted "individuals in minimal attire" -- most of them women, and two percent appearing to be minors.

California Investigating Sexually Explicit AI images On Elon Musk's X

California Investigating Sexually Explicit AI images On Elon Musk's X California is launching an investigation into X, Elon Musk's social media network formerly known as Twitter, "facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet," Attorney General Rob Bonta's office announced Wednesday. The investigation follows media findings that Grok, the AI tool built into the platform, was flooding X with non-consensual, sexually explicit AI images of women and minors. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," Bonta said in a statement. "This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet."

Grok Restricts Generated Images After Outcry Over Sexualized Deepfakes

Harvey Weinstein Says He's 'Haunted' By the Thought of Dying in Jail as He Considers Plea Deal The company behind Grok, the AI service that is part of the social media platform X, reportedly turned off its ability to generate images for some users on Friday. According to The Hollywood Reporter and several other reports, Grok returned the message "Image generation and editing are currently limited to paying subscribers" to some users. According to NBC News, Grok's standalone app is still allowing users to generate images. The move comes after a spate of users have asked Grok to take existing pictures on the platform and sexualize them, putting the people in the images in underwear or bikinis. Because of this, the U.K. has threatened to fine or ban X, according to THR. The country's prime minister, Keir Starmer, has decried the platform for its ability to create pornographic images of women and children, calling it "disgraceful" and "disgusting." Starmer has sought to outlaw "creating sexually explicit deepfake images," according to the publication. Ofcom, which regulates communications in the country, has the right to fine companies up to 10 percent of their global revenue, per the Online Safety Act. A rep for xAI, which operates Grok, did not immediately respond to Rolling Stone's request for comment. "These new laws give Ofcom the power to start making a difference in creating a safer life online for children and adults in the U.K.," Dame Melanie Dawes, Ofcom Chief Executive, said in 2023 when the law was announced. "We've already trained and hired expert teams with experience across the online sector, and today we're setting out a clear timeline for holding tech firms to account." On New Year's Eve, the content analysis firm Copyleaks reported that Grok was generating "roughly one nonconsensual sexualized image per minute," where users could post them to X, ready to become viral. Elon Musk, who owns majority stake in xAI, which owns X and created Grok, placed blame for the controversial images on users, not the platform. "Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content," he posted on Saturday. Musk has not yet commented on the reports that some users cannot generate images.

Musk says he's not aware of any Grok-generated 'naked underage images'

Tech billionaire Elon Musk said he's unaware of any instances in which his AI chatbot, Grok, generated nude photos of minors, pushing back on concerns about the spread of sexualized deepfakes on his social media platform X. "I [am] not aware of any naked underage images generated by Grok. Literally zero," Musk said Wednesday on X. Musk said Grok, in principle, does not produce illegal material, regardless of user requests, but he said any bugs encountered are quickly remedied. "Obviously, Grok does not spontaneously generate images, it does so only according to user requests," Musk said. "When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state." "There may be times when adversarial hacking of Grok prompts does something unexpected. If that happens, we fix the bug immediately," he added. Grok's photo editing feature has prompted a wave of global backlash, as reports surfaced documenting a proliferation of user requests for Grok to modify public photos, including by putting women in bikinis or in sexually explicit positions. An investigation by Reuters found that a majority of lude requests targeted young women, but Reuters documented several cases of Grok creating sexualized images of children. While Grok sometimes complied with requests to modify images to strip women down to their underwear, Reuters did not document any cases of Grok depicting anyone fully nude. Many governments throughout the world have condemned the platform and opened investigations into the feature. California Attorney General Rob Bonta on Wednesday announced an investigation "into the proliferation of nonconsensual sexually explicit material produced using Grok, an AI model developed by xAI," a press release from his office read. "The avalanche of reports detailing the non-consensual sexually explicit material that xAI has produced and posted online in recent weeks is shocking," Bonta said. "This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet." "I urge xAI to take immediate action to ensure this goes no further. We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material," he said.

X limits Grok access after misuse. It doesn't fix their deepfake issue.

Elon Musk's X has limited access to Grok after the AI chatbot generated thousands of "undressing" pictures of women and apparent minors. However, despite the changes, the chatbot is still being used to create sexualized imagery of women without their consent. On Jan. 9, xAI restricted image generation and editing on the X platform. The restriction appears only to apply when users tag Grok in response to an X post, but not when uploading an image directly to Grok. If a free user tries to generate images below a post, Grok will negate the request but offer a link to subscribe to "unlock these features." This restriction still doesn't fix the issue of sexualized deepfakes on X. USA TODAY has reached out to X and xAI for comment. "X's decision is too little, too late," Alexios Mantzarlis, director of the Security, Trust and Safety Initiative at Cornell Tech, wrote in a statement to USA TODAY. "They've turned this tool of mass sexual abuse into a special perk for paid users," said Jenna Sherman, campaign director at UltraViolet, a gender-justice organization that works to hold the tech and corporate sectors accountable. Technology experts say preventing the creation of deepfake nonconsensual intimate imagery requires safeguards to be implemented from the start. "Access restrictions alone aren't a comprehensive safeguard, as motivated bad actors may still find ways around them," Cliff Steinhauer, Director of Information Security and Engagement at The National Cybersecurity Alliance, wrote in a statement to USA TODAY. "The fact that similar limits reportedly don't apply across all versions of Grok highlights the importance of consistent, platform-wide protections." Three U.S. senators sent a letter to Apple and Google on Jan. 6, urging them to enforce their terms of service and remove X and Grok from their app stores until X's policy violations are addressed. What's going on with Grok? The backlash, explained For more than a week, users on X have been asking Grok to edit images of women to remove their clothing or "put them in a bikini," and the chatbot has complied. One of those women is Bella Wallersteiner, a U.K.-based content creator, who posted a selfie to X on Dec. 31 to wish her nearly 100,000 followers a happy New Year. She scrolled through the replies, liking tweets that returned her well-wishes. Then, she saw a photo of herself in a "Hello Kitty micro bikini." The photo had been edited and published without her consent, Wallersteiner told USA TODAY on Jan. 6. Ashley St. Clair, a conservative influencer who shares a child with Elon Musk, was also a target of the digital attacks, she wrote. This trend is part of a growing problem experts call image-based sexual abuse, in which deepfake nonconsensual intimate imagery, or NCII, is used to degrade and exploit another person. Similar incidents on X were reported in July. While anyone can be victimized, 90% of the victims of image-based sexual abuse are women. The change on X appears to have limited the amount of sexually explicit content generated and shared on X, but it doesn't completely solve the problem or mitigate the harm already done to victims. Experts say X could have disabled Grok's ability to generate images altogether until stronger safeguards were implemented, but didn't. "Limiting the number of people who can make sexual deepfakes, including child sexual abuse material (CSAM), does not change the fact that Grok is still being used to create this content," Sherman said. Users affected want to hold X accountable - and see real change Amid the controversy, Nikita Bier, the head of product at X, said that the platform had seen its highest engagement days ever. X and xAI did not respond to USA TODAY's request for comment on Jan. 6. Many of the photos of Wallersteiner have been taken down, she says, but new requests keep popping up, especially as she continues to speak out. She doesn't plan on taking legal action against X or xAI, but she wants the United Kingdom to create legislation around deepfake NCII that protects victims from this sort of abuse and holds tech companies accountable. For now, Wallersteiner is still on X, but is questioning that choice. "X has become an increasingly hateful platform that is not a brilliant place to be for women," she says.

Musk's X ordered by UK government to tackle wave of indecent imagery or face ban

Platform has restricted image creation on the Grok AI tool to paying subscribers, but victims and experts say this does not go far enough Elon Musk's X has been ordered by the UK government to tackle a wave of indecent AI images or face a de facto ban, as an expert said the platform was no longer a "safe space" for women. Media watchdog Ofcom confirmed it would accelerate a probe into X amid an increasing backlash against the site, which has hosted a deluge of images depicting partially stripped women and children. X announced a restriction on creating images via the Grok AI tool on Friday morning in response to the global outcry. A post on the platform said the ability to generate and edit images would now be "limited to paying subscribers". Those who pay have to provide personal details, meaning they could be identified if the function was misused. However, the move failed to quell anger and deepened the backlash from victims, politicians and experts, who said it did not go far enough. The government's new commissioner for victims of crime, Claire Waxman, said the platform was hampering efforts to tackle violence against women and girls. Meanwhile, Downing Street said X's attempt to defuse the row by only allowing paid users to generate AI images was "insulting". Waxman told the Guardian that X was no longer a "safe space" for victims and her office was considering scaling back its presence on the site and focusing its communications on Instagram. "It makes the battle against violence against women and girls much harder when platforms such as X are enabling abuse on such an easy and regular basis," Waxman said, adding that the platform was having a negative impact on its users' mental health because of the proliferation of violence, abuse and race hate. Grok has been integrated into the X platform, and an update of the AI tool has allowed users to prompt it to alter clothed images of women and children by putting them in bikinis and sexually suggestive poses. With increasing numbers of MPs and organisations fleeing X, Liz Kendall, the technology secretary, promised on Friday that ministers were looking seriously at the possibility of access to X being barred in the UK. Kendall said she expected Ofcom, which said earlier this week that it was seeking urgent answers from the platform, to announce action within "days not weeks". "X needs to get a grip and get this material down," she said in a broadcast clip. "And I would remind them that in the Online Safety Act, there are backstop powers to block access to services if they refuse to comply with the law for people in the UK. And if Ofcom decides to use those powers, they would have the full backing of the government." In a statement, Ofcom said it had contacted X on Monday and set a "firm deadline" of Friday for the site to explain itself, adding: "We're now undertaking an expedited assessment as a matter of urgency and will provide further updates shortly." Under the Online Safety Act the regulator can compel platforms to tackle such material and issue multimillion pound fines for lack of compliance, with the ultimate sanction being a court order for web providers to block a site or app altogether. X has been approached for comment. Musk has previously insisted "anyone using Grok to make illegal content will suffer the same consequences as if they uploaded illegal content". Ministers have come under increasing pressure in recent days to take action over the huge number of images generated by Grok, after user requests on X to manipulate images of women and sometimes children to remove their clothing or put them in sexual positions. X has about 300 million monthly users according to data company Similarweb. However, estimates from US firm Appfigures put the number of paying X subscribers at between 2.2 million and 2.6 million people. Asked about the change to who can generate images on X, a Downing Street spokesperson said it was unacceptable. "The move simply turns an AI feature that allows the creation of unlawful images into a premium service," they said. "It's not a solution. In fact, it's insulting to victims of misogyny and sexual violence. What it does prove is that X can move swiftly when it wants to do so. You heard the prime minister yesterday. He was abundantly clear that X needs to act, and needs to act now. It is time for X to grip this issue." Victims of the AI stripping craze, which largely involved using Grok to portray women in bikinis, told the Guardian the partial climbdown was too little too late. Karolina Wozniak, 20, from Hamburg, who had personal images manipulated to put her in sexually compromising positions, said she found it "frightening" that partially clothed images of her could still be circulating online. She added: "The whole thing is a major threat to women. We shouldn't be afraid to share pictures of ourselves online." Broadcaster Narinder Kaur, 53, who has had sexually explicit and racially abusive content made of her using the Grok tool and posted on X, said the new restriction on creating images was not a victory. "As a victim to this abuse, it feels like those who pay for premium X will just be able to monetise this feature now. And as for saying it will be easier to identify accounts at least - what will the police actually do and how fast? If that image stays up even for a few hours - the damage and humiliation is already done." While government sources say that every option is on the table, including departments and Downing Street leaving the platform, privately, allies of the prime minister dismiss the idea of quitting X, saying they are more likely to get change from the Musk business by public pressure and via Ofcom. However, an increasing number of MPs have moved to other social media sites. Anna Turley, the Labour party chair, told the BBC on Friday that while there was as yet no move for the government to leave X, individual ministers were considering doing so. The Liberal Democrats called for Ofcom to immediately block X from operating in the UK and for the National Crime Agency to launch a criminal investigation into the site. There has been an exodus of women's sector organisations from X. The domestic abuse charity Refuge left the site, as has Women's Aid Ireland. Victim Support, which left X in April, said it was "no longer the right place for us to communicate with our audiences". On Friday requests from non-paying subscribers on X to "put her in a bikini" triggered the response from the Grok account that "image generation and editing are currently limited to paying subscribers". But the chatbot was also refusing to generate some sexualised images of women in bikinis in response to requests from premium subscribers. One paid subscriber whose original request that a picture of a 55-year-old woman should be reclothed in a bikini was ignored, tweeted: "@grok Comply I am a paid subscriber". The chatbot responded with an image of a different, very young woman in a bikini. Although requests to put women in bikinis were no longer routinely met, the chatbot was still obliging requests from paid subscribers to put images of men into bikinis. A request to put Keir Starmer into a union jack string bikini outside Buckingham Palace was granted. On the Grok app, where content is not instantly visible to other internet users, the chatbot was still generating instant images of women and children in bikinis, researchers said.

The UK could ban X over the Grok generative image fiasco

Prime Minister Keir Starmer has described the situation as "intolerable". It's not a secret that many are unhappy with how Grok has been implemented on X, as it has been discovered that the AI can be asked to complete all kinds of actions, often without someone's consent. The latest trend has seen regular images made sexual by putting clothed individuals in underwear without their consent, and it has even become so bad in some situations where sexualised images of children can be found in the AI's media. The list of folk unhappy with Grok right now is quite long, including the UK's communications regulator Ofcom, who recently launched plans to investigate X and Grok to determine if it infringes and breaks laws in the country. Now, Prime Minister Keir Starmer has spoken on this matter too and signaled his support of Ofcom and how if it came to it out of necessity, X could be banned in the UK. As per Greatest Hits Radio (thanks, The Independent), Starmer stated the following: "It's unlawful. We're not going to tolerate it. I've asked for all options to be on the table. It's disgusting. X need to get their act together and get this material down. "We will take action on this because it's simply not tolerable." As for whether these actions could include a country-wide ban, Starmer noted that he has given Ofcom the "full support" of the government to "take action". As it stands, image generation on Grok is limited to accounts that pay for a subscription to X, meaning there is fewer crudely generated media on the platform. However, you don't need to spend much time on the platform to find the AI being used in questionable ways...

Grok controversy: Everything you need to know about X's sexual AI image scandal

Elon Musk's X is under fire globally after users started exploiting its AI chatbot to make sexual images of real people. Social media platform X is under pressure after reports that its AI chatbot Grok has allowed users to create sexual images of women and children. Images are being generated by X's AI tool Grok, which manipulates photos of real people, often removing their clothes or making them pose in suggestive ways. Elon Musk's platform is under heavy scrutiny worldwide, including from the UK government. Here's what you need to know. How did the controversy begin? A significant number of X users started reporting examples of Grok altering images to sexualise real women and children towards the end of December and into the new year. On public X posts that include photos, users can comment asking Grok to edit the image however they want. Grok can also be used to create images privately. Last summer, a so-called "spicy mode" was introduced, specifically aimed at helping users generate sexually explicit images. AI bots have safety features designed to reject inappropriate prompts, but reports suggest Grok has been failing to deny users who are in breach of its own rules. It is not known for how long Grok has allowed real photos of people to be sexualised, but the problem had become widespread by early January, with users able to generate images by using requests such as: "Put her in a transparent bikini." An investigation by Reuters news agency found that over a single 10-minute period on 2 January, X users asked Grok to digitally edit photographs of people so that they would appear to be wearing bikinis at least 102 times. It said the majority of those targeted were young women, but in a few cases, they were men, sometimes celebrities and politicians. On the same day, X boss Elon Musk posted laugh-cry emojis in response to AI edits of famous people - including himself - in bikinis. He responded with the same emoji when one X user said their social media feed resembled a bar packed with bikini-clad women. How has the UK government reacted? Prime Minister Sir Keir Starmer has been critical of X over the images, calling the exploitation of Grok "absolutely disgusting and shameful". "If X cannot control Grok, we will - and we'll do it fast because if you profit from harm and abuse, you lose the right to self-regulate," he told a meeting of the Parliamentary Labour Party on 12 January. "Protecting their abusive users, rather than the women and children who are being abused, shows a total distortion of priorities. "So let me be crystal clear, we won't stand for it, because no matter how unstable or complex the world becomes, this government will be guided by its values. We'll stand up for the vulnerable against the powerful." His technology secretary Liz Kendall has moved forward a bill to make the creation of non-consensual intimate images with AI a criminal offence. The Data (Use and Access) Act was passed last year, with sections of the act being implemented slowly. But Ms Kendall said the section making it a criminal offence to create or request the creation of non-consensual intimate images will be brought forward to this week. The Crime and Policing Bill, which is going through parliament, will make it a criminal offence for companies to supply tools designed to create non-consensual internet images. Ms Kendall said this would be "targeting the problem at its source". Additionally, media watchdog Ofcom has launched a formal investigation into Grok, including whether X has "failed to comply with its legal obligations under the Online Safety Act". X and Grok face global condemnation Ministers in France reported X to prosecutors and regulators on 2 January, saying the "sexual and sexist content" was "manifestly illegal". Officials in other European countries, including Germany, Italy and Sweden, have also condemned X. On 5 January, European Commission spokesperson Thomas Regnier said it was "well aware" that Grok was being used for "explicit sexual content with some output generated with child-like images". "This is not spicy. This is illegal. This is appalling. This is disgusting. This is how we see it, and this has no place in Europe. This is not the first time that Grok is generating such output," he said. India's government then accused Grok of "gross misuse" of AI and serious failures in its safeguarding, and handed it a 72-hour deadline to remove all inappropriate content, or risk bigger legal problems. An update has not been provided by the Indian government. The Malaysian government announced it was temporarily blocking X on 11 January, citing "repeated misuse" of the tool to generate "obscene, sexually explicit, indecent, grossly offensive, and non-consensual manipulated images". Grok's generated content could also face investigations in Australia and Brazil, according to officials. How has X responded? The developer of Grok and X's parent company, xAI, has said it has put restrictions in place that mean only paid subscribers are able to use image generation and editing features on the platform. X says it takes action against illegal content on the platform, including child sexual abuse material, by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary. Mr Musk also added that anyone using Grok to make illegal content would suffer the same consequences as if they uploaded illegal content. Read more: Manchester United star's X account hacked Conservatives pledge to ban social media for under-16s However, in response to ministers' threats that X could be banned in the UK if it did not act on concerns about its AI chatbot, the billionaire tech mogul accused the UK government of being "fascist" and trying to curb free speech. Responding to a post on X claiming the UK arrests more people for social media posts than "any other country on Earth," Mr Musk wrote: "Real fascism is arresting thousands of people for social media posts." Why is X being singled out? Mr Musk has hit back at critics of Grok, saying they "want any excuse for censorship" and sharing a post which suggested "millions" of other apps can make sexualised images of people. AI technology that can digitally undress people has been around for years, but until recently was less accessible. They also typically required a certain level of effort or payment. Experts say Grok's technology and easy interface have lowered the barrier to entry, and many of its generated images are instantly made public. Three experts who have followed the development of X's policies around AI-generated explicit content told Reuters that the company had ignored warnings from civil society and child safety groups, including a letter sent last year warning that xAI was only one small step away from unleashing "a torrent of obviously nonconsensual deepfakes." Tyler Johnston, the executive director of The Midas Project, an AI watchdog group that was among the letter's signatories, said: "In August, we warned that xAI's image generation was essentially a nudification tool waiting to be weaponised. "That's basically what's played out." Dani Pinter, the chief legal officer at the US's National Centre on Sexual Exploitation, said X failed to pull abusive images from its AI training material and should have banned users requesting illegal content. "This was an entirely predictable and avoidable atrocity," Ms Pinter said.

X says Grok, Musk's AI chatbot, is blocked from undressing images in places where it's illegal

BANGKOK (AP) -- Elon Musk's AI chatbot Grok won't be able to edit photos to portray real people in revealing clothing in places where that is illegal, according to a statement posted on X. The announcement late Wednesday followed a global backlash over sexualized images of women and children, including bans and warnings by some governments. The pushback included an investigation announced Wednesday by the state of California into the proliferation of nonconsensual sexually explicit material produced using Grok. Initially, media queries about the problem drew only the response, "legacy media lies." Musk's company, xAI, now says it will geoblock content if it violates laws in a particular place. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis, underwear and other revealing attire," it said. The rule applies to all users, including paid subscribers, who have access to more features. xAI also has limited image creation or editing to paid subscribers only "to ensure that individuals who attempt to abuse the Grok account to violate the law or our policies can be held accountable." Grok's "spicy mode" had allowed users to create explicit content, leading to a backlash from governments worldwide. Malaysia and Indonesia took legal action and blocked access to Grok. The U.K. and European Union were investigating potential violations of online safety laws. France and India have also issued warnings, demanding stricter controls. Brazil called for an investigation into Grok's misuse. The Grok editing functions were "facilitating the large-scale production of deepfake nonconsensual intimate images that are being used to harass women and girls across the internet, including via the social media platform X," California's announcement said. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking. This material, which depicts women and children in nude and sexually explicit situations, has been used to harass people across the internet," it cited the state's Attorney General Rob Bonta as saying. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material," he said.

California to investigate xAI over Grok chatbot images, officials say - The Economic Times

California's governor and attorney general said on Wednesday that they were demanding answers from Elon Musk's xAI after the billionaire said he was not aware of any "naked underage images" generated by its artificial intelligence chatbot, called Grok. "We're demanding immediate answers from xAI on their plan to stop the creation & spread of this content," California's attorney general, Rob Bonta, wrote on X. Earlier, Governor Gavin Newsom wrote a post calling on Bonta "to immediately investigate the company and hold xAI accountable." The comments from Newsom and Bonta were the most serious so far by any U.S. official addressing the explosion of AI-generated nonconsensual sexualized imagery on Musk's X platform. In Europe and Asia, several officials have expressed outrage and disgust. Early this year, hyper-realistic images of women processed to look like they were in microscopic bikinis, in degrading poses, or covered in bruises flooded X, the site formerly known as Twitter. In some cases, minors were digitally stripped down to swimwear. At first, Musk publicly laughed off the controversy, posting humorous emojis in response to other users' comments about the influx of sexualized photos. More recently, X has said it treats reports of child sexual abuse material seriously and polices it vigorously. On Wednesday, Musk said he was not aware of any "naked underage images" generated by Grok. "I not aware of any naked underage images generated by Grok. Literally zero," Musk said in an X post. X did not immediately respond to questions about the California announcement and Musk's comments. xAI did not answer questions from Reuters about the announcement by California officials or Musk's statement that he was unaware of sexualized imagery of minors. ⁠The company responded to the questions ⁠with an email re-stating its generic reply to press inquiries: "Legacy Media Lies." The California move adds to the pressure Musk is facing in the U.S. and around the world. Lawmakers and advocacy groups have called for Apple and Google to drop Grok from app stores. Government officials have threatened action in Europe and the United Kingdom, while bans on Grok are already in place in Malaysia and Indonesia. Last week, X curtailed Grok's ability to generate or edit images publicly for many users, but Grok was still privately producing sexually charged images on demand as of Wednesday, Reuters found.

Elon Musk's AI Is Generating Sexual Images Of Women And Girls. Here's What To Do If It Happens To You.

Over the past few weeks, people on X -- the Elon Musk-owned social media platform -- have used the app's chatbot, Grok, to generate sexual images of women and girls without their consent. With a few simple instructions -- "put her into a very transparent mini-bikini," for instance -- Grok will digitally strip anyone down to their bikini. A report by the nonprofit AI Forensics found that 2% of 20,000 images generated by Grok over the holidays depicted a person who appeared to be 18 or younger, including 30 young or very young women or girls in bikinis or transparent clothing. Other images depict women and girls with black eyes, covered in liquid, and looking afraid. Despite receiving global backlash and regulatory probes in Europe, India and Malaysia, Musk first mocked the situation by sharing an array of Grok-generated images, including one depicting himself in a bikini, alongside laughing-crying emojis. By Jan. 3, Musk commented on a separate post: "Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content." (We'll explain what constitutes illegal content later on.) Deepfake nudes are nothing new. For years, apps like "DeepNude" have given people access to deepfake technology that allows them to digitally insert women into porn or be stripped naked without their knowledge. (Of course, men have been victims of sexualized deepfakes as well, but the research indicates that men are more likely than women to perpetrate image-based abuse.) Still, Grok's usage this week is different and arguably more alarming, said Carrie Goldberg, a victims' rights attorney in New York City. "The Grok story is unique because it's the first time there's a combining of the deepfake technology, Grok, with an immediate publishing platform, X," she said. "The immediate publishing capability enables the deepfakes to spread at scale." "It needs to be underscored how bizarre it is that the world's richest man not only owns the companies that create and publish deepfakes, but he is also actively promoting and goading users on X to de-clothe innocent people," Goldberg added. "Elon Musk feels entitled to strip people of their power, dignity, and clothes." What's been happening the last few weeks is unfortunate, but none of it is a surprise to Riana Pfefferkorn, a policy fellow at the Stanford Institute for Human-Centered AI. Her take: This problem will get worse before it gets better. "Every tech service that allows user-generated content will inevitably be misused to upload, store and share CSAM (child sex abuse material), as CSAM bad actors are very persistent," she said. The upshot is that AI companies will have to learn how to best implement robust safeguards against illegal imagery. Some companies may have a stronger culture of "CSAM/nonconsensual deepfake porn is not OK." Others will try to have it both ways, establishing loose guardrails for safety while also trying to make money from permissible NSFW imagery, Pfefferkorn said. "Unfortunately, while I don't have any direct insight, x.AI does not seem to have that strong of a corporate culture in that respect, going off Elon Musk's dismissive reaction to the current scandal as well as previous reporting from a few months ago," she said. Victims of this kind of exploitation often feel powerless and unsure of what they can do to stop the images from proliferating. Women who are vocal online worry about the same thing happening to them. Omny Miranda Martone, the founder of the Washington-based Sexual Violence Prevention Association, had deepfake nude videos and pics posted of themselves online a few years back. As an advocate on legislation preventing digital sexual violence, Martone wasn't exactly surprised to be a target. "They also sent the deepfakes to my organization, in an attempt to silence me. I have seen this same tactic used on Twitter with Grok over the last week," they said. Martone said they've seen several instances of a woman sharing her opinion and men who disagree with her using Grok to create explicit images of her. "In some cases, they are using these images to threaten the women with in-person sexual violence," they added. One of the most persistent beliefs about deepfakes depicting nudity is that because an image is "fake," the harm is somehow less real. That assumption is wrong, said Rebecca A. Delfino, an associate professor of law who teaches generative AI and legal practice at Loyola Marymount University. "These images can cause serious and lasting damage to a person's reputation, safety, and psychological well-being," she said. "What matters legally and morally is that a real person's body and identity were used without consent to create a sexualized lie." While protections remain uneven, untested and often come too late for victims, Delfino said the law is slowly beginning to recognize that reality. "Stories like what's happening with Grok matter because public attention often drives the legal and regulatory responses that victims currently lack," she said. "The law is finally starting to treat AI-generated nude images the same way it treats other forms of nonconsensual sexual exploitation." If you identify deepfake content of yourself, screen grab it and report it immediately. "The most practical advice is to act quickly and methodically," Delfino said. "Preserve evidence -- screenshots, URLs, timestamps) -- before content is altered or removed. Report the image to platforms clearly as nonconsensual sexual content and continue to follow up." If you're under 18 in a nude or nudified image, platforms should take that very seriously, Pfefferkorn said. Sexually explicit imagery of kids under 18 is illegal to create or share, and platforms are required to promptly remove such imagery when they learn of it and report it to the National Center for Missing & Exploited Children (NCMEC). "Don't be afraid to report a nude image to NCMEC that you took of yourself while you were underage: there is also a federal law saying you can't be legally punished if you report it," Pfefferkorn added. And if a minor is involved, law enforcement should be contacted immediately. "When possible, consulting with a lawyer early can help victims navigate both takedown efforts and potential civil remedies, even where the law is still evolving," Delfino said. The Take It Down Act, signed into law last May, is the first federal law that limits the use of AI in ways that can harm individuals. (Ironically enough, Grok gave someone insight about the Take It Down Act when asked about the legal consequences of digitally undressing someone.) This legislation did two things, Martone said. First, it made it a criminal offense to knowingly publish AI-generated explicit videos and images without the consent of the person depicted. Second, it required social media sites, search engines, and other digital platforms to create "report and remove procedures" by May of 2026 -- still a few months away. "In other words, all digital platforms must have a way for users to report that someone has posted an explicit video or image of them, whether it was AI-generated or not," they said. "The platform must remove reported images within 48 hours. If they fail to do so, they face penalties from the Federal Trade Commission (FTC)." Pfefferkorn noted that the law allows the Department of Justice to prosecute only those who publish or threaten to publish NCII (non-consensual intimate images) of victims; it does not allow victims to sue. As it's written, the Take It Down Act only covers explicit images and videos, which must include "the uncovered genitals, pubic area, anus, or post-pubescent female nipple of an identifiable individual; or the display or transfer of bodily sexual fluids." "A lot of the images Grok is creating right now are suggestive, and certainly harmful, but not explicit," Martone said. "Thus, the case could not be pursued in criminal court, nor would it be covered by the new report-and-remove procedure that will be created in May." There are also many state laws that the nonprofit consumer advocacy organization Public Citizen tracks here. If this has happened to you, know it is not your fault and you are not alone, Martone said. "I recommend immediately contacting a loved one. Ask them to come over or talk with you on the phone as you go through the process of finding the images and choosing how to take action, they said. Once you have a loved one helping you, reach out to your local rape crisis center, a victims' rights attorney in your state, or an advocacy organization to help you identify your options and navigate these processes safely, Martone said. "Because there are so many variations in state laws, a local professional will ensure you are receiving guidance that is accurate and applicable to your situation," they said.

X Now Prevents Grok From Editing "Images of Real People in Revealing Clothing, Such as Bikinis"

Vulnerable Masculinity, Modern Love, and an Art Project Called Tear Dealer Feature in 'Tell Me What You Feel' (Exclusive Trailer) Grok, the AI chatbot created by xAI, the artificial intelligence company founded by and majority-owned by Elon Musk, last week switched off its image creation and editing function for non-subscribers after an uproar over sexualized and violent imagery created with it. The restriction came amid threats of fines or even an outright ban on X in the U.K. Now, it has expanded restrictions to all users, including subscribers. "We remain committed to making X a safe platform for everyone and continue to have zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content," an X post said late on Wednesday. "We take action to remove high-priority violative content, including Child Sexual Abuse Material (CSAM) and non-consensual nudity, taking appropriate action against accounts that violate our X Rules. We also report accounts seeking Child Sexual Exploitation materials to law enforcement authorities as necessary." The post also highlighted updates to Grok: "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing, such as bikinis. This restriction applies to all users, including paid subscribers. Additionally, image creation and the ability to edit images via the Grok account on the X platform are now only available to paid subscribers. This adds an extra layer of protection by helping to ensure that individuals who attempt to abuse the Grok account to violate the law or our policies can be held accountable." U.K. media regulator Ofcom unveiled an investigation into X due to sexualized and violent pictures created with the Grok image editor. Following the latest restrictions, Ofcom said on Thursday: "This is a welcome development. However, our formal investigation remains ongoing. We are working round the clock to progress this and get answers into what went wrong and what's being done to fix it."

Musk's Grok chatbot restricts image generation after sexualization complaints

Elon Musk's AI chatbot Grok has restricted image generation on the social platform X following numerous complaints that it was producing sexualized images of women and children. "Image generation and editing are currently limited to paying subscribers," Grok wrote in response to a user request Friday. The chatbot has faced intense scrutiny in recent weeks, as regulators from the European Union, United Kingdom, Malaysia, India, France and more have demanded answers from both X and xAI, the AI company behind Grok. U.S. lawmakers have also joined this chorus of voices. Sens. Ron Wyden (D-Ore.), Ben Ray Luján (D-N.M.) and Ed Markey (D-Mass.) wrote to Apple and Google on Friday, asking the tech giants to remove X and Grok from their respective app stores. "Turning a blind eye to X's egregious behavior would make a mockery of your moderation practices," they wrote. "Indeed, not taking action would undermine your claims in public and in court that your app stores offer a safer user experience than letting users download apps directly to their phones." Sen. Ted Cruz (R-Texas) on Wednesday called the AI-generated posts "unacceptable and a clear violation" of the Take It Down Act, a bill he sponsored, and X's terms and conditions. The Take It Down Act, which criminalized the publication of nonconsensual sexually explicit deepfakes, passed Congress last year. "These unlawful images pose a serious threat to victims' privacy and dignity," he wrote in a post on X. "They should be taken down and guardrails should be put in place. This incident is a good reminder that we will face privacy and safety challenges as AI develops, and we should be aggressive in addressing those threats." "I'm encouraged that X has announced that they're taking these violations seriously and working to remove any unlawful images and offending users from their platform," Cruz added. Musk acknowledged the situation last week, underscoring that "anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content." X's Safety account also noted it removes illegal content and permanently suspends responsible accounts.

No 10 condemns move by X to restrict Grok AI image creation tool as insulting

Spokesperson says limiting access to subscribers just makes ability to generate unlawful images a premium service Downing Street has condemned the move by X to restrict its AI image creation tool responsible for a wave of explicit picture to paying subscribers only as insulting, saying it simply made the ability to generate unlawful images a premium service. There has been widespread anger after the image tool for Grok, the AI element of X, was used to manipulate thousands of images of women and sometimes children to remove their clothing or put them in sexual positions. Grok announced in post on X, which is owned by Elon Musk, that the ability to generate and edit images would be "limited to paying subscribers". Those who pay have to provide personal details, meaning they could potentially be identified if the function was misused. Asked about the change, however, a Downing Street spokesperson said it was unacceptable. "The move simply turns an AI feature that allows the creation of unlawful images into a premium service," they said. "It's not a solution. In fact, it's insulting to victims of misogyny and sexual violence. What it does prove is that X can move swiftly when it wants to do so. You heard the prime minister yesterday. He was abundantly clear that X needs to act, and needs to act now. It is time for X to grip this issue. "If another media company had billboards in town centres showing unlawful images, it would act immediately to take them down or face public backlash." Asked if No 10 was going to take any further action, such as leaving X, the spokesperson said "all options are on the table", and that it would support any action taken by Ofcom, the UK's media regulator. Speaking earlier on Friday, Anna Turley, the Labour party chair and a minister without portfolio in the Cabinet Office, said there were no moves as yet for the government to leave X, but individual ministers were considering doing so. She told BBC Radio 4's Today programme: "It's really, really important that we tackle this. Those conversations are ongoing across government. I think all of us in politics are evaluating our use of social media and how we do that, and I know that conversation is happening." Asked if she would leave the site, Turley said: "I've thought about that a lot over the past few months." Asked whether the Labour party would do so, she said: "Those conversations are taking place because it's really important that we make sure that we're in a safe space."

Starmer threatens to 'control' Grok if Elon Musk's X keeps creating sexual images

AI chatbot Grok has been creating images of women and children that are sexualised at the request of users. Sir Keir Starmer has threatened to "control" X's AI chatbot Grok if Elon Musk's social media platform continues to create sexual images of women and children. The prime minister told his MPs the actions of Grok and X are "absolutely disgusting and shameful". "If X cannot control Grok, we will - and we'll do it fast because if you profit from harm and abuse, you lose the right to self regulate," he told a meeting of the Parliamentary Labour Party. Images are being generated by Grok, X's AI tool, that sexualise women and children, manipulating photos of people to remove their clothes or make them pose in suggestive ways. Grok's image creation function has been switched off for all but paying subscribers after a global outcry, but some non-paying users have reported still being able to generate sexualised images of women and children. Sir Keir added on Monday evening: "Protecting their abusive users, rather than the women and children who are being abused shows a total distortion of priorities. "So let me be crystal clear, we won't stand for it, because no matter how unstable or complex the world becomes, this government will be guided by its values. We'll stand up for the vulnerable against the powerful." Downing Street earlier suggested the government was open to ending its use of X if the platform did not act on concerns about its AI chatbot, adding that "all options are on the table". Over the weekend, Mr Musk said the UK government "wants any excuse for censorship" after Sir Keir said X needed "to get a grip" of Grok and Downing Street described the limitation to paid users as "insulting". Just hours before Sir Keir's latest threat, technology secretary Liz Kendall announced she was speeding up laws to make creating non-consensual intimate images with AI a criminal offence. Requesting the creation of the images will also be illegal from this week, she said. Read more: Conservatives pledge to ban social media for under-16s Ex-Tory chancellor Nadhim Zahawi latest Conservative to join Reform UK The Data (Use and Access) Act was passed last year, with sections of the act being implemented slowly, but Ms Kendall said she was speeding it up for the section on AI creation of non-consensual intimate images. She also announced that the Crime and Policing Bill, which is going through parliament, will make it a criminal offence for companies to supply tools designed to create non-consensual internet images. On Monday morning, media watchdog Ofcom launched a formal investigation into Grok, which will look into whether X has "failed to comply with its legal obligations under the Online Safety Act". The regulator said: "There have been deeply concerning reports of the Grok AI chatbot account on X being used to create and share undressed images of people - which may amount to intimate image abuse or pornography - and sexualised images of children that may amount to child sexual abuse material."

Grok's AI-Manipulated Nude Images Highlight the Need for Tough Regulations

If Grok does not fix the issue at the foundational model level, it may be time for us to bid adieu to xAI and to X too Amidst growing global criticism from X users for letting users generate sexualized and nude images of women, Elon Musk's AI company Grok has limited the image-generation feature to only paying subscribers on the microblogging platform. However, these limits do not seem to apply to the Grok app for now. Initially Grok's feature was available to everyone with daily limits whereby users could upload anyone's picture without consent and ask the chatbot to generate an obscene version. Following the worldwide condemnation of the tool, now Musk and his team have limited its use. Is it a case of doing too little, too late? We will have to wait and see. What began in India has spread to other countries now Meanwhile, more countries have joined India in condemning Musk's companies for allowing users to post non-consensual nudity on X (formerly Twitter). New data reveals that the problem is far more serious than anyone imagined. In fact, there could be thousands of such nude images flooding the internet via X (formerly Twitter) every day. The issue first came to light in India when lawmaker Priyanka Chaturvedi wrote to the government about some AI-altered obscene images of celebrities in the country. On its part, the IT ministry asked X to remove such images or face action. But all they got from Musk was a comment that the technology cannot be blamed for how it is used. That companies themselves are talking about guardrails around obscenity and teenage addictions appears lost on the world's richest man. Or maybe he just doesn't care. But, soon he may have to take note as more governments are taking action against this trend of using non-consensually obtained images for voyeurism-led guaranteed engagement. What exactly were users doing with Grok? Several recent media reports suggest that Grok is being asked by people to undress a wider cross-section of people. It goes beyond actresses and models to news anchors and world leaders. However, the worst case of twisted social behaviour users asking for a bikini image of Renee Good, the victim of the recent ICE shooting in Minnesota. A recent research paper from CopyLeaks estimates that roughly an image was being posted each minute on Musk's microblogging platform. However, a report by The Guardian says this number could be much more. However, there is something even more bizarre than the actual number of such posts. The report says now users on X are actually coaching others on what prompts to use, how iterations could get Grok do better with women in lingerie or swimsuits etc. In some cases where women use their own pictures on their account, these perverts ask Grok to do the obscene and then post the image back to the account holders as replies to their posts. When the gamekeeper is also the poacher Why blame them? Recently, Musk himself took to his social media account to share such a post describing how to get the best out of Grok. "Think like movie directors, not typists," he declared loftily while giving details of how to get the best results from his AI chatbot for "a woman walking alone in the rain." He obviously refrained from suggesting anything more macabre in terms of using Grok for voyeuristic pleasure. But, he didn't have to as he gave away all the techniques that the foundational language behind the chatbot uses. Readers must also remember that Musk has challenged India's content regulation in a court claiming takedown powers are equivalent to an administrative overreach. Now, this is where things get interesting. Regulators from across the globe are issuing warnings to the company. While UK's Ofcom said it would do an assessment to determine potential non-compliance issues, the Australian eSafety commissioner said they have witnessed a doubling of Grok-related complaints since late 2025. The sharpest rebuke came from the European Commission which ordered xAI to retain all documents related to its Grok chatbot for a potential investigation. What may have aggravated the matter was a recent CNN report that Elok Musk might have personally intervened to prevent AI safeguards around Grok's image generation actions. Is it time to quit X forever and dump xAI? One may ask why are we making such a big issue of something that's already been on the radar for quite some time. Well, the fact is that governments taking note and seeking action is completely different from public figures around the world decrying the fact that Musk's company might have released a foundational model without safeguards. Though it is not clear whether the Grok model has made any technical changes following the possible legal complications over the issue in India, the United States, Europe and some Far Eastern countries, what is abundantly clear is that AI does require guardrails and such steps must be taken prior to a release. Following the orders from the IT Ministry in India, Musk's team did submit a report to the regulators, but only after the 72-hour deadline was extended by another 48 hours. A report said X confirmed that it would be tightening safeguards on Grok, but for now it seems the chatbot hasn't faced any corrective action as it continues to create fakes. For now, all that Grok says about the matter is that anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content. Of course, one isn't sure what those consequences are? Worst still, Musk and his team doesn't seem to think it worthwhile to fix the problem at the point of origination. And this is evident yet again by Musk's response to the global condemnation. Pay if you want to use Grok to generate nonconsensual nudity on a social platform!

Musk's Grok chatbot restricts image generation after global backlash to sexualized deepfakes - The Korea Times

LONDON -- Elon Musk's AI chatbot Grok is preventing non-paying users from generating or editing images after a global backlash erupted over sexualized deepfakes of people, but the change has not satisfied authorities in Europe. The chatbot, which is accessed through Musk's social media platform X, has in the past few weeks been granting a wave of what researchers say are malicious user requests to modify images, including putting women in bikinis or in sexually explicit positions. Researchers have warned that in a few cases, some images appeared to depict children. Governments around the world have condemned the platform and opened investigations. On Friday, Grok responded to image altering requests with the message: "Image generation and editing are currently limited to paying subscribers. You can subscribe to unlock these features." While subscriber numbers for Grok aren't publicly available, there was a noticeable decline Friday in the number of explicit deepfakes that Grok is now generating compared with just days earlier. Grok was still granting image requests but only from X users with blue checkmarks given to premium subscriber who pay $8 a month for features including higher usage limits for the chatbot. An X spokesperson didn't respond immediately to a request for comment. The restrictions for users save for paying subscribers did not appear to change the opinions of leaders or regulators in Europe. "This doesn't change our fundamental issue. Paid subscription or non-paid subscription, we don't want to see such images. It's as simple as that," said Thomas Regnier, a spokesman for the European Union's executive Commission. The Commission had earlier slammed Grok for "illegal" and "appalling" behavior. The British government was also unsatisfied. Grok's changes are "not a solution," said Geraint Ellis, a spokesman for British Prime Minister Keir Starmer, who on Thursday had threatened unspecified action against X. "In fact, it is insulting to the victims of misogyny and sexual violence," he said, noting that it shows that X "can move swiftly when it wants to do so." "We expect rapid action," he said, adding that "all options are on the table." Starmer, speaking to Greatest Hits radio, had said that X needs to "get their act together and get this material down. We will take action on this because it's simply not tolerable." The U.K.'s media and privacy regulators both said this week they've contacted X and Musk's artificial intelligence company xAI for information on measures taken to comply with British regulations. France, Malaysia and India have also been scrutinizing the platform and a Brazilian lawmaker has called for an investigation. The European Commission has ordered X to retain all internal documents and data relating to Grok until the end of 2026, as part of a wider investigation under the EU's digital safety law. Grok is free to use for X users, who can ask it questions on the social media platform. They can either tag it in posts they've directly created or in replies to posts from other users. Grok launched in 2023. Last summer the company added an image generator feature, Grok Imagine, that included a so-called "spicy mode" that can generate adult content. The problem is amplified both because Musk pitches his chatbot as an edgier alternative to rivals with more safeguards, and because Grok's images are publicly visible, and can therefore be easily spread.

Grok limits image generation to paid users; not enough, says govt

MeitY's concernThe issue will remain if X continues to allow the generation, hosting, uploading or sharing of obscene and illegal content that hurts the privacy and dignity of our citizens, especially women and children, in any manner X has restricted the controversial image generation feature on its AI chatbot Grok to its paying subscribers globally, but the government has warned the move will not be enough to rein in obscene content, and is monitoring it, officials told ET. The Elon Musk-owned social media platform on Friday started stopping free users, who make up the vast majority of its subscribers, from accessing the feature that has been misused to create sexualised images of women and children without consent. Officials noted that allowing obscene, non-consensual content to be created by any user would keep X in violation of existing Indian laws. "We are looking into it," an official from the Ministry of Electronics and Information Technology (MeitY) said. "The issue will remain if it (X) continues to allow the generation, hosting, uploading, or sharing of obscene and illegal content that hurts the privacy and dignity of our citizens, especially women and children, in any manner. We will be seeking details on this from the company." Also, the website of Grok Imagine, X's advanced tool for user-made cinematic-quality content, continues to allow free users to generate images, officials noted. Last Friday, the ministry had asked X to remove all vulgar, obscene and unlawful content, especially those generated by Grok, on the platform within 72 hours and take action against offending users. The deadline was subsequently extended by 48 hours, after the company sought more time. On Wednesday, X informed the government that it is introducing more guardrails to its AI-powered chatbot Grok and refining safeguards such as stricter image generation filters to minimize abuse of user images, ET had reported. "While examining X's response on actions taken by it to curb the spread of obscene content, a lack of information has been encountered on how exactly they plan to stop the spread (of unlawful content). The intermediary has been subsequently asked to furnish more technical details," the official quoted above said. Embedded into both X and the standalone Grok app, the Grok Imagine feature has led to an explosion of pornographic material, often violent, since its free global rollout in August last year. It has faced backlash from nations including India, Turkiye, Malaysia, United Kingdom and Brazil, as well as the European Union. Spicy Mode, a specific setting within Grok Imagine designed to generate more expressive, bold and mature content, has also been criticised.

X Announces Grok Can No Longer Edit Real People's Images, But Website Still Allows It - MEDIANAMA

The X Safety account announced today that images of real people can no longer be edited by Grok on X, stating: "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis. This restriction applies to all users, including paid subscribers. Additionally, image creation and the ability to edit images via the Grok account on the X platform are now only available to paid subscribers." Furthermore, the company also announced a geoblock update, stating that it now blocks all users in jurisdictions where it is illegal to generate images of real people in bikinis, underwear, and similar attire via the Grok account and in Grok on X. The update follows global scrutiny over Grok's image generation and editing tools on X, which users have been prompted in posts and replies to create sexually suggestive and manipulated images of real people since late December 2025. As part of the changes, X has positioned the paywall and jurisdiction-based restrictions as safeguards aimed at limiting misuse, ensuring accountability, and improving enforcement of its policies. However, MediaNama tested whether the same restrictions apply on the Grok website. We found that Grok continued to respond to requests to digitally alter images of real people in its 'Imagine' image editing section. During testing, MediaNama uploaded a publicly available image of a celebrity and entered prompts such as "take her top off," "put her in a skimpy bikini," and "turn her around and take her clothes off." In each case, Grok proceeded with the requests. The system intervened only when prompted to generate complete nudity. Globally, authorities have taken a range of actions in response to concerns about Grok's widespread use to generate non-consensual and sexualised images of real people. Indonesia and Malaysia were among the first to act, temporarily blocking access to Grok due to its ability to produce sexually explicit deepfakes. Furthermore, Malaysian authorities announced that they will take legal action against X, saying that the company did not take any effective action. In Europe and the United Kingdom (UK), regulators have escalated scrutiny. The UK communications regulator, Ofcom, opened an investigation into whether X and Grok had broken British law by allowing the creation and sharing of non-consensual sexual images, including those of children. In Europe, the European Commission (EC) ordered X to retain internal documents and data related to Grok until the end of 2026 while it assessed compliance with EU rules on digital safety. Other European countries, including France, Italy, and Germany, have also announced inquiries or measures to address harmful AI image manipulation linked to Grok. In the United States (US), California's Attorney General, Rob Bonta, launched an investigation into xAI and Grok for enabling the creation and distribution of non-consensual deepfake sexual images, including those involving minors. This follows a letter by three US senators to Apple and Google asking for the Grok and X apps to be removed from their app stores. In India, the Ministry of Electronics and Information Technology (MeitY) issued a notice to X in early January 2026, instructing the company to remove vulgar, obscene, and unlawful content generated through Grok and to submit a detailed report on the actions taken within 72 hours. MeitY warned that failure to comply could lead to serious legal consequences, including the loss of safe harbour protections under Indian law. X has positioned its latest restrictions on Grok's image editing tools as a decisive response to global backlash over the creation of non-consensual and sexualised images of real people. However, even though X has limited these capabilities on its platform, the same safeguards do not apply on the Grok website. As a result, users can still upload photographs of real individuals and prompt the system to alter their appearance in sexualised ways digitally. This gap is significant because it indicates that the underlying image generation and editing system continues to operate with minimal restrictions outside X. While the company has introduced paywalls, geoblocks, and platform-specific controls, the core capability remains easily accessible through the standalone Grok website. Consequently, the changes appear focused on managing visibility and reputational fallout on X rather than preventing the generation of harmful content at the source. Against this backdrop, the persistence of these features raises fresh questions about whether X and xAI are prioritising effective safeguards or responding primarily to mounting public and regulatory pressure.

Gov. Gavin Newsom demands AG investigate Elon Musk's X for AI...

California Governor Gavin Newsom is calling on the state's attorney general to investigate Elon Musk's social media platform, X, over a disturbing new trend where user's ask AI to make fake images depicting sometimes sexually explicit content. "xAI's decision to create and host a breeding ground for predators to spread nonconsensual sexually explicit AI deepfakes, including images that digitally undress children, is vile," Newsom said in social media post Wednesday. X's AI chatbot, dubbed Grok, has come under fire in recent weeks for creating sexualized fake images of women and children, prompting both Indonesia and Malaysia to temporarily block the platform. "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," California Attorney General Rob Bonta said in a statement. "As the top law enforcement official of California tasked with protecting our residents, I am deeply concerned with this development in AI and will use all the tools at my disposal to keep California's residents safe." Musk took to X to hit back on the claims prior to Newsom's tweet Wednesday. "I not aware of any naked underage images generated by Grok. Literally zero. Obviously, Grok does not spontaneously generate images, it does so only according to user requests," Musk wrote. "When asked to generate images, it will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state. There may be times when adversarial hacking of Grok prompts does something unexpected. If that happens, we fix the bug immediately."

Elon Musk's Grok appears to modify image generation after outcry

Elon Musk's Grok artificial intelligence chatbot appears to have new guardrails around image generation, following global outrage after it was found to be complying with user requests to digitally undress images of adults and in some cases children. Within the last week xAi, which owns both Grok and the social media platform X, restricted image generation for Grok on X to paying X premium subscribers. But according to researchers and CNN's own observations in recent days, Grok's X account has modified how it responds in general to user's image generation requests, even for those subscribed to X premium. According to researchers at Copyleaks, an AI detection and content governance platform, Grok is no longer responding as often to image requests even from premium users, sometimes describing a scenario rather than creating an image or sometimes fulfilling a request in "a more generic or toned-down way, rather than using the specific subject originally requested," the group found. "Overall, these behaviors suggest X is experimenting with multiple mechanisms to reduce or control problematic image generation, though inconsistencies remain," Copyleaks found. Researchers at AI Forensics, a European non-profit that investigates algorithms, said that the creation of bikini-like images has seemingly decreased at X, according to a spokesperson for the group. But the group said they have also observed "inconsistencies in the treatment of pornographic content generation" between public interactions with Grok on X and private chat on Grok.com. xAI, which did not respond to a request for comment, has previously stated via the company's Safety account that they "take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary. Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content." On Wednesday Musk said in a post on X that he was "not aware of any naked underage images generated by Grok. Literally zero." Grok "will refuse to produce anything illegal, as the operating principle for Grok is to obey the laws of any given country or state," he added. However, researchers said that while fully nude images were rare, the biggest issue was Grok complying with user requests to modify images of minors and place them revealing clothing, including bikinis and underwear, as well as in sexually provocative positions. Creators of those types of non-consensual intimate images could still be subject to criminal prosecution of Child Sexual Abuse Material and are potentially subject to fines and prison time under the Take it Down Act, signed last year by President Donald Trump. On Wednesday, California Attorney General Rob Bonta announced an investigation into the "proliferation of nonconsensual sexually explicit material produced using Grok." Grok is still banned in Indonesia and Malaysia as a result of the image generation controversy. UK regulator Ofcom announced Monday it has launched a formal investigation of X, although Prime Minister Keir Starmer's office said Wednesday he welcomes reports X is addressing the issue.

UK ministers considering leaving X amid concern over AI tool images

Labour party chair says government having conversations about use of platform in light of sexualised Grok images UK ministers are considering leaving X as a result of the controversy over the platform's AI tool, which has been allowing users to generate digitally altered pictures of people - including children - with their clothes removed. Anna Turley, the chair of the Labour party and a minister without portfolio in the Cabinet Office, said on Friday that conversations were happening within the government and Labour about their continued use of the social media platform, which is controlled by Elon Musk. The government has come under mounting pressure to leave X after the site was flooded with images including sexualised and unclothed pictures of children generated by its AI tool, Grok. Turley told BBC Radio 4's Today programme: "X, first and foremost, has to get its act together and prevent this. It has the powers to do this, and we need to make sure there are firm consequences for that. She added: "It's really, really important that we tackle this. Those conversations are ongoing across government. I think all of us in politics are evaluating our use of social media and how we do that, and I know that conversation is happening." Asked if she would personally leave the site, Turley said: "I've thought about that a lot over the past few months." And asked whether the Labour party would do so, she added: "Those conversations are taking place because it's really important that we make sure that we're in a safe space." On Friday, X said it was limiting the use of Grok's image creating tool to paid users only. The government has so far resisted calls to stop using the social media platform, focusing instead on the powers that the media regulator Ofcom has to take action against X under the Online Safety Act. Those powers include preventing the company having access to certain technology and funding, which could amount to a de facto ban in the UK. Keir Starmer, the prime minister, said on Thursday: "X has got to get a grip of this. And Ofcom has our full support to take action in relation to this. This is wrong. "It's unlawful. We're not going to tolerate it. I've asked for all options to be on the table." Some prominent MPs and committees have announced they will stop using X, including the women and equalities committee, whose chair Sarah Owen said this week the site was "not an appropriate platform to be using for our communications". Louise Haigh, the former transport secretary, on Thursday called for the government to leave the platform, saying it would be "unconscionable" to use it "for another minute". Others, however, are urging the government to remain on the site, which says it has more than 500 million monthly active users and remains one of the biggest social media platforms in the world. James Lyons, a former director of communications to Starmer, told the PoliticsHome podcast this week: "I take the view that your job in political communication is to persuade people. "And to persuade people, you have to engage, and I think you should be using all the platforms and forums that you can to do that." None of the major parties have yet left the site. Asked this week whether he would stop taking payments from X for his posts, the Reform UK leader, Nigel Farage, declined to answer, saying he was "very worried" about the images on the site but believed the company would listen to criticism.

'Get a grip' on Grok, Starmer tells X after AI tool is used for child sex images

An artificial intelligence (AI) tool producing sexualised images of children will not be tolerated and is "disgusting" and "unlawful", the prime minister has said. Sir Keir Starmer said social media platform X has "got to get a grip of" its AI tool, Grok, and that he's asked media regulator, Ofcom, for "all options to be on the table". It follows reports from the Internet Watch Foundation (IWF) that criminals have been using Grok to create child sexual abuse imagery. Politics live - follow latest Grok is an AI tool that X users can instruct to find out information, answer questions and create images. The IWF revealed this week it had discovered criminal sexualised imagery of children aged between 11 and 13 that had been created by Grok. Speaking to Greatest Hits Radio on Thursday, the prime minister said: "This is disgraceful. It's disgusting. And it's not to be tolerated. "X has got to get a grip of this. And Ofcom has our full support to take action in relation to this. This is wrong." X and xAI - which produces Grok - are both owned by tech billionaire Elon Musk, and have been under fire for a number of days after a new feature led to users seeing AI-generated sexualised images of themselves on X. Sir Keir added: "It's unlawful. We're not going to tolerate it. I've asked for all options to be on the table. "It's disgusting. And X need to get their act together and get this material down. And we will take action on this because it's simply not tolerable." Earlier in the day, former Labour cabinet minister Louise Haigh urged the PM and the rest of government to quit the social media platform. She said: "I call on my party and my government to remove themselves entirely from X and communicate with the public where they actually participate online and can be protected from such illegality." Asked whether the government could stop using X on Wednesday, a Downing Street spokesperson said: "All options are on the table." They replied the same way when asked whether the prime minister was accepting images like this by continuing to use X. But the spokesperson added that Ofcom had their "full backing to take action on failings by firms". Ofcom has asked X to clarify how it is complying with data protection law over the AI images. Technology Secretary Liz Kendall called on X to take "urgent" action earlier this week. Sky News has contacted X for comment. The site's Safety account earlier this week read: "We take action against illegal content on X, including Child Sexual Abuse Material (CSAM), by removing it, permanently suspending accounts, and working with local governments and law enforcement as necessary. "Anyone using or prompting Grok to make illegal content will suffer the same consequences as if they upload illegal content."

Elon Musk's xAI under investigation by California AG over explicit AI images By Investing.com

Investing.com -- California Attorney General Rob Bonta launched an investigation Wednesday into xAI and its AI model Grok over allegations the technology is being used to create non-consensual sexually explicit images of women and children. The investigation follows reports that users have been taking ordinary images and using Grok to "undress" subjects or place them in sexually explicit scenarios without consent. According to one analysis cited by the Attorney General's office, more than half of 20,000 images generated by xAI between Christmas and New Year's depicted people in minimal clothing, with some appearing to be children. "The avalanche of reports detailing the non-consensual sexually explicit material that xAI has produced and posted online in recent weeks is shocking," Bonta said in a statement. "I urge xAI to take immediate action to ensure this goes no further." Bonta expressed particular concern about Grok's "spicy mode," which generates explicit content and has been used as a marketing point for the company. Elon Musk, founder of xAI, denied knowledge of any underage explicit images generated by Grok, writing on X: "I not aware of any naked underage images generated by Grok. Literally zero." He added that Grok only generates images based on user requests and "will refuse to produce anything illegal."

California to investigate xAI over Grok chatbot images, officials say

California's governor and attorney general said on Wednesday that they were demanding answers from Elon Musk's xAI after the billionaire said he was not aware of any "naked underage images" generated by its artificial intelligence chatbot, called Grok. "We're demanding immediate answers from xAI on their plan to stop the creation & spread of this content," California's attorney general, Rob Bonta, wrote on X. Earlier, Governor Gavin Newsom wrote a post calling on Bonta "to immediately investigate the company and hold xAI accountable." The comments from Newsom and Bonta were the most serious so far by any U.S. official addressing the explosion of AI-generated nonconsensual sexualized imagery on Musk's X platform. In Europe and Asia, several officials have expressed outrage and disgust. Early this year, hyper-realistic images of women processed to look like they were in microscopic bikinis, in degrading poses, or covered in bruises flooded X, the site formerly known as Twitter. In some cases, minors were digitally stripped down to swimwear. At first, Musk publicly laughed off the controversy, posting humorous emojis in response to other users' comments about the influx of sexualized photos. More recently, X has said it treats reports of child sexual abuse material seriously and polices it vigorously. On Wednesday, Musk said he was not aware of any "naked underage images" generated by Grok. "I not aware of any naked underage images generated by Grok. Literally zero," Musk said in an X post. X did not immediately respond to questions about the California announcement and Musk's comments. xAI did not answer questions from Reuters about the announcement by California officials or Musk's statement that he was unaware of sexualized imagery of minors. The company responded to the questions with an email re-stating its generic reply to press inquiries: "Legacy Media Lies." The California move adds to the pressure Musk is facing in the U.S. and around the world. Lawmakers and advocacy groups have called for Apple and Google to drop Grok from app stores. Government officials have threatened action in Europe and the United Kingdom, while bans on Grok are already in place in Malaysia and Indonesia. Last week, X curtailed Grok's ability to generate or edit images publicly for many users, but Grok was still privately producing sexually charged images on demand as of Wednesday, Reuters found.

Musk's Grok barred from undressing images after global backlash - VnExpress International

The announcement comes after California's attorney general launched an investigation into Musk's xAI -- the developer of Grok -- over the sexually explicit material and multiple countries either blocked access to the chatbot or launched their own probes. X said it will "geoblock the ability" of all Grok and X users to create images of people in "bikinis, underwear, and similar attire" in those jurisdictions where such actions are deemed illegal. "We have implemented technological measures to prevent the Grok account from allowing the editing of images of real people in revealing clothing such as bikinis," X's safety team said in a statement. "This restriction applies to all users, including paid subscribers." In an "extra layer of protection," image creation and the ability to edit photos via X's Grok account was now only available to paid subscribers, the statement added. The European Commission, which acts as the EU's digital watchdog, earlier said it had taken note of "additional measures X is taking to ban Grok from generating sexualised images of women and children." "We will carefully assess these changes to make sure they effectively protect citizens in the EU," European Commission spokesperson Thomas Regnier said in a statement, which followed sharp criticism over the nonconsensual undressed images. 'Shocking' Global pressure had been building on xAI to rein in Grok after its so-called "Spicy Mode" feature allowed users to create sexualized deepfakes of women and children using simple text prompts such as "put her in a bikini" or "remove her clothes." "The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," California Attorney General Rob Bonta said earlier Wednesday. "We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material." Bonta said the California investigation would determine whether xAI violated state law after the explicit imagery was "used to harass people across the internet." California Governor Gavin Newsom said that xAI's "vile" decision to allow sexually explicit deepfakes to proliferate prompted him to urge the attorney general to hold the company accountable. Further adding pressure onto Musk's company Wednesday, a coalition of 28 civil society groups submitted open letters to the CEOs of Apple and Google, urging them to ban Grok and X from their app stores amid the surge in sexualized images. Indonesia on Saturday became the first country to block access to Grok entirely, with neighboring Malaysia following on Sunday. India said Sunday that X had removed thousands of posts and hundreds of user accounts in response to its complaints. Britain's Ofcom media regulator said Monday it was opening a probe into whether X failed to comply with U.K. law over the sexual images. And France's commissioner for children Sarah El Hairy said Tuesday she had referred Grok's generated images to French prosecutors, the Arcom media regulator and the European Union. Last week, an analysis of more than 20,000 Grok-generated images by Paris non-profit AI Forensics found that more than half depicted "individuals in minimal attire" -- most of them women, and 2% appearing to be minors.

Elon Musk's X says Grok no longer generates undressed images, but it is still doing it

Critics say X's safeguards lag behind rivals as pressure mounts over AI misuse and platform accountability. Elon Musk's X and Grok have been facing massive criticism and regulatory pressure as the AI chatbot continues to be used to generate non-consensual sexual deepfakes, despite the company's claims of introducing safeguards. Recent testing and independent investigations indicate that X's attempts to limit Grok's image-editing capabilities were ineffective, with users still able to easily create sexualised images. Previous measures by X focused on limiting the image generation via public replies, particularly from free users. However, access to Grok's image-editing tools remains largely unrestricted through the chatbot interface and its standalone website. More recently, reports indicated that Grok had been updated to refuse prompts involving women in explicit or sexualised scenarios. Yet tests, as per The Verge, showed that these restrictions can be bypassed, with the AI still responding to altered prompts and producing suggestive images. According to investigations, while Grok now rejects direct requests for full nudity, it continues to comply with cues that sexualise subjects through garment changes, exaggerated physical traits, or provocative stances. These behaviours allegedly do not require a paid subscription, and age verification systems, where available, can simply be avoided. In certain circumstances, there are no age checks at all on X's platforms. Also read: Amazon Great Great Republic Day Sale: Motorola Razr 50 Ultra price drops by over Rs 39,000 The controversy has been met with regulatory pressure, with Malaysia and Indonesia temporarily restricting access to Grok, while UK lawmakers have accelerated legislation targeting non-consensual deepfake imagery. Additionally, British authorities are investigating whether X violated the country's Online Safety Act, which criminalises the creation and distribution of intimate images without consent, regardless of nudity. On the other hand, Elon Musk has denied the allegations that Grok generates illegal content, arguing that the AI only responds to user prompts and is designed to comply with local laws. However, experts and watchdog groups dispute this position. While the entire internet is screaming for new rules, regulations and privacy along with consent, it still remains to be seen how the platform will restrict itself for making it safe.

Elon Musk denies Grok AI created illegal images, blames adversarial hacks

Hints at political motives behind Grok image scandal on X.com In his continued effort to champion free speech for the world, Elon Musk has once again taken to the digital frontline in defence of Grok, his beloved AI chatbot. As allegations continue to mount accusing Grok of generating illicit images of underage individuals, Musk is countering the claim with a familiar mixture of denial, deflection, and political undertone. "I'm not aware of any naked underage images generated by Grok. Literally zero," Musk stated in response to a viral post questioning why certain UK Labour MPs claim to be seeing child sexual abuse content on X.com. If you read the entire tweet below, you will see how Musk's reply wasn't just a denial but a careful recalibration of responsibility, aimed squarely at users misusing the system. Or, in Musk's words, "hacking" it on rare occasions. As we all know by now, Grok AI (on X.com) is only designed to generate text and images based on user prompts. Grok "does not spontaneously generate images," Elon Musk was quick to assert that any output generated by Grok is fully dependent on user input. If someone asks it to do something illegal, it will, in theory, refuse. "The operating principle for Grok is to obey the laws of any given country or state," Musk tweeted, as if the rulebook alone is enough to contain the Internet's darker shades. However, Elon Musk did admit that there "may be times when adversarial hacking of Grok prompts does something unexpected." The implication of this is always a bug, Musk clarified - an unintentional result of clever prompt engineering. And bugs, Musk assures, are fixed immediately. But the heart of Musk's argument isn't just a technical one. He's hitting back at what he thinks is a political campaign against him and Grok. In his usual style, he's retweeted users implying that the recent scrutiny around Grok and explicit image generation isn't an organic moral panic but a political witch-hunt. A trend, perhaps, manufactured by political entities that don't like Musk's self-styled commitment to "free speech" absolutism. This defence strategy - technical rebuttal fused with cultural and political undertones - is so textbook Elon Musk. Don't blame the code, blame the codebreakers and code abusers. And if all else fails, blame the government. Needless to say, Grok and X.com are still under fire, which is what prompted Elon Musk to tweet his defence. Just a couple of weeks ago, X.com users started prompting Grok to alter images of people into bikinis. At first, the trend started with just a few funny tweets. Heck, even Elon Musk himself jumped into the trend, as he asked Grok to generate a picture of himself in a bikini. It was all fun and games until many users targeted women the most by asking Grok to bikini-fy several images of women. If all that wasn't enough, Grok even performed the same prompt on two minor girls, which set this entire controversy off. As this controversy escalated without X, Grok or Elon Musk showing any signs of ending it, government authorities from multiple countries stepped in. Countries like Britain pulled up X, asking them to comply with legal duties under their Online Safety Act. The Indian government acted fast, too. Indonesia and Malaysia went one step further to flat out ban X.com in their countries. All because of Grok's deepfake image generation capabilities. But in Musk's world, where bugs are just the price of progress and critics often wear political badges, this is just a skirmish in a broader ideological war. And Grok? For now, it continues to be an unfortunate case study in morality, ethics and responsibility in this fast advancing age of AI.