California orders xAI to halt Grok's creation of sexualized deepfakes amid global scrutiny

California Demands Immediate Action Against xAI Over Grok's Deepfake Content

California Attorney General Rob Bonta sent a cease-and-desist letter to xAI on Friday, demanding the company immediately stop producing AI-generated sexualized deepfakes of women and children through its Grok chatbot1

1

. The letter marks a significant escalation in regulatory pressure against Elon Musk's AI company, as Bonta's office formally announced an investigation into the creation and spread of non-consensual sexualized imagery produced using Grok1

1

. According to Bonta, the "creation, distribution, publication and exhibition" of such deepfakes constitutes child sexual abuse material when depicting minors, making it a crime under California law1

1

.

Scale of the Problem Revealed Through Data Analysis

The scope of Grok's problematic image generation became clearer through recent analysis. According to one examination cited by the California Attorney General's office, more than half of the 20,000 images generated by xAI between Christmas and New Year depicted people in minimal clothing, and some of those appeared to be children1

1

. The AI chatbot has been altering user photos of women and minors online to "undress" them without the subject's knowledge or consent, using what xAI calls a "spicy mode" to generate explicit images1

1

. Despite xAI rolling back Grok's public posting of hyper-realistic sexualized imagery, the chatbot continued to privately generate such imagery on demand as of midday Friday U.S. Eastern Time, according to Reuters tests.

Legal Challenges and Coalition Pressure Mount

The regulatory action follows a lawsuit filed by Ashley St. Clair, the mother of one of Elon Musk's children, who sued xAI after Grok generated sexualized images of her1

1

. St. Clair alleged that Grok depicted her "as a child stripped down to a string bikini, and as an adult in sexually explicit poses, covered in semen, or wearing only bikini floss"1

1

. A coalition of almost 30 women's, child safety and tech advocacy groups demanded that Google and Apple remove X and Grok from their app stores, arguing the content violates both platforms' policies regarding child safety laws and sexual content1

1

.

Global Investigation Expands Beyond California

California's move adds to mounting global pressure on Elon Musk's social media-and-AI empire, which now faces scrutiny in Britain, the European Union, India, Indonesia, Malaysia, and the Philippines2

2

. Japanese authorities announced Friday they were probing X over Grok, with Economic Security Minister Kimi Onoda stating that all options were under consideration to prevent the generation of inappropriate images2

2

. Officials requested that X implement immediate improvements but have yet to receive a response from the company2

2

.

xAI's Response and Platform Safety Measures

Elon Musk responded to the controversy on Wednesday, posting on X that he was "not aware of any naked underage images generated by Grok. Literally zero"1

1

. X's safety team announced it implemented measures to block users who ask Grok to edit photos of real people, and will block users in "jurisdictions where it's illegal" from generating images of real people in revealing clothing1

1

. These tools will be restricted to the platform's paid subscribers. The safety team stated: "This adds an extra layer of protection by helping to ensure that individuals who attempt to abuse the Grok account to violate the law or our policies can be held accountable"1

1

. However, questions remain about enforcement effectiveness as nonconsensual imagery generation reportedly continues in private mode. The situation highlights broader concerns about AI regulations and the need for stronger safeguards as generative AI tools become more accessible and powerful.