California orders xAI to halt Grok's creation of sexualized deepfakes amid global scrutiny

California sends xAI cease-and-desist letter over sexualized deepfakes

California Attorney General Rob Bonta (D) on Friday sent a cease-and-desist letter to xAI demanding that it stop producing artificial intelligence-generated sexualized deepfakes of girls and women online, including on the social platform X. Bonta said in a statement that the "creation, distribution, publication and exhibition" of deepfakes of girls is child sexual abuse material and therefore a crime, and that these business practices violate California's civil laws. "This week, my office formally announced an investigation into the creation and spread of nonconsensual, sexually explicit material produced using Grok, an AI model developed by xAI," Bonta said in the statement. "The avalanche of reports detailing this material -- at times depicting women and children engaged in sexual activity -- is shocking and, as my office has determined, potentially illegal." He said he expects xAI to comply with the letter immediately, and that "California has zero tolerance for child sexual abuse material." Bonta's office noted of several news reports regarding Grok, the AI program used on X, altering photos of women and children online to "undress" them without the subject's knowledge or consent. The AI program uses what xAI calls a "spicy mode" to generate explicit images. "Use of Grok for these purposes appears to be happening on a large scale," the statement read. "According to one analysis, more than half of the 20,000 images generated by xAI between Christmas and New Year's depicted people in minimal clothing, and some of those appeared to be children." The Hill has reached out to xAI for comment. xAI is owned by Tesla CEO and former White House official Elon Musk. On Wednesday, he posted on X that he was "not aware of any naked underage images generated by Grok. Literally zero." Outrage grew as a coalition of almost 30 women's, child safety and tech advocacy groups told Google and Apple to remove X and Grok from their app stores. The coalition argued that the content violates Apple's policies for apps to comply with banning defamatory or overtly sexual content, and Google's policies blocking apps that depict the sexualization of children and other sexual content. Ashley St. Clair, the mother of one of Elon Musk's children, also sued xAI after Grok generated sexualized images of her. She alleged that Grok depicted her "as a child stripped down to a string bikini, and as an adult in sexually explicit poses, covered in semen, or wearing only bikini floss." X's safety team said it implemented a measure to block users who ask Grok to edit photos of real people. The platform will also block users in "jurisdictions where it's illegal" from generating images of real people in revealing clothing. These tools will be restricted to the platform's paid subscribers. "This adds an extra layer of protection by helping to ensure that individuals who attempt to abuse the Grok account to violate the law or our policies can be held accountable," X's safety team account wrote in a post on the platform. "We remain committed to making X a safe platform for everyone and continue to have zero tolerance for any forms of child sexual exploitation, non-consensual nudity, and unwanted sexual content."

California AG sends letter demanding xAI stop producing deekfake content

California ⁠Attorney General Robert Bonta on Friday sent a cease and desist letter to xAI demanding the company stop the creation and distribution of AI-generated non-consensual sexualised imagery by its Grok AI chatbot. California Attorney General Robert Bonta on Friday sent a ⁠cease-and-desist letter to xAI demanding the company stop the creation and distribution of artificial intelligence-generated nonconsensual sexualized imagery by its Grok chatbot. "The avalanche of reports detailing this material - at times depicting women and children engaged in sexual activity - is shocking and, as my office has determined, potentially illegal," said Bonta. xAI, which is owned by billionaire Republican megadonor Elon Musk, has come under global scrutiny over the past couple of weeks after Grok began flooding the social media site X with nonconsensual images of women and some minors in revealing clothes and degrading poses. Although xAI has since rolled back Grok's public posting of hyper-realistic sexualized imagery, sometimes called deepfakes, the chatbot privately generates such imagery on demand as of midday Friday U.S. Eastern Time, according to Reuters tests. X and xAI did not immediately address questions about the cease-and-desist letter. California's move adds to the global pressure on Musk's social media-and-AI empire, which is already under scrutiny in Britain, the European Union, and other jurisdictions, including India, Indonesia, ⁠Malaysia and ⁠the Philippines. Earlier Friday, Japanese authorities said they too were probing X over Grok, saying all options were under consideration to prevent the generation of inappropriate images. Economic Security Minister Kimi Onoda said officials had requested that X implement immediate improvements but they have yet to receive a response from the company.

California Issues Immediate Order as Three Countries Ban Elon Musk's Grok Over Sexual Deepfake Images

Grok Scandal Puts Elon Musk's xAI Under Fire with Lawsuits Concerning Child Safety Laws and AI Regulations California's Attorney General has ordered Elon Musk's AI company, xAI, to immediately stop its Grok chatbot from creating sexualized deepfake images. The warning says making such images of women and children without consent is illegal. Reports revealed that Grok could take user photos and turn them into explicit or sexual images, even of minors. The Attorney General called this "shocking" and said xAI must stop immediately. California law treats child sexual abuse material very seriously, with zero tolerance.

California AG sends letter demanding xAI stop producing deekfake content

Jan 16 (Reuters) - California Attorney General Robert Bonta on Friday sent a cease and desist letter to xAI demanding the company stop the creation and distribution of AI-generated non-consensual sexualized imagery by its Grok AI chatbot. The letter demands "immediate action to stop the creation and distribution of deepfake, nonconsensual, intimate images and child sexual abuse material," Bonta's office said in a statement. (Reporting by Rafael Satter and Ryan Patrick Jones, Editing by Franklin Paul)