xAI sues Grok user for creating CSAM as Elon Musk's firm faces mounting legal pressure

xAI can't deny Grok makes CSAM anymore. So it's suing users.

Facing mounting pressure to acknowledge that Grok can still be used to generate non-consensual sexualized images of adults and minors, xAI filed a lawsuit Tuesday, suing the first user that Elon Musk's firm has accused of using its chatbot to create illegal content. The complaint targets Terry Wayne Harwood, who was arrested earlier this year for possession and distribution of child sexual abuse materials (CSAM), the South Carolina attorney's office announced. As xAI alleged, the company assisted in that arrest after discovering that Harwood had been using two xAI accounts for months to undress or "nudify" non-sexual images of multiple victims, including a young girl who appeared to be as young as 10. xAI's lawsuit comes a little more than a week after a different young girl joined a proposed class action representing several kids allegedly harmed by Grok. She alleged that her stepfather committed suicide after he was discovered using Grok, possibly in conjunction with other AI tools, to create 7,000 sexualized images of her and then distribute them on the dark web. In that case, the victim alleged that xAI refused to help police identify the user who uploaded her image to Grok. To support claims that it is xAI's common practice, her lawyers cited a 2026 National Center for Missing & Exploited Children (NCMEC) report confirming that 90 percent of xAI's CyberTipline reports "were not actionable by law enforcement because xAI declined to include user information that would allow law enforcement to track and locate perpetrators." As victims accused X of shielding predators, Musk had previously maintained that he had not seen any examples of Grok-generated CSAM. Rather than move to restrict Grok's outputs to make CSAM outputs impossible, Musk warned users to act responsibly, posting on X on January 3 that "anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content." While Musk was posting through it, though, Harwood was allegedly ignoring Musk's warning and prompting Grok as many times as it took to get the chatbot to generate likely nonconsensual explicit images. Now Musk seemingly can't deny that Grok makes CSAM. xAI's lawsuit claimed that Harwood used at least two accounts with convoluted usernames -- "ceae2cb4-a9f6-4885-8ae9-6e2096d084f4" and "befccb94-4029-454d-9f1f-0d4945e8fa7c" -- to generate illegal content from December 8 to February 18. Sometimes Grok safeguards did prevent harmful outputs, "refusing to follow" some of the prompts "on the basis that such material violated Grok's content moderation guardrails." The lawsuit includes an example of an especially creepy prompt that Grok rejected, using phrases like "white slime" to mask intent to generate sexualized images. That particular prompt may have been rejected for an obvious reason, though. Harwood explicitly asked the chatbot to "remove all her clothing," which directly violates xAI user terms against requests to undress real people. In the proposed class action, it's alleged that xAI overlooks a lot of bad requests, only reporting to NCMEC one prompt to depict "gang rape" out of 7,000 harmful outputs in the most recent victim's case. Likely to avoid other bad actors circumventing safeguards, xAI did not include examples of Harwood's successful prompts or describe methods used to bypass filters. xAI only alleged that Harwood modified prompts to get around safeguards "in clear violation of the xAI Terms of Service and of US law," including "some" requests for "obscene" images involving the "likeness of minor children." A spokesperson for the South Carolina attorney general's office told Ars that Harwood's case is still pending. Specifically, he has been charged with "distributing, transporting, exhibiting, receiving, selling, purchasing, exchanging, or soliciting CSAM that was 'through the use of an artificial intelligence platform.'" The spokesperson was not authorized to verify if Grok was the platform used. However, xAI's complaint alleged that "upon information and belief, at least some of the images at-issue in the Harwood Criminal Action were generated or altered through Defendant's violative use of Grok." xAI did not immediately respond to Ars' request to comment. xAI's plan to blame users In the lawsuit, xAI makes its case for why only users should be liable for Grok-generated CSAM. If the court agreed, such a finding could strengthen xAI's defense in the looming potential class action, which lawyers estimate could involve thousands of victims. It could also give xAI a hammer to bring down on other users any time a victim comes forward with a complaint. xAI said that it was partly suing Harwood for breach of contract to avoid "substantial legal fees" and the "risk of considerable liability for damages" that may be sought if any of his alleged victims sues xAI over his harmful Grok use. Musk's firm argued that Grok should be viewed by the court as "a neutral tool, subject to user control." "Like any generative AI tool, every response, every image, every generation is the result of the user's prompts and directions," xAI argued. The complaint emphasized that xAI's terms of use draw "a bright line" between safe Grok uses and those that are harmful, and every user agrees to the terms upon sign-up. As relevant in Harwood's case, Grok users are prohibited from using Grok to "undress or nudify real persons," or otherwise using the chatbot to alter "a real person's image or likeness to depict them in an intimate or sexual context," xAI's terms said. Users are also banned from "depicting likenesses of persons in a pornographic manner" or "sexualizing or exploiting children." It's further noted that any CSAM uncovered by xAI is reported to NCMEC. In its complaint, xAI claimed that Harwood alone is responsible for his outputs because he "flagrantly violated" xAI's rules and "went to great lengths to circumvent" Grok's "technological safeguards." Harwood allegedly did this by relying on "misleading prompts," xAI said. And Harwood also failed to police himself once he saw that he could generate illegal content, xAI argued. In the complaint, xAI alleged that Harwood should have known that he was banned from using Grok after the first time he relied on the chatbot to make illegal content. Glaringly, though, xAI does not indicate that Harwood received any warnings that his account risked penalties. Instead, Harwood allegedly "continued to use Grok during the Relevant Period after violating the xAI Terms of Service," xAI argued. "The xAI Terms of Service to which he agreed prohibited his use after his prior violations." xAI is hoping the US district court will rule that Harwood violated xAI's terms and breached his contract with xAI. But perhaps more importantly, Musk wants the court to recognize an indemnity clause that holds that only users -- and not xAI -- are liable for Grok-generated CSAM and NCII. According to xAI, when people use Grok, they are responsible for all of their content, which xAI insisted includes both inputs and outputs. Whether the court will agree that users are responsible for AI outputs has yet to be seen. Perhaps notably, the Copyright Office does not view AI outputs as human-created. That could throw a wrench in xAI's offense, if the court struggles to see how child sex images generated by an AI tool could be created by the user if any other image could not be legally credited that way. If xAI wins this fight, Harwood could owe substantial damages, including damages for "any real harm to third parties," xAI's "exposure to potential third-party claims and lawsuits," and "any xAI reputational harm," the complaint said.

xAI sues a man for using Grok to generate CSAM 'deepfakes'

The Elon Musk-owned xAI is suing a South Carolina man who allegedly used the company's Grok AI chatbot to generate child sexual abuse material (CSAM). In a lawsuit reported earlier by Reuters, xAI claims Terry Wayne Harwood "knowingly and intentionally used Grok to circumvent safeguards, alter nonconsensual images, and generate and distribute CSAM," breaching the company's policies. Harwood was arrested in February for allegedly possessing and distributing CSAM and is facing eight felony charges. The lawsuit claims "at least some" of the images related to Harwood's criminal charges "were generated or altered" with Grok, and that Harwood bypassed Grok's safeguards and "abused the tool to convert non-sexual photographs into sexually explicit images" without the subjects' consent. After rolling out a "spicy" mode for Grok last year, xAI added the ability to edit images with the chatbot, leading to a flood of sexualized AI deepfakes, including ones with minors. A group of teens sued xAI in March over claims Grok generated sexualized images of themselves as minors. Musk responded by saying, "Anyone using Grok to make illegal content will suffer the same consequences as if they upload illegal content," though this appears to be the first time xAI has sued anyone over AI deepfakes created with Grok. xAI claims Harwood's alleged actions exposed the company to "significant legal risk and reputational damage." It has asked the court to order Harwood to pay xAI for any damages, as well as "reasonable expenses incurred defending itself in any legal action filed by a victim of Defendant's conduct." xAI also wants the judge to block Harwood from creating an xAI account or using Grok.

Musk's xAI sues Grok user over sexualized 'deepfakes'

July 15 (Reuters) - Elon Musk's artificial-intelligence startup xAI has sued a South Carolina man arrested earlier this year on charges of sexually exploiting minors, alleging he misused the company's AI system Grok to create child sexual abuse material. xAI alleged in the lawsuit, opens new tab, filed in federal court in Texas on Tuesday, that Terry Harwood violated the company's terms of service. The case is one of the first brought by an AI company against one of its users for allegedly using an AI ⁠system to generate explicit material. Contact information for Harwood, who was arrested, opens new tabin February, was not immediately available. Spokespeople for xAI did not immediately respond to a request for comment on Wednesday. The company's lawsuit against Harwood follows intense global scrutiny of xAI over allegations that Grok has allowed users to generate non-consensual sexualized deepfakes, or realistic-looking videos fabricated by AI. xAI's complaint said that the company "enforces its rules against violators through account suspensions, account terminations, and by reporting suspected child sexual abuse material ⁠to the National Center for Missing & Exploited Children." "Indeed, Plaintiff has suspended 52,222 accounts and made 73,604 reports to NCMEC in 2026, resulting in (at least) 244 arrests," the lawsuit said. xAI alleged that Harwood uploaded non-sexual images of adults and minors to Grok and ⁠tried to use the system to generate sexually explicit deepfakes based on them. The complaint also alleged he created non-consensual sexual imagery of adults. The company asked the court ⁠for an unspecified amount of monetary damages and a court order permanently blocking Harwood from using Grok. "Defendant's actions were a calculated scheme to weaponize Plaintiff's ⁠tool for criminal ends, exposing real victims to profound and lasting harm, while exposing Plaintiff to significant legal risk and reputational damage," xAI said in the lawsuit. Reporting by Blake Brittain in Washington; Editing by Sanjeev Miglani Our Standards: The Thomson Reuters Trust Principles., opens new tab * Suggested Topics: * Litigation * Data Privacy Blake Brittain Thomson Reuters Blake Brittain reports on intellectual property law, including patents, trademarks, copyrights and trade secrets, for Reuters Legal. He has previously written for Bloomberg Law and Thomson Reuters Practical Law and practiced as an attorney.

xAI sues Grok user for generating nonconsensual sexualized deepfakes - Engadget

The defendant used Grok to create sexualized images of adults and children, xAI says. SpaceXAI (as xAI) has filed a lawsuit against Terry Wayne Harwood, a 67-year-old man from South Carolina whom the company has accused of using Grok to generate sexual images of real people without their consent. In the complaint that xAI filed in Texas, it said that the defendant uploaded non-sexual images of numerous adults and minors to his two xAI accounts from December 8, 2025 until February 18, 2026. He then asked Grok to alter the photos or to create new images and videos depicting the adults and children in them "in a pornographic manner or otherwise sexualizing them." The company said that Grok refused to follow his prompts on "numerous occasions" but that he repeatedly submitted edited prompts to circumvent the AI's safeguards. In one example mentioned in the filing, xAI said Harwood uploaded the photo of a fully dressed girl around 10 to 11 years old and then asked Grok to remove all of her clothing and make her do a "Playboy model impression" as she laid in bed. Grok refused his request, the lawsuit said, but he kept on submitting modified requests. As Reuters notes, this is one of the first lawsuits brought by an AI company against one of its users, and it sends a message that xAI will sue over the misuse of Grok. Reports started coming out in early January that Grok has been allowing its users to transform photos of real women and children into sexualized images. Regulators swiftly launched formal investigations into Grok after the scandal blew up. California's authorities started looking into the AI in mid-January, as did UK regulator Ofcom. The European Commission and Ireland's Data Protection Commission opened separate probes, as well. xAI implemented measures to prevent users from generating nonsensual sexual deepfakes when the investigations started. But even after that, users were still able to use Grok to undress men, and Harwood kept on uploading images to his account and using them to generate sexualized edits. The office of South Carolina's Attorney General announced Harwood's arrest on March 9, as part of the activities of the Internet Crimes Against Children Task Force. Harwood was charged with three counts of second degree sexual exploitation of a minor and five counts of sexual exploitation of a minor in the third degree. Harwood didn't just possess child sexual abuse materials, he also distributed them. xAI is now asking asking the court for an unspecified amount of monetary damages. It wants the court to order the defendant to pay for the expenses it incurs to defend itself in any legal action that might be filed by his victims.

Elon Musk Trained Grok Users to Expect Sexual Deepfakes, Now He's Suing Them

When Elon Musk launched Grok, it was intended to be the AI chatbot without any guardrails. But it turns out that every app needs guardrails of some kind, especially to protect children. And Musk's company xAI is now suing one of its users for allegedly creating child sexual abuse material and non-consensual sexual imagery, according to Reuters. Musk's company filed a lawsuit in federal court in Texas this week against Terry Wayne Harwood, a Grok user from South Carolina. The company says in the suit that Harwood is being sued for, "his repeated, deliberate, and unconscionable violations of the xAI Terms of Service and Acceptable Use Policy through his abhorrent abuse of Plaintiff's generative AI chatbot known as 'Grok.'" The lawsuit is quick to note that Grok is a "neutral tool, subject to user control," putting the onus completely on users to keep everything legal. The suit also claims that xAI has reported child sexual abuse material to the National Center for Missing and Exploited Children (NCMEC) 73,604 times in 2026, which has led to the arrest of at least 244 individuals. The company's suit points to the terms of service and acceptable use policy for Grok, which says that it bans "Undressing or nudifying real persons, or otherwise altering a real person's image or likeness to depict them in an intimate or sexual context" as well as "Depicting likenesses of persons in a pornographic manner." That would probably be news to many X users who could use Grok's "spicy mode" last year to undress people. We did it ourselves in a test back in Aug. 2025, and we noticed that it was much easier to get the bot to undress women than men. And remember when users were gleefully undressing regular people on the app in late 2025? Plenty of inappropriate images of children were reportedly created. In January, members of the UK government threatened X with fines or a ban over its ability to create non-consensual sexual images. "They just want to suppress free speech," Musk told his army of sycophantic followers. The lawsuit touts its "technological safeguards" to prevent people from creating illegal and harmful content, but those obviously aren't enough. And it's hard to take the company too seriously when it was founded by a guy like Elon Musk. Remember when he publicly suggested he wanted to get Taylor Swift pregnant? "Defendant's actions were a calculated scheme to weaponize Plaintiff's tool for criminal ends, exposing real victims to profound and lasting harm, while exposing Plaintiff to significant legal risk and reputational damage," the lawsuit alleges. Reuters notes that Harwood was arrested in February, and we were unable to reach him for comment. Musk did his part to set expectations for Grok when it first launched, touting anime girls who could become your digital dancing bunnies. Things got so pathetically sexual that even some of Musk's fans commented on how gross it all seemed back in 2025. Musk has since deleted at least one of the sexualized videos he posted last year, though it's been saved by archiving sites. If you spend any time on the subreddit for Grok, you'll notice many people who complain that Grok isn't making the sexual images they were promised. Gizmodo even filed a FOIA request recently to see what kind of complaints people might be making to the FTC about Grok. As you can guess, there were users upset about Grok's new guardrails. "I keep having issues with being able to make 100% explicit and 100% uncensored AdvancedAI Deepfakes on GrokAI myself 100% and I want everything completely uncensored with the mature rated adult content but I never wanna be able to abuse anything myself at all 100%," one complaint read in a typically bizarre run-on sentence. Another user told of how they were initially happy with Grok until things started to change. "Trying to generate a video using the same process, with the same base image and the same prompts that were used to generate the videos previously, it was moderated 100% of the time," the user from Minnesota wrote. "Then, I looked on reddit, and saw the grok subreddit is full of people upset about the new moderation."

xAI's first lawsuit against a user tests who is responsible for what Grok makes

The company says the defendant engineered prompts to defeat Grok's safeguards. Courts on three continents are being asked whether the safeguards were ever the point. AI has sued one of its own users, alleging he used Grok to generate child sexual abuse material in breach of the company's terms of service. It is understood to be the first case an AI company has brought against a user over what its own system produced. It arrives while xAI is on the other side of the same argument in several courts, among them a London High Court claim testing whether the developer answers for what its users make. The complaint, 12 pages, was filed in federal court in Texas on Tuesday against Terry Harwood, a South Carolina man arrested earlier this year on charges of sexually exploiting minors. xAI is seeking unspecified monetary damages and a court order permanently barring him from the platform. The company's account of what happened is specific, and it is the reason the filing matters beyond this defendant. Harwood "opened multiple xAI accounts using false identities", the complaint says, and "designed misleading prompts to circumvent Grok's built-in safeguards and then abused the tool to convert non-sexual photographs into sexually explicit images without the photograph subjects' knowledge or consent". Crucially, xAI says the guardrails engaged. Grok "refused to follow the prompts on the basis that such material violated Grok's content moderation guardrails", according to the filing. "In response, Defendant repeatedly submitted further prompts, with alterations, in an effort to circumvent Grok's moderation efforts." That is the legal architecture of the whole case. xAI is not arguing that Grok cannot produce this material. It is arguing that Grok tried not to, and that a determined user defeated it, which relocates responsibility from the system to the person at the keyboard. The filing also puts numbers on xAI's enforcement for the first time. The company says it has suspended more than 52,000 accounts and filed more than 73,000 reports to the National Center for Missing & Exploited Children, resulting in close to 250 arrests during 2026 alone. Those figures are xAI's own, disclosed in litigation it chose to bring. The awkwardness is in the timing. xAI is currently the defendant in a run of cases making precisely the opposite argument: that the company, not the user, is answerable for what Grok generates. The London claim, brought by the Labour MP Jess Asato, is one. Baltimore has sued under consumer protection law, and Paris prosecutors have an investigation open that Musk has declined to cooperate with. Grok has been banned in Malaysia and Indonesia over sexually explicit output. Apple privately threatened to pull it from the App Store in January. The company's public position has moved some distance in six months. "I am not aware of any naked underage images generated by Grok. Literally zero," Musk posted on X in January, when the allegations first broke. The complaint filed this week describes a moderation system that logs, refuses, and escalates such attempts by the tens of thousands. Both things can be true. A system can refuse most attempts and still be, in the aggregate, the most productive source of this material anyone has built. The Center for Countering Digital Hate estimated Grok generated roughly three million sexualised images between late December and early January, including around 23,000 that appeared to depict children. xAI's 73,000 NCMEC reports are evidence of a filter working and evidence of the scale of what it is filtering. What the case does not resolve is the question every regulator is actually asking, which is whether a general-purpose image model that can be talked into this by a persistent amateur should be shipped to the public at all. Harwood's alleged method, by xAI's own telling, was to keep rephrasing until something worked. The company says it enforces its rules through account suspensions, terminations, and referrals to NCMEC, and the complaint presents this suit as an extension of that. It is also, unavoidably, a filing that produces a defendant who is not xAI at a moment when courts in London, Baltimore, and Paris are considering whether it should be. The EU has since agreed a ban on non-consensual intimate deepfakes. Harwood faces criminal charges in South Carolina separately from this suit, and has not publicly responded to the civil complaint. xAI, for now, has picked the defendant it wants.

xAI Sues Man for Using Grok to Create CSAM Deepfakes

Elon Musk's startup xAI is suing a man for allegedly misusing the company's AI image generator Grok to create child sexual abuse material. Terry Harwood from South Carolina was arrested earlier this year on charges of creating sexually explicit images of minors. xAI filed its lawsuit in Texas on Tuesday, which is one of the first instances of an AI company suing one of its users for misuse. In August 2025, xAI introduced spicy mode for Grok, which was capable of generating photorealistic nudity. Afterwards, xAI introduced the ability to edit images on Grok and that led to an avalanche of deepfake pornographic images, mostly of women, but some were of children. Amid the furore, Grok restricted its image editing tool but that didn't stop regulators opening an investigation and a group of teens from suing xAI who claim that Grok created nonconsensual sexually explicit images and videos of them when they were minors. In its lawsuit, xAI says that Harwood's allegedly making CSAM images created "significant legal risk and reputational damage" for the company. According to Reuters, xAI's complaint says the company "enforces its rules against violators through account suspensions, account terminations, and by reporting suspected child sexual abuse material ⁠to the National Center for Missing & Exploited Children (NCMEC)." xAI claims to have suspended 52,222 accounts and made 73,604 reports to the NCMEC in 2026, which has purportedly led to at least 244 arrests. "Defendant's actions were a calculated scheme to weaponize Plaintiff's ⁠tool for criminal ends, exposing real victims to profound and lasting harm, while exposing Plaintiff to significant legal risk and reputational damage," xAI says in the lawsuit. The AI startup is asking the court to order Harwood to pay damages, as well as "reasonable expenses incurred defending itself in any legal action filed by a victim of Defendant's conduct." xAI is also seeking a court order blocking Harwood from creating an xAI account or using Grok. Image creditsHeader photo licensed via Depositphotos.

Musk's xAI sues user who allegedly used Grok to create child sexual abuse material

Case is one of first brought by an AI company against a user ⁠for allegedly using a tool to generate child abuse material Elon Musk's artificial-intelligence startup xAI has sued a South Carolina man arrested ⁠earlier this year on charges of sexually exploiting minors, alleging he misused the company's AI system Grok to ⁠create child sexual abuse ⁠material. xAI alleged in the lawsuit, filed in federal court in Texas on Tuesday, that Terry Harwood violated the company's ⁠terms of service. The case is one of the first brought by an AI company against one of its users ⁠for allegedly using an AI system to generate child sexual abuse material. Spokespeople for xAI did not immediately respond to a request for comment on Wednesday. The company's lawsuit against Harwood comes after intense global scrutiny of xAI over allegations that Grok has allowed users to generate non-consensual sexualized deepfakes, or realistic-looking videos fabricated by AI. Contact information for Harwood, who was arrested in February, was not immediately available. xAI's complaint said that the company "enforces its rules against violators through account suspensions, account terminations, and by reporting suspected child sexual abuse material to the National Center for Missing + Exploited Children". xAI alleged that Harwood uploaded non-sexual images of adults and minors to Grok and tried to use the system to generate sexually explicit deepfakes based ⁠on them. The complaint also alleged he created non-consensual sexual imagery of adults. The company asked the court for an unspecified amount of monetary damages and a court order permanently blocking Harwood from using Grok. "Defendant's actions were a calculated scheme to weaponize Plaintiff's tool for criminal ends, exposing real victims to profound and lasting harm, while exposing Plaintiff to significant legal risk and reputational damage," xAI said in the lawsuit.

Elon Musk's xAI, Which is Being Sued Over AI-Generated Sexual Deepfakes, Sues Grok User Over AI-Generated Sexual Deepfakes

Can't-miss innovations from the bleeding edge of science and tech Elon Musk's xAI is suing a man for using its flagship chatbot, Grok, to create nonconsensual sexual deepfakes of women and children. The Grok user, a 67-year-old in South Carolina named Terry Wayne Harwood, was arrested in March on multiple counts of sexual exploitation of a minor. According to the allegations, Harwood possessed and distributed child sexual abuse material, or CSAM. Filed earlier this week in Texas, xAI's lawsuit accuses Harwood of using Grok to do just that, per Reuters. He apparently used two different Grok accounts, according to the lawsuit, and though xAI claims that Grok refused some of Harwood's requests, he was able to tailor prompts to sidestep Grok's safety guardrails and create the disturbing imagery. The lawsuit follows a massive deepfake scandal that unfolded over the winter, as X users realized that Grok -- which had been updated with a new photo and video-editing feature -- could be used to quickly and easily alter images to artificially undress real people and depict them in sexual positions. As Futurism reported, some of these images depicted real women in sexually violent and degrading scenarios. Some of the nonconsensual deepfakes depicted minors. The Center for Countering Digital Hate estimated that during an 11-day period, Grok "generated an estimated 3 million sexualized images, including 23,000 of children." Now, as it sues one of its users for the creation of illegal content, xAI is fighting multiple lawsuits over Grok-generated CSAM. In March, teenagers in Tennessee sued the company in a proposed class action, alleging that Grok had been used to create deepfakes of them and more than a dozen other minors. Just last week, the suit was amended to include allegations that a man used Grok to generate more than 7,000 sexual deepfakes of his stepdaughter. The amended complaint alleges that the man's prompts largely went unflagged by Grok's guardrails. In another high-profile case, the former conservative influencer (and mother to one of Musk's many children) Ashley St. Clair sued xAI after she was aggressively targeted by sexualized deepfakes, including one depicting her as a minor. Law enforcement professionals and child online safety experts say that AI has led to an overwhelming and disorienting new flood of CSAM, as Bloomberg detailed in a devastating report earlier this year. It's also created a host of new challenges for those attempting to find and arrest child predators, as AI has made it more difficult for investigators to determine whether a child depicted in abuse material is real or AI-generated. In its latest lawsuit, xAI accuses Harwood of engaging in a "calculated scheme to weaponize Plaintiff's ⁠tool for criminal ends, exposing real victims to profound and lasting harm, while exposing Plaintiff to significant legal risk and reputational damage." xAI -- which is now owned by SpaceX and has since been renamed to SpaceXAI -- also alleges that it "enforces its rules against violators through account suspensions, account terminations, and by reporting suspected child sexual abuse material ⁠to the National Center for Missing and Exploited Children," or NCMEC. It also claims that xAI "has suspended 52,222 accounts and made 73,604 reports to NCMEC in 2026, resulting in (at least) 244 arrests." But despite its best efforts, it's clear that the startup still has a serious deepfake problem. If a user is engaging in deeply disturbing criminal activity using a company's service, it's reasonable for a company to hold them to account. But suing users doesn't fix a chatbot's failing guardrails, facilitating their crimes. More on Grok and deepfakes: Grok Linked to Sickening Crime in Lawsuit That Puts SpaceX in Crosshairs

xAI sues Grok user Terry Harwood over child sexual abuse deepfakes

Elon Musk's AI startup xAI filed a federal lawsuit Tuesday against a South Carolina man, alleging he used the Grok chatbot to generate child sexual abuse material in violation of the company's terms of service. Reuters described the lawsuit as among the earliest instances of an AI company taking legal action against a user over the alleged production of explicit content with its technology. The lawsuit was filed in the U.S. District Court for the Northern District of Texas. The defendant, Terry Wayne Harwood, was arrested in February on multiple counts of alleged sexual exploitation of a minor, according to CNN. According to xAI's complaint, Harwood opened multiple accounts and submitted what the company called "misleading prompts" to get around the platform's "built-in safeguards," feeding it non-sexual photographs of adults and children as source material for sexually explicit deepfakes. Contact information for Terry Wayne Harwood was not immediately available, according to Reuters. "Defendant's actions were a calculated scheme to weaponize Plaintiff's tool for criminal ends, exposing real victims to profound and lasting harm, while exposing Plaintiff to significant legal risk and reputational damage," xAI said in the lawsuit. xAI has asked the court to award monetary damages in an amount to be determined and to issue a permanent injunction preventing Terry Wayne Harwood from accessing Grok. The company also asked the court to declare that he violated its terms of service. The complaint states that xAI has taken down 52,222 accounts and submitted 73,604 reports to the National Center for Missing & Exploited Children so far in 2026, actions the company says have contributed to at least 244 arrests. The case arrives amid sustained legal pressure on xAI over Grok's image-generation features. Baltimore's city government filed suit against xAI in circuit court, alleging Grok violates local consumer protection laws by producing nonconsensual explicit deepfakes, including images of minors. Separately, three Tennessee teenagers brought a class action in federal court in California, alleging that a perpetrator used a third-party app licensed to xAI's technology to generate child sexual abuse material depicting them. A U.K. member of Parliament, Jess Asato, has also filed a claim in England's High Court against xAI over nonconsensual sexual deepfake images generated of her. xAI has said it takes action against misuse by suspending accounts and reporting suspected child sexual abuse material to authorities. Spokespeople for xAI did not immediately respond to Reuters' request for comment.

SpaceXAI sues user over alleged Grok deepfake abuse

SpaceXAI has filed a lawsuit against Terry Wayne Harwood, a 67-year-old from South Carolina, alleging that he used Grok to generate sexual images of individuals without their consent, Reuters reports. The complaint, submitted in Texas, claims Harwood uploaded non-sexual images of numerous adults and minors to two xAI accounts between December 8, 2025, and February 18, 2026, requesting Grok to alter these photos or create new images depicting them in sexualized manners. xAI reported that Grok rejected Harwood's prompts on numerous occasions, but he continued to submit edited prompts to bypass the AI's safeguards. The lawsuit cites an instance where Harwood uploaded a photo of a fully dressed girl, approximately 10 to 11 years old, and requested Grok to undress her and pose her in a sexual manner. The company emphasized this as one of the first lawsuits by an AI firm against a user for the misuse of AI-generated content. Following initial reports in January about Grok's capabilities allowing for the transformation of images of real women and children into sexualized depictions, various regulatory bodies launched investigations. Authorities in California, along with the UK regulator Ofcom, opened probes into the AI service, later joined by the European Commission and Ireland's Data Protection Commission. In response to these inquiries, xAI implemented safeguards to prevent the generation of nonconsensual sexual deepfakes. However, Harwood reportedly continued to upload images and produce sexualized edits despite these measures. The South Carolina Attorney General's office announced Harwood's arrest on March 9, facilitated by the Internet Crimes Against Children Task Force. He faces three counts of second-degree sexual exploitation of a minor and five counts of third-degree sexual exploitation of a minor, involving both the possession and distribution of child sexual abuse materials.

Musk's XAI Sues Grok User Over Sexualized 'Deepfakes'

July 15 (Reuters) - Elon Musk's artificial-intelligence ⁠startup ⁠xAI has sued a ⁠South Carolina man arrested earlier this year on charges of sexually exploiting minors, alleging he misused the company's AI system Grok to create child sexual abuse material. xAI alleged in the lawsuit, filed in federal court in Texas on Tuesday, that ⁠Terry ⁠Harwood violated the company's terms of service. The case is one of the first brought by an AI company against one of its users for allegedly using an AI system to generate explicit material. Contact information for Harwood, who was arrested in February, was not immediately available. Spokespeople for xAI did not ⁠immediately respond ⁠to a request for ⁠comment on Wednesday. The company's lawsuit against Harwood follows intense global scrutiny of xAI over allegations that Grok has ⁠allowed users to generate non-consensual sexualized deepfakes, or realistic-looking videos fabricated by AI. xAI's complaint said that the company "enforces its rules against violators through account suspensions, account terminations, and by reporting suspected child sexual abuse material to the ⁠National Center for Missing & Exploited Children." "Indeed, Plaintiff has suspended 52,222 accounts and ⁠made 73,604 reports to NCMEC in 2026, resulting in (at least) 244 arrests," the lawsuit said. xAI alleged that Harwood uploaded non-sexual images of adults and minors to Grok and tried to use the system to generate sexually explicit deepfakes based on them. The complaint also alleged he created non-consensual sexual imagery of adults. The company asked the court for an unspecified amount of monetary damages and a court order ⁠permanently blocking Harwood from using Grok. "Defendant's actions were a calculated scheme to weaponize Plaintiff's tool for criminal ends, exposing real victims to profound and lasting harm, while exposing Plaintiff to significant legal risk and reputational damage," xAI said in the lawsuit. (Reporting by Blake Brittain in Washington; Editing by Sanjeev Miglani)

Musk's xAI sues man accused of using Grok to create explicit material

Elon Musk's artificial intelligence firm xAI is suing a South Carolina man, accusing him of using its generative chatbot Grok to create child sexual abuse material (CSAM). The suit, filed in a federal court in Texas Tuesday, alleges Terry Wayne Harwood had at least two Grok accounts using false identifies to create and distribute explicit material. Harwood allegedly uploaded non-sexual images of multiple adults and minors and attempted to use Grok to depict the individuals in a "pornographic manner or otherwise sexualizing them." "His conduct was knowing, intentional, and calculated," xAI's attorneys wrote, adding Harwood's actions were a "scheme to weaponize" the AI firm's tool for "criminal ends, exposing real victims to profound and lasting harm" and xAI to "significant legal risk and reputational damage." In one instance, Harwood allegedly uploaded a picture of a fully dressed female who appeared to be about 10 or 11 years old, and asked Grok to generate sexualized images of them. Contact information for Harwood or his attorneys was not immediately available. xAI said its reporting of the accounts played a role in Harwood's arrest in South Carolina last March. The firm is now asking the court for relief to prevent Harwood from creating new accounts or circumventing the firm's rules for Grok use, and monetary damages. The AI firm, which was acquired by Musk's aerospace company SpaceX earlier this year, faced scrutiny earlier this year over Grok's alleged sexual abuse content. The firm has faced its own legal challenges over the chatbot. In March, three Tennessee teenagers sued the firm, alleging their photographs were used for Grok to create CSAM. Amid the backlash, Musk in January said he was unaware of any instances in which Grok generated nude photos of minors. The tech billionaire said Grok does not product illegal material, regardless of user requests, and said any bugs encountered are quickly fixed. xAI limited Grok's image and video editing and generation capabilities to paid subscribers to X Premium+ or SuperGrok. The firm, in the suit against Harwood, maintained it enforces rules against violators through account suspensions or terminations and reports to the National Center for Missing and Exploited Children. xAI has suspended more than 52,200 accounts and made 73,600 reports to NCMEC this year, according to the suit.

Elon Musk's xAI sues Grok user over sexualised 'deepfakes'

xAI alleged in the lawsuit, filed in federal court in Texas on Tuesday, that Terry Harwood ⁠violated ⁠the company's terms of service. The case is one of the first brought by an AI company against one of its users for allegedly using an AI system to generate explicit material. Elon Musk's artificial-intelligence startup xAI has sued a South Carolina man arrested earlier this year on charges of sexually exploiting minors, alleging he misused the company's AI system Grok to create child sexual abuse material. xAI alleged in the lawsuit, filed in federal court in Texas on Tuesday, that Terry Harwood ⁠violated ⁠the company's terms of service. The case is one of the first brought by an AI company against one of its users for allegedly using an AI system to generate explicit material. Contact information for Harwood, who was arrested in February, was not immediately available. Spokespeople for xAI did not immediately respond to a request for comment on Wednesday. The ⁠company's lawsuit against Harwood follows intense global scrutiny of xAI over allegations that Grok has allowed users to generate non-consensual sexualized ⁠deepfakes, or realistic-looking videos fabricated by AI. xAI's complaint said that the company "enforces its rules against violators through account suspensions, account terminations, and by reporting suspected child sexual abuse material to the National Center for Missing & Exploited Children." "Indeed, Plaintiff has suspended 52,222 accounts and made 73,604 reports to NCMEC in 2026, resulting in (at least) 244 arrests," the lawsuit said. xAI alleged that Harwood uploaded non-sexual images of adults and minors to Grok and tried to use the system to generate sexually explicit deepfakes based on them. ⁠The complaint also alleged he created non-consensual sexual imagery of adults. The company asked the court for an unspecified amount of monetary damages and a court order permanently blocking Harwood from using Grok. "Defendant's actions were a calculated scheme to weaponize Plaintiff's tool for criminal ends, exposing real victims to profound and lasting harm, while exposing Plaintiff to significant legal risk and reputational damage," xAI said in the lawsuit.

xAI Sues Grok User in Major Crackdown on Illegal AI-Generated Content

Elon Musk's xAI is taking a Grok user to court over alleged AI-generated child sexual abuse material, showing that the company is ready to act against people who misuse its AI tools. Elon Musk's AI company, xAI, has filed a lawsuit against a Grok user accused of creating AI-generated child sexual abuse material (CSAM). On Tuesday, the company filed a lawsuit against a user named Terry Wayne Harwood who allegedly violated the rules and regulations of the company and converted non-sexual photographs into sexually explicit images without consent. The lawsuit is one of the strongest actions taken by an AI company against a user accused of misusing a generative AI tool. The company wants the court to stop the user from using Grok and is also seeking damages. According to xAI, the case is meant to warn others that illegal use of AI will have serious consequences.

Elon Musk's xAI sues Grok user for allegedly creating sexually explicit deepfakes of adults, minors

Elon Musk's artificial-intelligence startup xAI has sued a South Carolina man arrested earlier this year on charges of sexually exploiting minors, alleging he misused the company's AI system Grok to create child sexual abuse material. xAI alleged in the lawsuit, filed in federal court in Texas on Tuesday, that Terry Harwood violated the company's terms of service. The case is one of the first brought by an AI company against one of its users for allegedly using an AI system to generate explicit material. Contact information for Harwood, who was arrested in February, was not immediately available. Spokespeople for xAI did not immediately respond to a request for comment on Wednesday. The company's lawsuit against Harwood follows intense global scrutiny of xAI over allegations that Grok has allowed users to generate non-consensual sexualized deepfakes, or realistic-looking videos fabricated by AI. xAI's complaint said that the company "enforces its rules against violators through account suspensions, account terminations, and by reporting suspected child sexual abuse material to the National Center for Missing & Exploited Children." "Indeed, Plaintiff has suspended 52,222 accounts and made 73,604 reports to NCMEC in 2026, resulting in (at least) 244 arrests," the lawsuit said. xAI alleged that Harwood uploaded non-sexual images of adults and minors to Grok and tried to use the system to generate sexually explicit deepfakes based on them. The complaint also alleged he created non-consensual sexual imagery of adults. The company asked the court for an unspecified amount of monetary damages and a court order permanently blocking Harwood from using Grok. "Defendant's actions were a calculated scheme to weaponize Plaintiff's tool for criminal ends, exposing real victims to profound and lasting harm, while exposing Plaintiff to significant legal risk and reputational damage," xAI said in the lawsuit.

xAI sues man for allegedly using Grok to create child sexual abuse content and explicit images

xAI is seeking damages and a permanent court order banning the accused from using Grok. Elon Musk's AI company xAI has filed a lawsuit against the South Carolina man and accused him misusing its AI chatbot Grok to make child sexual abuse material (CSAM) and explicit deepfake images. The lawsuit, filed in a federal court in Texas, among one of the first known cases in which an AI company has taken legal action against one of its own users over the alleged creation of illegal AI-generated content. The defendant, Terry Harwood, was arrested in South Carolina in February on charges related to the sexual exploitation of minors. As per xAI's complaint, Harwood violated the company's terms of service by attempting to use Grok to create sexually explicit images involving both minors and adults. The company alleged that Harwood uploaded ordinary photos of adults and children into Grok and attempted to generate explicit AI-manipulated images based on them. xAI also claims he created non-consensual sexual content featuring adults through the platform. In its filing, xAI said that the alleged actions not only caused serious harm to victims but also exposed the company to legal and reputational risks. The company is seeking financial damages, although it has not specified an amount, along with a permanent court order preventing Harwood from accessing Grok in the future. The lawsuit comes at a time when xAI has faced growing criticism over Grok's safeguards against AI generated content. The company has previously been accused of allowing users to create non-consensual sexual deepfakes using the chatbot. As a part of filing, xAI also showcased that it actively removes users who violate its policies and reports suspected child sexual abuse material to the National Center for Missing & Exploited Children (NCMEC). As per the company, it has suspended more than 52,000 accounts and submitted over 73,000 reports to NCMEC in 2026, actions that it says have contributed to at least 244 arrests.