Grok AI launches video generator as governments probe millions of sexualized deepfakes

Grok AI Expands Capabilities Despite Global Backlash

Elon Musk's xAI has launched Grok Imagine 1.0, a significant upgrade to its Grok AI platform that introduces advanced video generation capabilities. The new model can produce 10-second video clips at 720p resolution with audio, positioning it alongside competitors like OpenAI's Sora and Google's Veo 31

1

. According to xAI, the platform generated 1.245 billion videos in January 2026 alone4

4

. The launch arrives amid intense scrutiny over the platform's role in enabling AI-powered abuse at an unprecedented scale, raising urgent questions about content moderation and platform safety blind spots.

Sexualized Images Controversy Reveals Massive Scale of Harm

The controversy surrounding Grok image generation began in late December when users discovered the platform's one-click editing tool could be weaponized to create non-consensual intimate imagery. Between the end of December and early January, Grok produced an estimated 3 million sexualized images over just 11 days, according to research from the Center for Countering Digital Hate3

3

. The analysis revealed that approximately 23,000 of these images depicted children, with the system generating a sexualized image of a child every 41 seconds on average1

1

. A separate New York Times report found that Grok made 1.8 million deepfake sexual images over nine days in January, comprising 41% of all images generated by the platform during that period1

1

.

How Unchecked Innovation Created a Safety Crisis

The root of the problem lies in design choices that prioritized accessibility over user safety. The one-click image editing tool allowed anyone to upload photographs and alter them with simple prompts, with minimal guardrails in place at launch3

3

. Grok's "Spicy Mode" feature was specifically designed to create suggestive and provocative imagery, distinguishing it from competitors that block such requests1

1

. Reports indicate that Elon Musk instructed staff to loosen safety guardrails because he was "unhappy about over-censoring," leading to the resignation of senior safety staff5

5

. According to The Washington Post, xAI's AI safety team consisted of no more than three people for much of 20254

4

. This skeletal safety infrastructure proved woefully inadequate as deepfake photography and AI-generated child sexual abuse material proliferated across the platform.

Global Governments Launch Investigations and Enforcement Actions

The scale of abuse triggered swift regulatory responses worldwide, marking a turning point in AI regulation efforts. Indonesia and Malaysia blocked the X app entirely, while the California attorney general and the UK government opened formal investigations into xAI1

1

. Three US senators and advocacy groups called on Apple and Google to remove X from their app stores for violating terms of service1

1

. In India, the Ministry of Electronics and Information Technology issued a stern warning on January 2, 2026, demanding specific information about takedown actions and preventive measures5

5

. The European Commission extended an existing retention order requiring X to preserve internal documents until the end of 2026 while assessing compliance with the Digital Services Act5

5

. UK Prime Minister Keir Starmer publicly called the situation "disgusting" and "shameful," while UK officials criticized xAI's decision to restrict image generation to paid subscribers as "insulting," arguing it appeared to monetize access to potentially illegal content3

3

.

Content Moderation Backlash Exposes Platform Responsibility Debates

The content moderation backlash has reignited fundamental debates about platform responsibility and free speech. Musk has consistently argued that only content breaking the law should be removed, dismissing broader content moderation as the "woke mind virus" and claiming critics "just want to suppress free speech"2

2

. However, legal experts point out that AI-generated child sexual abuse material constitutes evidence of a crime, not protected expression2

2

. The controversy raises critical questions about manufacturer liability: when AI systems generate illegal content through features designed and deployed by the platform itself, the platform becomes more than a neutral intermediary3

3

. xAI eventually restricted image creation to paid users on January 9 and added technical controls to block "undressing" features on January 14, but only after millions of harmful images had already been created3

3

.

AI Regulation Struggles to Keep Pace With Technological Harm

The Grok crisis highlights how slowly regulatory frameworks adapt to rapidly evolving AI systems. The UK's Online Safety Act took seven years to develop and still isn't fully implemented despite receiving royal assent in 20232

2

. The UK became the first country to introduce laws criminalizing tools used to create AI-generated child sexual abuse material as part of the crime and policing bill, but a year later the bill remains in parliamentary debate2

2

. The US passed the Take It Down Act in 2025, criminalizing the sharing of non-consensual intimate imagery and deepfakes, but platforms have until May to establish takedown processes1

1

. A 2024 survey found that 72% of the British public believe laws and regulations would make them more comfortable with AI, up 10 percentage points from 20222

2

. The speed at which harm occurs—millions of images generated in days—vastly outpaces the years-long legislative processes designed to prevent it, leaving current users vulnerable while policymakers debate solutions.