New York Seizes 12 Celebrity Deepfake Websites Hosting AI-Generated Content of 1,200 Victims

Manhattan DA Takes Down Dozen Celebrity Deepfake Websites

The Manhattan District Attorney's Office announced the New York seizure of 12 deepfake websites on Monday, marking what officials believe is the largest takedown of platforms hosting AI-generated deep-fake pornography to date1

1

2

2

. District Attorney Alvin Bragg said the sites unlawfully shared, published, and sold nonconsensual sexually explicit AI-generated videos depicting approximately 1,200 victims, overwhelmingly women1

1

. The victims included social media influencers, actors, activists, athletes, musicians, and politicians1

1

.

Enforcement Action Under Criminal Procedure Laws

The seizures were carried out under New York's criminal procedure laws following a seizure warrant issued by a New York State Supreme Court1

1

. The celebrity deepfake websites, with names including phrases like "AI Celebs," "sex celebrity," or promising "real deepfakes," hosted hyper-realistic deep-fake content that swapped the faces of public figures into existing pornographic videos1

1

2

2

. Videos could be several minutes in length and included fully sexually explicit scenes, with some videos watched tens of thousands of times1

1

. One website claimed to have been online since 20181

1

.

Investigation Targets Politicians and Public Figures

Research by Benjamin Shultz of the American Sunlight Project revealed that 56 European politicians had been mentioned or appeared on the websites, including 16 US politicians and two based in the UK1

1

. On at least two of the seized websites, elected officials were given profiles containing their name and position, along with a grid of AI-altered sexually explicit content hosting their likeness1

1

. Bragg stated that a victim came forward to report a site, triggering the investigation, though no arrests have been confirmed2

2

.

Growing Industry of AI-Generated Content

As generative AI technology has improved since deepfakes first emerged at the end of 2017, a dark industry of websites, apps, and bots has appeared designed to "undress" photos of women or create graphic sexual videos1

1

. The images and videos are increasingly realistic and can be generated in seconds1

1

. Shultz noted that many videos appeared to be reposts of illegal content that had originally been shared on the MrDeepFakes website, a notorious deepfake platform that was taken offline after its creators were unmasked by reporters1

1

.

Legal Framework and Ongoing Investigations

Bragg emphasized that in New York State, it is illegal to share real photos and videos of people engaging in sex without their permission, and likewise illegal to share that content if it's created or altered by artificial intelligence tools2

2

. The Manhattan District Attorney's Office says its investigations into who is responsible for the websites, plus those who uploaded damaging videos of victims, are ongoing1

1

. "Our investigations into deep-fake pornography sites and content creators are just beginning. This is part of our broader work to dismantle systems that breed and foster crime," Bragg added2

2

.

Take It Down Act Enables Removal

While laws against deepfakes have been slow to emerge, the US Take It Down Act increasingly allows images to be removed, with part of the law allowing law enforcement officials to take down and seize websites1

1

. In May, the United States began enforcing the law requiring tech platforms to remove sexual deep-fakes and other non-consensual intimate imagery2

2

. President Donald Trump signed the Take It Down Act last year, criminalizing the online distribution of non-consensual sexual imagery that is often created using cheap and widely available artificial intelligence tools2

2

.

Significance of the Takedowns

Leonie Oehmig, a researcher at the Institute for Strategic Dialogue who has studied the deepfake creation ecosystem, called the takedowns "a significant win in the fight against synthetic nonconsensual intimate imagery." Oehmig noted that enforcement of state laws shows that the harmful content can be tackled and that the seized sites don't appear to be accessible even when using VPNs to try and access them from other locations1

1

. "This shows that meaningful, coordinated enforcement is achievable even where regulation itself remains fragmented across jurisdictions," Oehmig said1

1

. However, Oehmig cautioned that there are many other parts of the deepfake industry causing serious harm, and effective intervention needs to target the entire ecosystem1

1

.