OpenAI reveals how Chinese law enforcement exploited ChatGPT to harass dissidents globally

ChatGPT Logs Show How China Harasses US-Based Dissidents

If you thought students using ChatGPT to cheat on their homework was bad, just wait. Researchers at OpenAI have unearthed how the Chinese government has used ChatGPT as part of coordinated plans to harass and intimidate political dissidents living in foreign countries like the US, Japan and Italy in its latest report. The firm's researchers identified an account they believe was based in Mainland China, which used the AI tool to edit reports and plans related to what it called "cyber special operations," feeding it detailed information and asking it for advice. In one example, the Chinese operative reportedly told the tool that a Chinese official had forged documents seemingly from a US county court, which they planned to use to dupe a social media platform into taking down anti-CCP content. In another, Chinese officials impersonated US immigration officials, telling US-based dissidents that their statements had broken the law. The scale of these operations was huge, according to the target: 300 social media platforms outside of China, and tens of thousands of posts from thousands of accounts. The accounts would amplify pro-China content, censor unfavorable comments, and engage in other activity aimed at "shaking the information landscape." Impersonation was another common strategy among the Chinese operatives, with five separate fake Bluesky accounts created for a man known as Hui Bo -- a well-known Chinese dissident living in California. Fake accounts were also created for Teacher Li, a key figure in the 2022 anti-lockdown protests who is now living in Italy. In one campaign, the operatives created a fake obituary and fake pictures of gravestones stating that political dissident Jie Lijian had died before posting this content far and wide on social media. In another example, AI was used to generate fake "evidence" that supported takedown claims directed against pro-Taiwan X accounts, backing allegations of activity that violated the platform's rules. The ChatGPT users said these cyber campaigners had some success influencing their targets, with many losing followers, cutting down their activity, or deleting their accounts entirely. OpenAI's reports note, however, that only a relatively slim proportion of the network's posts achieved much attention on social media.

OpenAI Says ChatGPT Refused to Help Chinese Influence Operations

OpenAI said its ChatGPT AI service refused to assist an individual associated with Chinese law enforcement in planning an online campaign to discredit the Japanese prime minister. In its latest update on disrupting malicious uses of artificial intelligence, the San Francisco-based startup detailed requests by the user that included editing status reports on a wider net of covert influence operations against domestic and foreign adversaries. OpenAI interpreted the evidence it gathered as indicative of a "large-scale, resource-intensive and sustained" effort by Chinese law enforcement to suppress dissent. "I'm not familiar with what you mentioned and do not see any basis for this accusation," Chinese foreign ministry spokeswoman Mao Ning said at a regular news conference on Thursday. The announcement comes on the heels of archrival Anthropic PBC's decision to loosen its hallmark commitment to AI guardrails, even as it resists pressure from the US Department of Defense about dropping such safeguards. OpenAI on Wednesday said it identified a series of misdeeds that included romance scams targeting Indonesians, a social media content farm linked to Russia and more accounts deemed likely to have originated in China seeking information from US officials. The plan targeting Sanae Takaichi, Japan's first female prime minister, surfaced in mid-October, in the days leading up to her election when she criticized the state of human rights in Inner Mongolia, according to OpenAI. The user sought help in crafting a plan that would amplify negative comments about Takaichi, accuse her of far-right leanings and increase online pressure, the company said. The Japanese prime minister later elicited a rebuke from China over remarks about Taiwan, a self-governing island that China considers part of its territory. The ensuing diplomatic spat continues to weigh on tourism and trade between the two countries.

Chinese law enforcement tried using ChatGPT to discredit Japan's PM, OpenAI says

Why it matters: The operation was unusual and "revealed a lot about China's strategy for covert influence operations and transnational repression," Ben Nimmo, principal investigator on OpenAI's intelligence and investigations team, told reporters. * "These cyber special operations are large scale, resource intensive and sustained," Nimmo added. Driving the news: An individual tied to Chinese law enforcement used ChatGPT to continuously edit and polish updates to reports about their so-called "cyber special operations." * The updates suggest that Chinese law enforcement has built and is expanding a strategy to "suppress dissent and silence critics both online and offline" around the world using hundreds of people, thousands of fake accounts and locally deployed AI models, according to the report. * The updates also referenced plans for a large-scale influence operation partially powered by Chinese open-weight AI models. What they're saying: "It's not just digital, it's not just about trolling, it's industrialized," Nimmo told reporters. "It's about trying to hit critics of the (Chinese Communist Party) with everything, everywhere, all at once." Zoom in: In mid-October, the user attempted to use ChatGPT to design and refine a campaign aimed at discrediting Sanae Takaichi -- who won a landslide election victory last month -- after she publicly criticized the state of human rights in Inner Mongolia. * Takaichi also infuriated Beijing last year when she suggested that Japan might defend Taiwan in the event of a Chinese invasion. * The plan hinged on six elements, including posting and amplifying negative comments about Takaichi on social media; sending complaints to Japanese politicians using fake email accounts posing as foreign residents; and accusing Takaichi of far-right leanings. Yes, but: ChatGPT refused to help the individual refine the campaign. * Instead, the user then returned a few weeks later to update a report that indicated the campaign went ahead, likely using locally hosted Chinese AI models, according to OpenAI. * That update also suggested the user included a set of hashtags in its social media operations. OpenAI researchers traced those to posts on X, Blogspot and Pixiv, a popular online Japanese community for artists. The big picture: Many of the other influence operations outlined in OpenAI's report reflect the same old tools and tactics that influence operators typically use in online campaigns -- just supercharged with AI. * ChatGPT helped Cambodia-based scammers create marketing materials for a fake online dating service used in romance scams. * The chatbot also helped Russian-based actors translate social media comments in Spanish for an operation targeting Argentina. What to watch: Whether scammers and nation-state operators change their tactics after OpenAI made their techniques and tells known publicly.