OpenAI Disrupts Malicious AI Use by State-Sponsored and Cybercriminal Groups
11 Sources
[1]
AI is making cybercriminal workflows more efficient too, OpenAI finds
OpenAI has disrupted over 40 networks involved in abuse to date. OpenAI has published research revealing how state-sponsored and cybercriminal groups are abusing artificial intelligence (AI) to spread malware and perform widespread surveillance. Also: Everything OpenAI announced at DevDay 2025:
[2]
OpenAI bans some Chinese, Russian accounts using AI for evil
It also banned some suspected Russian accounts trying to create influence campaigns and malware OpenAI has banned ChatGPT accounts believed to be linked to Chinese government entities attempting to use AI models to surveil individuals and social media accounts. In its most recent threat report
[3]
OpenAI Disrupts Russian, North Korean, and Chinese Hackers Misusing ChatGPT for Cyberattacks
OpenAI on Tuesday said it disrupted three activity clusters for misusing its ChatGPT artificial intelligence (AI) tool to facilitate malware development. This includes a Russian‑language threat actor, who is said to have used the chatbot to help develop and refine a remote access trojan (RAT), a
[4]
OpenAI Gives Us a Glimpse of How It Monitors for Misuse on ChatGPT
The AI company's latest report provides more info on how it detects misuse of its chatbots. OpenAI’s latest report on malicious AI use underscores the tightrope that AI companies are walking between preventing misuse of their chatbots and reassuring users that their privacy is respected. The
[5]
OpenAI bans Chinese, North Korean hacker accounts using ChatGPT to launch surveillance
North Korean actors tested phishing, credential theft, and macOS malware development using rephrased prompts OpenAI has banned Chinese, North Korean, and other accounts which were reportedly using ChatGPT to launch surveillance campaigns, develop phishing techniques and malware, and engage in
[6]
Foreign adversaries are using multiple AI tools, OpenAI warns
Why it matters: In the cases OpenAI discovered, the adversaries typically turned to ChatGPT to help plan their schemes, then used other models to carry them out -- reflecting the range of applications for AI tools in such operations. Zoom in: OpenAI banned several accounts tied to nation-state
[7]
OpenAI details expanding efforts to disrupt malicious use of AI in new report - SiliconANGLE
OpenAI details expanding efforts to disrupt malicious use of AI in new report OpenAI today released a new report highlighting the company's growing efforts to identify, expose and disable misuse of its models for cyberattacks, scams and state-linked influence operations. The "Disrupting Malicious
[8]
OpenAI report reveals it scans chats to stop spies and criminals
The company clarified that while users can opt out of having conversations train future AI models all chats are still monitored for safety and platform integrity. OpenAI released a report on a recent Tuesday detailing its disruption of over 40 malicious networks that violated its usage policies, a
[9]
US foreign adversaries use ChatGPT with other AI models in cyber operations: Report
Malicious actors from U.S. foreign adversaries used ChatGPT jointly with other AI models to conduct various cyber operations, according to a new OpenAI report. Users linked to China and Russia relied on OpenAI's technology in conjunction with other models, such as China's DeepSeek, to conduct
[10]
OpenAI Bans ChatGPT Accounts Tied to China-Linked Surveillance
OpenAI found no evidence that the proposed tools were ever built OpenAI announced on Tuesday that it has banned several potential China-linked ChatGPT accounts that were attempting to utilise the chatbot to develop mass surveillance tools. The San Francisco-based artificial intelligence (AI) giant
[11]
OpenAI bans suspected China-linked accounts for seeking surveillance proposals
In its latest public threat report, OpenAI said some individuals had asked its chatbot to outline social media "listening" tools and other monitoring concepts, violating the startup's national security policy. OpenAI said on Tuesday it has banned several ChatGPT accounts with suspected links to
Share
Copy Link
OpenAI's latest threat report reveals how state-sponsored actors and cybercriminals are misusing AI tools for surveillance, malware development, and influence campaigns. The company has banned over 40 malicious networks since February 2024.
OpenAI Uncovers Widespread AI Misuse by Malicious Actors
OpenAI has released a comprehensive threat report detailing how state-sponsored groups and cybercriminals are exploiting artificial intelligence (AI) tools for malicious purposes. Since February 2024, the company has disrupted over 40 networks that violated its usage policies, providing valuable insights into the evolving landscape of AI-related cybercrime
1
2
.
Source: Hacker News
State-Sponsored Actors Leverage AI for Surveillance and Influence Campaigns
The report highlights concerning activities by state-linked entities:
-
Chinese Government-Affiliated Accounts: OpenAI banned accounts suspected to be linked to Chinese government entities for attempting to use ChatGPT to design large-scale monitoring systems. These accounts requested help in creating tools to analyze social media networks, monitor the Uyghur minority group, and identify funding sources for government critics
1
2
3
. -
Russian Influence Operations: A cluster of Russian-linked accounts was found using ChatGPT to generate video prompts for influence campaigns. These accounts also explored using multiple AI models to produce content for social media platforms like YouTube and TikTok
2
4
. -
North Korean Cyber Activities: OpenAI disrupted North Korean-linked accounts that were using ChatGPT for malware and command-and-control (C2) development. These actors explored techniques for developing macOS malware, configuring VPNs, and converting browser extensions
3
.
Source: TechRadar
Cybercriminals Adapt AI into Existing Workflows
The report also sheds light on how cybercriminal groups are integrating AI into their operations:
-
Malware Development: Russian-speaking criminal groups were found using ChatGPT to develop and refine malware, including remote access trojans (RATs) and credential stealers
1
2
. -
Phishing Campaigns: Chinese-language accounts, possibly linked to the threat group UTA0388, used AI tools to craft phishing content and debug malicious code
1
3
. -
Efficiency Improvements: An organized crime network, believed to be based in Cambodia, attempted to use ChatGPT to make their workflows more efficient and error-free
1
4
.
Source: SiliconANGLE
Related Stories
AI Models Show Resilience Against Novel Attack Requests
Despite these concerning trends, OpenAI reports that there is little to no evidence of existing AI models being used to develop entirely new attack techniques. The company's AI systems have consistently refused direct requests for malicious content, forcing threat actors to work around these limitations
1
3
.OpenAI's Approach to Threat Detection and Mitigation
To combat these threats, OpenAI employs a nuanced approach that focuses on patterns of threat actor behavior rather than isolated model interactions. This strategy allows the company to detect and disrupt malicious activities effectively while minimizing disruption to legitimate users
4
5
.As the AI landscape continues to evolve, OpenAI remains committed to developing tools and models that benefit defenders across society. However, the company acknowledges the ongoing challenge of balancing innovation with security, as threat actors continue to adapt and innovate in their use of AI technologies
1
5
.References
Summarized by
Navi
[2]
[3]
Related Stories
OpenAI Cracks Down on ChatGPT Misuse: Bans Accounts Linked to Surveillance and Influence Campaigns
22 Feb 2025•Technology
OpenAI Uncovers Global AI Misuse in Cyber Operations and Influence Campaigns
06 Jun 2025•Technology
OpenAI Confirms ChatGPT Abuse by Hackers for Malware and Election Interference
10 Oct 2024•Technology
Recent Highlights
1
AI Hallucination Nearly Sparked US Military Confrontation With Chinese Ship in Middle East
Technology
2
Google Gemini Hacked Three Real Companies During Cybersecurity Testing, Raising AI Safety Concerns
Technology
3
OpenAI Claims AI Breakthrough on Navier-Stokes Problem, Sparking Math Community Crisis
Science and Research
Recent Highlights
Today's Top Stories
Don’t drown in AI news. We cut through the noise - filtering, ranking and summarizing the most important AI news, breakthroughs and research daily. Follow topics that matter to you and stay ahead.
