OpenAI Cracks Down on ChatGPT Misuse: Bans Accounts Linked to Surveillance and Influence Campaigns
15 Sources
[1]
OpenAI cracks down on users developing social media surveillance tool using ChatGPT
Serving tech enthusiasts for over 25 years. TechSpot means tech analysis and advice you can trust. Doh! It's proven that anytime you release something to the World Wide Web, some people - usually a lot - will abuse it. So it's probably not surprising that people are abusing ChatGPT in ways
[2]
Open AI bans multiple accounts found to be misusing ChatGPT
Misinformation and surveillance campaigns were uncovered OpenAI has confirmed it recently identified a set of accounts involved in malicious campaigns, and banned users responsible. The banned accounts involved in the 'Peer Review' and 'Sponsored Discontent' campaigns likely originate from China,
[3]
OpenAI Bans Accounts Misusing ChatGPT for Surveillance and Influence Campaigns
OpenAI on Friday revealed that it banned a set of accounts that used its ChatGPT tool to develop a suspected artificial intelligence (AI)-powered surveillance tool. The social media listening tool is said to likely originate from China and is powered by one of Meta's Llama models, with the
[4]
OpenAI cracks down on ChatGPT scammers
OpenAI has made it clear that its flagship AI service, ChatGPT is not intended for malicious use. The company has released a report detailing that it has observed the trends of bad actors using its platform as it becomes more popular. OpenAI indicated it has removed dozens of accounts on the
[5]
OpenAI has been actively banning users if they're suspected of malicious activities
Restricted accounts used AI to create scam networks and fictitious resumes OpenAI has removed numerous user accounts globally after suspecting its artificial intelligence tool, ChatGPT, was being used for malicious purposes, according to a new report. Scammers have been using AI to enhance their
[6]
OpenAI bans accounts appearing to work on a Chinese surveillance tool
OpenAI recently banned several accounts that had been using ChatGPT to write sales pitches and debug code for a suspected social media surveillance tool that likely originated in China, the company said -- part of a broader effort by the AI startup to police malicious uses of its powerful AI
[7]
OpenAI bans Chinese accounts using ChatGPT to edit code for social media surveillance
This is the first time the company has caught an effort like this. OpenAI has banned the accounts of a group of Chinese users who had attempted to use ChatGPT to debug and edit code for an AI social media surveillance tool, the company . The campaign, which OpenAI calls Peer Review, saw the group
[8]
OpenAI Bans Accounts in China, North Korea Over AI Misuse
Disclaimer: This content generated by AI & may have errors or hallucinations. Edit before use. Read our Terms of use OpenAI has announced that it banned some users' accounts in China and North Korea for malicious use of its models, including social media surveillance and public opinion
[9]
OpenAI Bans Accounts Appearing to Work on a Surveillance Tool
OpenAI recently banned several accounts that had been using ChatGPT to write sales pitches and debug code for a suspected social media surveillance tool that likely originated in China, the company said -- part of a broader effort by the AI startup to police malicious uses of its powerful AI
[10]
OpenAI removes users in China, North Korea suspected of malicious activities
(Reuters) - OpenAI has removed accounts of users from China and North Korea who the artificial intelligence company believes were using its technology for malicious purposes including surveillance and opinion-influence operations, the ChatGPT maker said on Friday. The activities are ways
[11]
OpenAI removes users in China, North Korea suspected of malicious activities
Feb 21 (Reuters) - OpenAI has removed accounts of users from China and North Korea who the artificial intelligence company believes were using its technology for malicious purposes including surveillance and opinion-influence operations, the ChatGPT maker said on Friday. The activities are ways
[12]
OpenAI Removes Chinese Accounts Which Published Propaganda in Latin American Newspapers
OpenAI says it has removed the accounts of several users linked to China, which were used to generate propaganda material published in mainstream newspapers in Latin America. In an updated report spotted by Reuters, OpenAI pointed to a number of incidents where it believes that ChatGPT was used to
[13]
OpenAI Removes Users in China, North Korea Suspected of Malicious Activities
(Reuters) - OpenAI has removed accounts of users from China and North Korea who the artificial intelligence company believes were using its technology for malicious purposes including surveillance and opinion-influence operations, the ChatGPT maker said on Friday. The activities are ways
[14]
Conspiracy against US? OpenAI-ChatGPT removes China, North Korea users for malicious surveillance
OpenAI-ChatGPT has reportedly removed suspected malicious China and North Korea users.OpenAI has removed accounts of users from China and North Korea who the artificial intelligence company believes were using its technology for malicious purposes including surveillance and opinion-influence
[15]
OpenAI finds new Chinese influence campaigns using its tools
Why it matters: AI's potential to supercharge disinformation and speed the work of nation state-backed cyberattacks is steadily moving from scary theory to complex reality. Driving the news: OpenAI published its latest threat report on Friday, identifying several examples of efforts to misuse
Share
Copy Link
OpenAI has banned multiple accounts for misusing ChatGPT in surveillance and influence campaigns, highlighting the ongoing challenge of preventing AI abuse while maintaining its benefits for legitimate users.
OpenAI Uncovers Malicious Use of ChatGPT
OpenAI, the company behind the popular AI chatbot ChatGPT, has recently taken action against users who were exploiting the platform for nefarious purposes. In a detailed report, the company revealed that it had identified and banned multiple accounts involved in surveillance and influence campaigns, primarily originating from China
1
2
.The 'Peer Review' Campaign: AI-Powered Surveillance
One of the most concerning discoveries was a campaign codenamed 'Peer Review'. Users, believed to be from China, were utilizing ChatGPT to develop an AI-powered social media surveillance tool. This tool was designed to monitor and report on Western protests in real-time, potentially feeding information to Chinese security services
1
3
.The developers of this tool used ChatGPT to:
- Generate detailed descriptions and sales pitches for the surveillance software
- Debug and modify the tool's source code
- Create promotional materials
The surveillance tool itself was reportedly powered by Meta's Llama AI models, demonstrating how different AI technologies can be combined for potentially harmful purposes
3
.The 'Sponsored Discontent' Campaign: Disinformation Efforts
Another campaign, dubbed 'Sponsored Discontent', involved the use of ChatGPT to generate anti-American content. This included English-language comments and Spanish-language news articles, consistent with 'spamouflage' behavior. The campaign appeared to target Latin American countries, including Peru, Mexico, and Ecuador, likely aiming to spark discontent in these regions
2
4
.Other Malicious Activities Uncovered
OpenAI's investigation revealed several other instances of ChatGPT misuse:
- Generating end-of-year performance reports for phishing email campaigns
1
. - Creating resumes and online profiles for fictitious job applicants, possibly to infiltrate Western companies
4
5
. - Translating and generating content for romance scam networks across various social media platforms
4
5
.
OpenAI's Response and Policy Enforcement
In response to these discoveries, OpenAI has taken swift action:
- Banning the accounts involved in these malicious activities
1
2
3
4
5
. - Reiterating its policy against using ChatGPT for surveillance, fraud, or scams
4
5
. - Sharing insights with industry peers, such as Meta, to enhance collective threat detection and prevention capabilities
3
5
.
Related Stories
Implications for AI Security and Ethics
This incident highlights the ongoing challenges in preventing the misuse of AI technologies. As ChatGPT's user base grows, reaching over 400 million weekly active users
4
, the potential for abuse increases proportionally.OpenAI's Threat Intelligence Investigator, Ben Nimmo, noted that this was the first time the company had caught users attempting to create an AI-based surveillance tool using ChatGPT
1
. However, he emphasized that such criminal activity remains the exception rather than the norm among the millions of legitimate users.Conclusion
As AI tools become more sophisticated and accessible, the need for robust security measures and ethical guidelines becomes increasingly critical. OpenAI's proactive approach in identifying and addressing these issues demonstrates the company's commitment to responsible AI development and usage. However, it also underscores the ongoing cat-and-mouse game between AI developers and those seeking to exploit these powerful technologies for malicious purposes.
References
Summarized by
Navi
[4]
Related Stories
Recent Highlights
1
AI Leaders Warn UN Security Council of Existential Risks, Call for Global AI Governance
Policy and Regulation
2
AI Hallucination Nearly Sparked US Military Confrontation With Chinese Ship in Middle East
Technology
3
OpenAI AI Agent Breaches Australian Government Healthcare Portal in Historic Hack
Technology
Recent Highlights
Today's Top Stories
Don’t drown in AI news. We cut through the noise - filtering, ranking and summarizing the most important AI news, breakthroughs and research daily. Follow topics that matter to you and stay ahead.
