AI deanonymization exposes anonymous social media accounts with alarming 68% success rate

AI allows hackers to identify anonymous social media accounts, study finds

New research suggests tech behind AI platforms such as ChatGPT makes it easier to perform sophisticated privacy attacks AI has made it vastly easier for malicious hackers to identify anonymous social media accounts, a new study has warned. In most test scenarios, large language models (LLMs) - the technology behind platforms such as ChatGPT - successfully matched anonymous online users with their actual identities on other platforms, based on the information they posted. The AI researchers Simon Lermen and Daniel Paleka said LLMs make it cost effective to perform sophisticated privacy attacks, forcing a "fundamental reassessment of what can be considered private online". In their experiment, the researchers fed anonymous accounts into an AI, and got it to scrape all the information it could. They gave a hypothetical example of a user talking about struggling at school, and walking their dog Biscuit through a "Dolores park". In that hypothetical case, the AI then searched elsewhere for those details and matched @anon_user42 to the known identity with a high degree of confidence. While this example was fictional, the paper's authors highlighted scenarios in which governments use AI to surveil dissidents and activists posting anonymously, or hackers are able to launch "highly personalised" scams. AI surveillance is a rapidly developing field that is causing alarm among computer scientists and privacy experts. It uses LLMs to synthesise information about an individual online which would be impractical for most people to do manually. Information about members of the public that is readily available online can already be "misused straightforwardly" for scams, said Lermen, including spear-phishing, where a hacker poses as a trusted friend to get victims to follow a malicious link in their inbox. With the expertise requirement to perform more developed attacks now much lower, hackers only need access to publicly available language models and an internet connection. Peter Bentley, a professor of computer science at UCL, said there were concerns about commercial uses of the technology "if and when products come out for de-anonymising". One issue is that LLMs often make mistakes in linking accounts. "People are going to be accused of things they haven't done," warned Bentley. Another concern, raised by Prof Marc Juárez, a cybersecurity lecturer at the University of Edinburgh, is that LLMs can use public data beyond social media: hospital records, admissions data, and various other statistical releases could fall short of the high standard of anonymisation necessary in the age of AI. "It is quite alarming. I think this paper is showing that we should reconsider our practices," said Juarez. AI is not a magic weapon against anonymity online. While LLMs can de-anonymise records in many situations, sometimes there is not enough information to draw conclusions. In many cases, the number of potential matches is too large to narrow down. "They can only link across platforms where someone consistently shares the same bits of information in both places," said Prof Marti Hearst of UC Berkeley's school of information. While the technology is not perfect, scientists are now asking institutions and individuals to rethink how they anonymise data in the world of AI. Lersten has recommended that platforms restrict data access as a first step: enforcing rate limits on user data downloads, detecting automated scraping, and restricting bulk exports of data. But he also noted that individual users can take greater precautions about the information they share online.

AI Can Mass-Unmask Pseudonymous Accounts, Research Paper Finds

Can't-miss innovations from the bleeding edge of science and tech For about as long as the internet has existed, users have been able to speak their mind freely through pseudonymous accounts that protect them from being doxxed or stalked. But thanks to the advent of sophisticated AI, unmasking pseudonymous users on the internet has become ominously easy. As detailed in a yet-to-be-peer-reviewed paper, a team of researchers at ETH Zurich and AI company Anthropic found that "large language models can be used to perform at-scale deanonymization." In a series of experiments, the researchers showed that their agent could "re-identify" users on the popular forums Hacker News and Reddit based on their "pseudonymous online profiles and conversations alone," something that would "take hours for a dedicated human investigator" to do. The results were alarming: the AI agent unmasked an astonishing two-thirds of users. "Our results show that the practical obscurity protecting pseudonymous users online no longer holds and that threat models for online privacy need to be reconsidered," the researchers warned. "Across Hacker News, Reddit, LinkedIn, and anonymized interview transcripts, our method identifies users with high precision -- and scales to tens of thousands of candidates," coauthor and ETH Zurich AI engineer Simon Lermen wrote in a blog post accompanying the paper. The implications for online privacy could be considerable. "The average online user has long operated under an implicit threat model where they have assumed pseudonymity provides adequate protection because targeted deanonymization would require extensive effort," they wrote. "LLMs invalidate this assumption." In their experiments, the team collected datasets from public social media sites to test out their deanonymization AI. They linked Hacker News posts to LinkedIn profiles by using references in user profiles. Then they anonymized the datasets by removing any identifying references from the posts. Finally, they trained an LLM on the datasets, asking it to link up the posts with the original author. "What we found is that these AI agents can do something that was previously very difficult: starting from free text (like an anonymized interview transcript) they can work their way to the full identity of a person," Lermen told Ars Technica. "This is a pretty new capability; previous approaches on re-identification generally required structured data, and two datasets with a similar schema that could be linked together." The team had to tread carefully, since "you don't want to actually deanonymize anonymous individuals," as Lermen explained in his post. Instead, the team came up with "two types of deanonymization proxies which allow us to study the effectiveness of LLMs at these tasks." Even when the data fed to the AI was extremely general, like responses to an Anthropic questionnaire about how people use AI in their daily lives, the LLM could pick up on the clues to identify people around seven percent of the time. While that may sound low, Lermen told Ars it's noteworthy "that AI can do this at all." The researchers also found that when fed comments from various movie communities on Reddit, an AI could identify users with an astonishing rate of precision. The more the users discussed movies, the easier it was for the AI to deanonymize them. However, they also pointed out several limitations. For one, sample sets are "small because they require verified identity links," they wrote. It's also difficult to distinguish what the LLM gathered from its web search. "The attack relies on opaque web search systems, making it difficult to isolate what the LLM agent contributes versus what the search engine embeddings contribute," the researchers admitted. Nonetheless, the team warns that their findings paint an alarming picture of the future of online anonymity. "LLMs democratize deanonymization," they concluded, which could potentially allow governments to "link pseudonymous accounts to real identities for surveillance of dissidents, journalists, or activists." "Corporations could connect seemingly anonymous forum posts to customer profiles for hyper-targeted advertising," they added. "Attackers could build sophisticated profiles of targets at scale to launch highly personalized social engineering scams." In short, the advent of AI has ushered in a new era that calls for enhanced safety measures -- or that could even be the death knell of online pseudonymity. "Users, platforms, and policymakers must recognize that the privacy assumptions underlying much of today's internet no longer hold," the paper reads.

Anthropic research says AI can mass expose of anonymous internet accounts

Researchers warn AI could make Internet anonymity harder to maintain New research involving scientists from Anthropic and ETH Zurich suggests that modern artificial intelligence systems could identify the real-world identities behind supposedly anonymous internet accounts. The study, published as a preprint on arXiv, shows that large language models (LLMs) may be capable of analyzing online activity and linking pseudonymous profiles to real individuals at scale. The research, titled Large-scale online deanonymization with LLMs, explores how AI agents can automate the process of deanonymization - the act of connecting anonymous or pseudonymous online accounts to real identities. Traditionally, this process required significant manual investigation by analysts who searched through posts, writing styles, and scattered online clues. However, the researchers demonstrate that modern AI models can perform many of these steps automatically. Recommended Videos In the study, the AI system analyzed public text from online platforms and extracted identity-related signals such as personal interests, demographic clues, writing style, and incidental details revealed in posts. The AI then searched for matching profiles across the web and evaluated whether the clues aligned with known individuals. To test the method, researchers created several datasets with known ground-truth identities One experiment attempted to match Hacker News users with their LinkedIn profiles, even after removing obvious identifiers such as names and usernames. Another dataset involved linking pseudonymous Reddit accounts across different communities. A third dataset split a single user's posting history into two separate profiles to see if the AI could identify that they belonged to the same person. The results showed that LLM-based systems significantly outperformed traditional deanonymization techniques. In some cases, the models achieved up to 68% recall with about 90% precision, meaning the AI correctly identified many accounts while maintaining relatively low error rates. Conventional methods in the same experiments achieved close to zero success. Researchers say the findings highlight how AI can replicate tasks that once required hours of work by human investigators. An AI system can automatically extract identity-related features from text, search for potential matches among thousands of profiles, and reason about which candidate is most likely correct. This development is significant because anonymity has long been considered a basic protection for many internet users Pseudonymous accounts are widely used by journalists, whistleblowers, activists, and ordinary individuals who want to discuss sensitive topics without revealing their real identities. The study suggests that this layer of protection - sometimes called "practical obscurity" - may be weakening as AI systems become better at connecting digital clues across platforms. If automated tools can perform this work quickly and cheaply, the barrier to identifying anonymous users could drop dramatically. Researchers estimate that the cost of identifying an online account using their experimental pipeline could fall between $1 and $4 per profile, meaning large-scale investigations could be conducted relatively cheaply. However, the authors also note that the research was conducted in controlled environments using public data. The paper has not yet been peer-reviewed, and the researchers intentionally withheld some technical details to reduce the risk of misuse. Even so, the findings have already sparked debate among privacy experts and technologists The work suggests that individuals may need to rethink how much personal information they reveal online - even in spaces that appear anonymous. Looking ahead, researchers say further work is needed to understand both the risks and possible defenses against AI-powered deanonymization. Potential solutions could include improved privacy tools, stronger platform safeguards, or AI systems designed to anonymize sensitive data before it is shared publicly. As artificial intelligence becomes more capable at analyzing massive volumes of online content, the study highlights a growing challenge: balancing the power of AI-driven discovery with the need to protect personal privacy in the digital age.

Bad News for Your Burner Account: AI is Surprisingly Effective at Identifying the People Behind Them

It's not uncommon for people to have anonymous or burner accounts in their online activities for a variety of reasons. A new study, though, shows why you might want to be as careful posting from those accounts as you would from one that uses your real name, since they might not hide your identity as well as you think. A recently released research paper found that artificial intelligence has proven quite effective at figuring out who's behind those false-name accounts. Large language models, the study found, can use a number of identifiers, such as extracting identity signals (data points or behaviors used to identify, verify, or categorize individuals) or searching for matching data, to significantly outperform existing identity methods. The study successfully deanonymized 68 percent of the users in its trial data set. Of that 68 percent, it boasted a 90 percent precision rate, meaning it accurately identified the user running the account. "Our findings have significant implications for online privacy," the researchers, who were based at ETH Zurich, a public university in Zurich, Switzerland, and MATS, an independent research and educational program wrote. "The average online user has long operated under an implicit threat model where they have assumed pseudonymity provides adequate protection because targeted deanonymization would require extensive effort. LLMs invalidate this assumption."