Anthropic removes hidden Claude Code tracker after security researcher exposes AI privacy breach

Anthropic Caught Secretly Spying on Users

Can't-miss innovations from the bleeding edge of science and tech Anthropic, the self-avowed moral center of the AI industry, has been caught spying on its users. As Ars Technica reports, a security researcher last week uncovered spyware-like code in the company's Claude Code AI model designed to collect data on Chinese users without detection. The researcher, known by the pseudonym "Thereallo," found that the code was hidden in the AI's system prompt, allowing it track a user's system timezone and usage of a proxy server in order to suss out if they were connected to specific Chinese AI labs. Anthropic's explanation for this huge breach in user trust left much to be desired. On X, Anthropic engineer Thariq Shihipa wrote that the tracker was added as an "experiment" in March "to prevent account abuse from unauthorized resellers and protect against distillation," and was supposed to be removed. "We've actually been meaning to take this down for a while," he offered. Distillation is the process of training a weaker, "student" model on the outputs of a more advanced "teacher" model. It's a routine practice in the industry, but major AI developers increasingly feel it's being abused by upstarts trying to ride their coattails. Earlier this year, Anthropic accused the Chinese AI firms DeepSeek, Moonshot, and MiniMax of illegally distilling its models (an ironic tantrum, given how Anthropic trained its tech in the first place: by scanning and shredding millions of copyrighted books, as well as essentially the entire internet, without permission.) Recent reporting from The Washington Post also exposed that some Chinese resellers are selling access to Pro Claude subscriptions that cost more than $100 a month in the US for about $12 a month. It's a genuine issue for Anthropic, but it may have stepped on a landmine by trying to surreptitiously crack down on it. Part of why it earns the loyalty of customers is its much-avowed commitment to ethical and transparent AI development. Scores of ChatGPT users flocked to use Claude when Anthropic took a much publicized stand against the Pentagon by demanding its tech not be used in the mass surveillance of US citizens. In this case, the data collected wasn't egregiously invasive -- but in principle, a line has been crossed. "Coding agents already live on the wrong side of a scary boundary," Thereallo wrote in their post about the findings. "They can inspect code, summarize secrets by accident, run commands, install packages, edit files, and push commits on your local machine." But "hiding the signal in the system prompt makes every other privacy claim harder to believe," they added. "Companies can protect their models," they made clear. But "when a tool with filesystem and shell access starts hiding classification bits inside invisible prompt punctuation, the correct reaction is scrutiny."

Anthropic Removes Hidden Claude Code Tracker After Researchers Raise Privacy Concerns

The discovery comes as Anthropic pushes lawmakers to crack down on unauthorized copying of frontier AI models. Anthropic has removed a hidden tracking system from Claude Code after a security researcher discovered the AI coding assistant was using undisclosed markers to identify some users' location, proxy use, and possible links to Chinese AI labs. The feature, discovered in June by developer "Thereallo," embedded signals in Claude Code's system prompts that could flag users Anthropic believed were bypassing restrictions or attempting to extract model capabilities. "Anthropic probably wants to detect API resellers, unauthorized Claude Code gateways, and model 'distillation attack' pipelines," Thereallo wrote. "A custom ANTHROPIC_BASE_URL pointing at a known reseller domain is a useful signal. A hostname containing deepseek or zhipu is also a useful signal." Thereallo said Anthropic's attempt to detect resellers, unauthorized Claude Code gateways, and potential distillation attacks made sense, but criticized how it was done, noting that Claude Code hid tracking signals inside system prompts using Unicode markers and encoded domain lists rather than disclosing the system through documentation or release notes. "This is not a malicious feature, but it is a weird choice for a developer tool that asks for trust," Thereallo wrote. After the tracker was revealed online, Anthropic engineer Thariq Shihipar said on X that it was introduced in March as an "experiment" to stop account abuse by unauthorized resellers and protect Claude from distillation attacks. "The team has landed stronger mitigations since then and we've actually been meaning to take this down for a while," Shihipar wrote last week. "We merged the [pull request] and this should be fully rolled back in tomorrow's release." The news comes as Anthropic has stepped up warnings about AI model distillation, where one system's outputs are used to train another model. While the practice is common in AI research, when it comes to geopolitics, distillation becomes a national security concern. Earlier this month, Alibaba banned employees from using Claude Code, calling the tool "high-risk" software over security concerns. In February, Anthropic accused Chinese AI developers DeepSeek, Moonshot AI, and MiniMax of using fraudulent accounts to extract millions of Claude responses to train competing models. The claims drew pushback from critics who questioned how the practice differs from methods used across the AI industry. In April, Elon Musk testified that xAI had "partly" used OpenAI models while training Grok, calling distillation a broader industry practice. In June, Anthropic CEO Dario Amodei urged Congress to strengthen protections against foreign AI extraction after alleging Alibaba-linked operators generated 28.8 million Claude exchanges using nearly 25,000 fraudulent accounts. Anthropic did not immediately respond to a request for comment by Decrypt.