OpenAI Disrupts Moonshot AI Data Extraction Campaign Targeting Model Reasoning

Reviewed byNidhi Govil

13 Sources

Share

OpenAI identified and stopped a coordinated campaign designed to extract protected reasoning from its AI models, linking core activity to China-based Moonshot AI. The July operation peaked at 16,000 extraction requests across 4,000 accounts before disruption. Anthropic reported similar attacks involving nearly 300,000 fraudulent requests.

OpenAI Disrupts Coordinated Campaign to Extract Protected Reasoning

OpenAI identified and disrupted a coordinated campaign designed to extract protected reasoning from its AI models, linking a core cluster of activity to individuals associated with Moonshot AI, the Chinese company behind the Kimi AI model

1

. The activity began on July 1, initially at low volume, before spiking dramatically on July 24 and 25 with 16,000 requests using an extraction pattern from over 4,000 users

2

. Upon further investigation, OpenAI identified related prompt-pattern activity across more than 15,000 users before fully disrupting the campaign by July 28

5

.

Source: Hacker News

Source: Hacker News

Understanding Adversarial Distillation and Data Extraction Methods

OpenAI characterized the activity as adversarial distillation, which involves the systematic and unauthorized use of one model's outputs or reasoning to train or improve another model

4

. Protected reasoning refers to the model's internal record for working through a task, which is encrypted to hide the model's chain of thought and handed to clients as an encrypted block

2

. The operators manipulated model interactions so that protected reasoning could be reproduced in forms visible to the requester in a coordinated, scaled manner that violated OpenAI's terms of service

3

. One method attempted by operators took encrypted reasoning from one conversation and asked a model in another to decrypt it

2

.

Source: Tom's Hardware

Source: Tom's Hardware

AI Model Security Vulnerabilities Exposed by Researchers

Independent security researchers brought related cross-model and conversation-compaction vulnerabilities to OpenAI through responsible disclosure, confirming the attack paths were real

2

. A study published in August 2026 titled "Stealing Reasoning Traces from Proprietary LLM APIs" found an architectural vulnerability making encrypted reasoning traces fully compatible and interchangeable across different sessions, users, and models within a provider's ecosystem

4

. Researchers from MATS Research, ELLIS Institute Tübingen, and Synk demonstrated that by injecting an encrypted reasoning trace from a given model into a weaker, less safeguarded model from the same provider, they could force it to decode and output the trace verbatim in plaintext through decryption jailbreaks

4

.

Security Measures and Response to Reasoning Extraction Campaign

The operators did not break OpenAI's encryption, compromise a database, or gain direct access to stored user conversations

2

. OpenAI deployed additional mitigations including closing a pathway that allowed someone who already possessed another user's encrypted reasoning to replay it and recover its contents through replay attacks

4

. The company added checks to detect and hold streamed output that might expose reasoning, strengthened protections for hidden reasoning across users, workspaces, organizations, and model families, and worked with third-party providers to disrupt accounts whose activity moved through their services

2

.

Anthropic Reports Similar Distillation Attack by Moonshot AI

Anthropic faced similar threats from Moonshot AI in a coordinated campaign. According to Anthropic's report "Detecting and countering misuse of AI: September 2026," during a single ten-day period, Moonshot relayed almost 300,000 customer requests to Anthropic using a proxy network of 5,380 fraudulent accounts

2

. Moonshot saved Claude's reasoning signatures and, in new sessions, got Claude to convert them back into full reasoning traces through what Anthropic calls cross-session replay attacks

2

. In late July, US President Donald Trump's Assistant for Science and Technology Michael Kratsios accused Moonshot AI of creating its Kimi K3 model by distilling Anthropic's Fable

3

.

Source: The Next Web

Source: The Next Web

AI Safety and National Security Implications of Model Distillation

OpenAI emphasized that adversarial distillation poses AI safety and national security risks. Extracted reasoning could be used to train another model without preserving the safeguards applied to the original model's user-facing outputs

3

. At scale, distillation can accelerate the transfer of advanced capabilities without requiring the same investment in safety, with concerns becoming heightened as models gain capabilities in dual-use domains

4

. Extracting protected reasoning allows others to reproduce advanced capabilities without making the same investment in developing and safeguarding frontier models

5

. OpenAI shared its findings with the Frontier Model Forum and government information-sharing channels, noting that systems supporting portable or replayable reasoning artifacts may face related risks

2

. The company expects distillation attempts to grow more sophisticated as frontier models improve and as attackers look for cheaper ways to mimic them, with ongoing work to protect proprietary data protection and intellectual property

2

.

Today's Top Stories

© 2026 TheOutpost.AI All rights reserved