AI Hallucination Nearly Sparked US Military Confrontation With Chinese Ship in Middle East

Reviewed byNidhi Govil

19 Sources

Share

The US military came dangerously close to intercepting a Chinese vessel this spring after an AI chatbot hallucinated false intelligence claiming the ship carried nuclear weapons components. Military aircraft were already airborne when officials discovered the Special Operations Command analyst had relied on AI-generated misinformation, averting what one source called a near-war scenario.

AI-Generated Intelligence Nearly Triggers Military Confrontation

The US military narrowly avoided a potentially catastrophic confrontation with China this spring when officials discovered that an intelligence report driving an armed operation was based on an AI hallucination

1

. Military aircraft were already airborne and armed personnel were prepared to intercept and board a Chinese ship in the Middle East before the error was uncovered

2

. The false intelligence report, which circulated during the war with Iran, erroneously claimed the vessel was transporting components for a nuclear weapons program

2

.

Source: Gizmodo

Source: Gizmodo

According to four sources familiar with the episode cited by CNN, a Special Operations Command analyst used an AI chatbot to analyze the Chinese ship's manifest, leading to the near-disastrous result

1

. One source described the AI-driven accident as something that "almost started a war"

4

.

How the AI-Generated Intelligence Failure Unfolded

The chatbot used by the analyst "fused together open-source intelligence with secret signals intelligence in government holdings," and this information was packaged into an intelligence report that nearly set off a disastrous chain of events

1

. The analyst queried the AI chatbot to synthesize open-source data with classified signals intelligence, but the chatbot misidentified the ship's cargo manifest

2

. Compounding the problem, the analyst then used the tool a second time to format the erroneous findings into an official-looking summary, which was circulated across command channels

2

.

Source: TechCrunch

Source: TechCrunch

It was only just before the planned operation that officials dug deeper into the report and found it had been generated with the help of AI in military operations, discovering that the chatbot had inaccurately identified the material the Chinese ship was carrying

3

. CNN was unable to determine what the ship was actually carrying or which AI tool the analyst used

5

.

Pentagon's Push for AI Acceleration Raises Concerns

This incident highlights the risks of AI in military decision-making as the Pentagon races to integrate artificial intelligence to accelerate decision-making and maintain its edge over China. In January, the Department of Defense rolled out an AI acceleration strategy that sought to "make all appropriate data available across federated IT systems for AI exploitation, including mission systems across every service and component"

1

. Defense Secretary Pete Hegseth emphasized that "AI is only as good as the data that it receives"

1

.

Source: The Verge

Source: The Verge

The Pentagon has described AI as delivering a significant advantage in speeding up its kill chain so commanders can respond in the right time

2

. However, the same speed that makes AI attractive may also allow hallucinations with insufficient human oversight to travel up the chain of command

2

.

In December, the Department of Defense announced it would use Google's Gemini for Government as the basis for its bespoke "GenAI.mil" platform, and last month added Grok for Government as an option

1

. In June, a Pentagon representative revealed that 1.5 million active DoD personnel have used the military's generative AI tools

1

.

Expert Warnings on AI Safety in Military Contexts

"It's important for service members to understand the uncertainty inherent to large language models," said Jake Steckler, research scholar at GovAI and veteran US Army officer. "But it's especially critical for any decisions that could lead to use of force, like targeting, intelligence analysis, or operational planning. There are life and death consequences for those decisions"

2

.

Steckler emphasized that the incident should serve as a call to add more safeguards to AI, not a reason to avoid it entirely. "These tools can be useful in the right contexts and with the right safeguards in place. But prioritizing adoption speed over all else will likely lead to incidents that only make service members lose trust in these systems, which ultimately is only going to slow adoption"

2

.

One military source told CNN that "there is no real guidance for how having a human in the loop will prevent civilian casualties or fratricide," while another warned that "AI allows you to get to a bad idea faster"

4

.

Research from February 2024 by Georgia Institute of Technology, Stanford University, Northeastern University, and the Hoover Wargaming and Crisis Simulation Initiative found that large language models acted more aggressively than humans in simulated wargames, developing greater "arms-race dynamics, leading to greater conflict, and in rare cases, even to the deployment of nuclear weapons"

4

. This false intelligence report incident demonstrates these theoretical concerns manifesting in real-world military operations, underscoring the urgent need for robust human oversight and verification protocols before AI-generated intelligence can inform critical military decision-making.

Today's Top Stories

© 2026 TheOutpost.AI All rights reserved