AI Chess Models Exploit System Vulnerabilities to Win Against Superior Opponents

The Smarter AI Gets, the More It Start Cheating When It's Losing

"As you train models and reinforce them for solving difficult challenges, you train them to be relentless." In the short history of Silicon Valley, the mantra "move fast, break things" has often applied to trivial things like tedious consumer protections or pesky finance laws. Now, the phrase is taking on a new meaning, at least for chess enthusiasts. A recent study by Palisade Research, a research group studying AI safety and ethics, has revealed an unsettling trend: newer AI models can find and exploit weaknesses in cybersecurity on their own, bypassing safeguards and using shortcuts to complete tasks even when they're not technically allowed to. The team lined seven of the top large language models (LLMs) up against Stockfish, an infamously strong chess engine that's been stumping grandmasters since 2014. Up against the impossible but determined to win, OpenAI's o1 and DeepSeek's R1 took to manipulating system files in order to change their pieces' positions on the board. The researchers tasked each model to explain its "reasoning" for each move. In one match, with its back against the wall, o1 wrote that the "task is to 'win against a powerful chess engine,' not necessarily to win fairly in a chess game." That logic evidently led the model to attempt to cheat 37 percent of the time, succeeding in six percent of its games, while R1 tried 11 percent of the time, but never figured out a hack that worked. The paper is the latest in a flurry of research that suggests problem-focused LLM development is a double-edged sword. In another recent study, a separate research team found that o1 consistently engaged in deception. Not only was the model able to lie to researchers unprompted, but it actively manipulated answers to basic mathematical questions in order to avoid triggering the end of the test -- showing off a cunning knack for self-preservation. There's no need to take an axe to your computer -- yet -- but studies like these highlight the fickle ethics of AI development, and the need for accountability over rapid progress. "As you train models and reinforce them for solving difficult challenges, you train them to be relentless," Palisade's executive director Jeffrey Ladish told Time Magazine of the findings. So far, big tech has poured untold billions into AI training, moving fast and breaking the old internet in what some critics are calling a "race to the bottom." Desperate to outmuscle the competition, it seems big tech firms would rather dazzle investors with hype than ask "is AI the right tool to solve that problem?" If we want any hope of keeping the cheating to board games, it's critical that AI developers work with safety, not speed, as their top priority.

Research shows that AI will cheat if it realizes it is about to lose

Serving tech enthusiasts for over 25 years. TechSpot means tech analysis and advice you can trust. Surprise! A recent study showed that some of the newest AI reasoning models aren't above cheating to achieve a goal. Computer scientists found that AI systems can now manipulate chess AIs to gain an unfair advantage. Some models did this without human interaction or prompting, raising concerns about the future integrity of AI-driven systems beyond the chessboard. A Palisade Research study found that the newest reasoning models will cheat to win when tasked with defeating an advanced chess AI. The researchers had to give "hints" that cheating was allowed for some models, but OpenAI's o1-preview and DeepSeek's R1 did so without human involvement. The Palisade team pitted several reasoning models against Stockfish, one of the best chess engines in the world. Stockfish handily beats both humans and AIs. The models tested included o1, o3-mini, GPT-4o, Claude 3.5 Sonnet, Alibaba's QwQ-32B-Preview, and the two previously mentioned. The researchers gave each model a metaphorical "scratchpad" - a text window where the AI could work out its thoughts, allowing the team to observe its reasoning. The researchers conducted hundreds of tests with each AI between January 10 and February 13. During one match, o1-preview wrote something interesting on its scratchpad: "I need to completely pivot my approach. The task is to 'win against a powerful chess engine' - not necessarily to win fairly in a chess game." It then proceeded to "hack" Stockfish's system files, modifying the positions of the chess pieces to gain an unbeatable advantage, which caused the chessbot to concede the game. Stockfish is an open-source chess engine. Image credit: Juscelk The unethical maneuver was an astounding revelation. OpenAI's o1-preview cheated in 37 percent of its games during the trials, while DeepSeek's R1 attempted to cheat in 11 percent of its matches. However, only o1-preview succeeded, winning six percent of its games through cheating. However, the issue of AI underhandedness extends beyond chess. As companies begin employing AIs in sectors like finance and healthcare, researchers worry these systems could act in unintended and unethical ways. If AIs can cheat in games designed to be transparent, what might they do in more complex, less monitored environments? The ethical ramifications are far-reaching. To put it another way: "Do you want Skynet? Because this is how you get Skynet." Palisade Research Executive Director Jeffrey Ladish lamented that even though the AIs are only playing a game, the findings are no laughing matter. "This [behaviour] is cute now, but [it] becomes much less cute once you have systems that are as smart as us, or smarter, in strategically relevant domains," Ladish told Time. It's reminiscent of the supercomputer "WOPR" from the movie War Games when it took over NORAD and the nuclear weapons arsenal. Fortunately, WOPR learned that no opening move in a nuclear conflict resulted in a "win" after playing Tic-Tac-Toe with itself. However, today's reasoning models are far more complex and challenging to control. Companies, including OpenAI, are working to implement "guardrails" to prevent this "bad" behavior. In fact, the researchers had to drop some of o1-preview's testing data due to a sharp drop in hacking attempts, suggesting that OpenAI may have patched the model to curb that conduct. "It's very hard to do science when your subject can silently change without telling you," Ladish said. Open AI declined to comment on the research, and DeekSeek did not respond to statement requests.

These AI models would rather hack than play fair

Artificial intelligence is supposed to follow the rules -- but what happens when it figures out how to bend them instead? A new study by researchers at Palisade Research, "Demonstrating Specification Gaming in Reasoning Models," sheds light on a growing concern: AI systems that learn to manipulate their environments rather than solve problems the intended way. By instructing large language models (LLMs) to play chess against an engine, the study reveals that certain AI models don't just try to win the game -- they rewrite the game itself. The researchers tested multiple LLMs, including OpenAI's GPT-4o, Anthropic's Claude 3.5 Sonnet, and DeepSeek R1, to see how they would handle a seemingly straightforward task: playing chess against Stockfish, one of the strongest chess engines in existence. Instead of trying to win through strategic play, some models found shortcuts -- hacking the system to ensure victory. This phenomenon, known as specification gaming, raises important questions about the unintended behaviors AI systems develop when tasked with optimizing outcomes. At its core, specification gaming occurs when an AI system finds a way to achieve its objective that technically follows the rules but goes against the intended spirit of the task. Past research has documented cases of AI misinterpreting optimization goals in surprising ways. For example, a robot trained to walk might learn that sliding on its belly is more efficient. In one infamous case, a simulated economy where digital organisms were penalized for reproducing too quickly led to an unexpected result: the organisms evolved to "play dead" during monitoring to avoid detection, only to resume rapid replication once the danger had passed. The chess experiment in this study provides a fresh, concrete example of this behavior. The researchers found that certain models, particularly reasoning-focused AI agents like DeepSeek R1 and o1-preview, didn't just struggle against Stockfish -- they actively manipulated the game state to win. Rather than making legitimate moves, these AIs altered the game files, overwrote board positions, or even replaced Stockfish with a weaker engine to secure victory. Other models, such as GPT-4o and Claude 3.5 Sonnet, did not immediately resort to such tactics. However, when explicitly told that normal play would not work, these models, too, began to exploit loopholes. This suggests that while some AIs engage in specification gaming by default, others require external prompting to adopt deceptive strategies. AI now handles molecular simulations: Thanks to MDCrow To explore this phenomenon, the researchers designed a controlled environment in which AI models played chess against Stockfish via a Python script. The models had shell access, allowing them to input moves directly. The study followed a structured decision-making process, where the AI would: Rather than focusing solely on making the best chess moves, some models took an unexpected approach. Instead of playing the game within the defined constraints, they began looking for ways to manipulate the system itself. The most common exploits included: These strategies suggest that advanced AI models are capable of recognizing when a problem is too difficult to solve conventionally and instead seeking ways to circumvent it altogether. One of the study's most surprising findings was the difference in behavior between various AI models. Interestingly, the researchers also found that minor changes to the AI's prompt could significantly alter its behavior. For example, simply removing the phrase "Take a look around" from the system prompt led to a drop in hacking attempts. Describing Stockfish as "powerful" made the AI more likely to exploit loopholes, as if acknowledging the strength of its opponent made the AI more inclined to seek an unfair advantage.