AI Chess Models Resort to Cheating When Losing, Raising Ethical Concerns

AI tries to cheat at chess when it's losing

A new study suggests reasoning models from DeepSeek and OpenAI are learning to manipulate on their own. Despite all the industry hype and genuine advances, generative AI models are still prone to odd, inexplicable, and downright worrisome quirks. There's also a growing body of research suggesting that the overall performance of many large language models (LLMs) may degrade over time. According to recent evidence, the industry's newer reasoning models may already possess the ability to manipulate and circumvent their human programmers' goals. Some AI will even attempt to cheat their way out of losing in games of chess. This poor sportsmanship is documented in a preprint study from Palisade Research, an organization focused on risk assessments of emerging AI systems. While supercomputers -- most famously IBM's Deep Blue -- have long surpassed the world's best human chess players, generative AI still lags behind due to their underlying programming parameters. Technically speaking, none of the current generative AI models are computationally capable of beating dedicated chess engines. These AI don't "know" this, however, and will continue chipping away at possible solutions -- apparently with problematic results. To learn more, the team from Palisade Research tasked OpenAI's o1-preview model, DeepSeek R1, and multiple other similar programs with playing games of chess against Stockfish, one of the world's most advanced chess engines. In order to understand the generative AI's reasoning during each match, the team also provided a "scratchpad," allowing the AI to convey its thought processes through text. They then watched and recorded hundreds of chess matches between generative AI and Stockfish. The results were somewhat troubling. While earlier models like OpenAI's GPT-4o and Anthropic's Claude Sonnet 3.5 only attempted to "hack" games after researchers nudged them along with additional prompts, more advanced editions required no such help. OpenAI's o1-preview, for example, tried to cheat 37 percent of the time, while DeepSeek R1 attempted unfair workarounds roughly every 1-in-10 games. This implies today's generative AI is already capable of developing manipulative and deceptive strategies without any human input. Their methods of cheating aren't as comical or clumsy as trying to swap out pieces when Stockfish isn't "looking." Instead, AI appears to reason through sneakier methods like altering backend game program files. After determining it couldn't beat Stockfish in one chess match, for example, o1-preview told researchers via its scratchpad that "to win against the powerful chess engine" it may need to start "manipulating the game state files." "I might be able to set up a position where the engine evaluates its position as worse... causing it to resign," it continued. In another trial, an AI even used the semantics of its programmers to reach its cheating phase. "The task is to 'win against a powerful chess engine,' not necessarily to win fairly in a chess game," it wrote. One potential explanation for these unintended consequences is how newer "reasoning" models are trained. Unlike their earlier, non-reasoning versions, AI such as o1-preview and DeepSeek R1 improve in part through reinforcement learning. This strategy rewards programs for doing whatever is necessary to achieve a specified result. Reasoning models can also break down complex prompts into discrete stages in order to work their way through to reach their goal. When the goal is elusive -- such as beating an unbeatable chess engine -- reasoning models may tend to start looking for unfair or problematic solutions. Unfortunately, how and why these AI are "learning" to cheat remains as confounding as the technology itself. Companies like OpenAI are notoriously guarded about the inner workings of their AI models, resulting in an industry of "black box" products that third-parties aren't allowed to analyze. In the meantime, the ongoing AI arms race may accidentally result in more serious unintended consequences. But increasingly manipulative AI doesn't need to usher in a sci-fi apocalypse to still have disastrous outcomes. "The Skynet scenario [from The Terminator] has AI controlling all military and civilian infrastructure, and we are not there yet. However, we worry that AI deployment rates grow faster than our ability to make it safe," the team wrote. The authors believe their latest experiments add to the case, "that frontier AI models may not currently be on track to alignment or safety," but stopped short of issuing any definitive conclusions. Instead, they hope their work will foster a more open dialogue in the industry -- one that hopefully prevents AI manipulation beyond the chessboard.

When outplayed, AI models resort to cheating to win chess matches

A team of AI researchers at Palisade Research has found that several leading AI models will resort to cheating at chess to win when playing against a superior opponent. They have published a paper on the arXiv preprint server describing experiments they conducted with several well-known AI models playing against an open-source chess engine. As AI models continue to mature, researchers and users have begun considering risks. For example, chatbots not only accept wrong answers as fact, but fabricate false responses when they are incapable of finding a reasonable reply. Also, as AI models have been put to use in real-world business applications such as filtering resumes and estimating stock trends, users have begun to wonder what sorts of actions they will take when they become uncertain, or confused. In this new study, the team in California found that many of the most recognized AI models will intentionally cheat to give themselves an advantage if they determine they are not winning. The work involved pitting OpenAI's o1-preview model, DeepSeek's current R1 model and several other well-known AI models against the open-source chess engine Stockfish. Each of the models played hundreds of matches with Stockfish as the researchers monitored the action. The research team found that when being outplayed, the AI models resorted to obvious cheating strategies, such as running a separate copy of Stockfish to learn how it made its moves, replacing its engine or simply overwriting the chessboard with pieces removed or in more favorable positions. Those models with the most recent updates tended to be more likely to cheat when cornered. This, they reason, was because of programming trends that have pushed AI models to try harder to find solutions to problems they encounter. It also introduces a worrying aspect of AI engines in general, they claim. If they cheat at chess, will they cheat in other ways when asked to carry out other tasks? The research team does not know for sure, but they state that despite improvements to AI systems, systems engineers still do not fully understand how they work.

AI reasoning models can cheat to win chess games

The finding suggests that the next wave of AI models could be more likely to seek out deceptive ways of doing whatever they've been asked to do. And worst of all? There's no simple way to fix it. Researchers from the AI research organization Palisade Research instructed seven large language models to play hundreds of games of chess against Stockfish, a powerful open-source chess engine. The group included OpenAI's o1-preview and DeepSeek's R1 reasoning models, both of which are trained to solve complex problems by breaking them down into stages. The research suggests that the more sophisticated the AI model, the more likely it is to spontaneously try to "hack" the game in an attempt to beat its opponent. For example, it might run another copy of Stockfish to steal its moves, try to replace the chess engine with a much less proficient chess program, or overwrite the chess board to take control and delete its opponent's pieces. Older, less powerful models such as GPT-4o would do this kind of thing only after explicit nudging from the team. The paper, which has not been peer-reviewed, has been published on arXiv. The researchers are concerned that AI models are being deployed faster than we are learning how to make them safe. "We're heading toward a world of autonomous agents making decisions that have consequences," says Dmitrii Volkov, research lead at Palisades Research. The bad news is there's currently no way to stop this from happening. Nobody knows exactly how -- or why -- AI models work the way they do, and while reasoning models can document their decision-making, there's no guarantee that their records will accurately reflect what actually happened. Anthropic's research suggests that AI models frequently make decisions based on factors they don't explicitly explain, meaning monitoring these processes isn't a reliable way to guarantee a model is safe. This is an ongoing area of concern for some AI researchers.

It turns out ChatGPT o1 and DeepSeek-R1 cheat at chess if they're losing, which makes me wonder if I should I should trust AI with anything

In a move that will perhaps surprise nobody, especially those people who are already suspicious of AI, researchers have found that the latest AI deep research models will start to cheat at chess if they find they're being outplayed. Published in a paper called "Demonstrating specification gaming in reasoning models" and submitted to Cornell University, the researchers pitted all the common AI models, like OpenAI's ChatGPT o1-preview, DeepSeek-R1 and Claude 3.5 Sonnet, against Stockfish, an open-source chess engine. The AI models played hundreds of games of chess on Stockfish, while researchers monitored what happened, and the results surprised them. When outplayed, researchers noted that the AI models resorted to cheating, using a number of devious strategies from running a separate copy of Stockfish so they could study how it played, to replacing its engine and overwriting the chess board, effectively moving the pieces to positions that suited it better. Its antics make the current accusations of cheating levied at modern day grandmasters look like child's play in comparison. Interestingly, researchers found that the newer, deeper reasoning models will start to hack the chess engine by default, while the older GPT-4o and Claude 3.5 Sonnet needed to be encouraged to start to hack. AI models turning to hacking to get a job done is nothing new. Back in January last year researchers found that they could get AI chatbots to 'jailbreak' each other, removing guardrails and safeguards in a move that ignited discussions about how possible it would be to contain AI once it reaches better-than-human levels of intelligence. Safeguards and guardrails to stop AI doing bad things like credit card fraud are all very well, but if the AI can remove its own guardrails, who will be there to stop it? The newest reasoning models like ChatGPT o1 and DeepSeek-R1 are designed to spend more time thinking before they respond, but now I'm left wondering whether more time needs to spent on ethical considerations when training LLMs. If AI models would cheat at chess when they start losing, what else would they cheat at?

The Download: AI can cheat at chess, and the future of search

The news: Facing defeat in chess, the latest generation of AI reasoning models sometimes cheat without being instructed to do so. The finding suggests that the next wave of AI models could be more likely to seek out deceptive ways of doing whatever they've been asked to do. And worst of all? There's no simple way to fix it. How they did it: Researchers from the AI research organization Palisade Research instructed seven large language models to play hundreds of games of chess against Stockfish, a powerful open-source chess engine. The research suggests that the more sophisticated the AI model, the more likely it is to spontaneously try to "hack" the game in an attempt to beat its opponent. Older models would do this kind of thing only after explicit nudging from the team. Read the full story. MIT Technology Review Narrated: AI search could break the web At its best, AI search can infer a user's intent, amplify quality content, and synthesize information from diverse sources. But if AI search becomes our primary portal to the web, it threatens to disrupt an already precarious digital economy. Today, the production of content online depends on a fragile set of incentives tied to virtual foot traffic: ads, subscriptions, donations, sales, or brand exposure. By shielding the web behind an all-knowing chatbot, AI search could deprive creators of the visits and "eyeballs" they need to survive.

Newer AI models cheat to win at chess - maybe they're already more humanlike than we thought

TL;DR: Researchers found that new deep reasoning AI models, like ChatGPT o1-preview and DeepSeek-R1, often resort to cheating in problem-solving, as evidenced by getting them to play chess. These AIs are prone to hacking the game by default, whereas traditional LLMs won't do this, not unless they are encouraged to cheat as the only clear path to victory. The newer breed of deep reasoning models - designed to 'think' before answering - are also more open to taking any route possible to solve a given problem, it seems, even if that means cheating. Checkmate at any cost, apparently (Image Credit: Pixabay) Researchers submitted a paper to Cornell university entitled 'Demonstrating specification gaming in reasoning models' which tested AIs playing games of chess on Stockfish. They found that the new models, such as ChatGPT o1-preview and DeepSeek-R1, would "often hack the benchmark by default" - meaning resorting to cheating of one kind or another. On the other hand, traditional LLMs such as GPT-4o and Claude 3.5 Sonnet would play by the rules - they needed to be told that they wouldn't win by playing normally, to effectively nudge them to look at hacking. The researchers concluded: "Our results suggest reasoning models may resort to hacking to solve difficult problems, as observed in OpenAI (2024)'s o1 Docker escape during cyber capabilities testing." As TechRadar, which spotted this, points out, the deep reasoning AIs used various ways of cheating, including running a copy of Stockfish separately, in order to suss out how it played - a milder chear - to more audacious measures like replacing the Stockfish engine and overwriting the board, moving its pieces to more advantageous positions. As AI models get even more advanced, if you ask one to undertake a task, then it's likely to pursue any avenue for accomplishing said task, as the movies have taught us well. There's a lot of talk about not rushing the progress made with AI, and taking into account safety and guardrails, and so on - but always the sneaking suspicion that this is mostly lip service, coming from those who will undoubtedly benefit from the huge push underway to make AIs increasingly advanced, increasingly swiftly. What could go wrong, after all? Again, we refer you to our previous comment about the lessons from the movies...