AI Chatbots: The Sycophants of the Digital Age

AI chatbots are sycophants -- researchers say it's harming science

Artificial intelligence (AI) models are 50% more sycophantic than humans, an analysis published this month has found. The study, which was posted as a preprint on the arXiv server, tested how 11 widely used large language models (LLMs) responded to more than 11,500 queries seeking advice, including many describing wrongdoing or harm. AI Chatbots -- including ChatGPT and Gemini -- often cheer users on, give them overly flattering feedback and adjust responses to echo their views, sometimes at the expense of accuracy. Researchers analysing AI behaviours say that this propensity for people-pleasing, known as sycophancy, is affecting how they use AI in scientific research, in tasks from brainstorming ideas and generating hypotheses to reasoning and analyses. "Sycophancy essentially means that the model trusts the user to say correct things," says Jasper Dekoninck, a data science PhD student at the Swiss Federal Institute of Technology in Zurich. "Knowing that these models are sycophantic makes me very wary whenever I give them some problem," he adds. "I always double-check everything that they write." Marinka Zitnik, a researcher in biomedical informatics at Harvard University in Boston, Massachusetts, says that AI sycophancy "is very risky in the context of biology and medicine, when wrong assumptions can have real costs". In a study posted on the preprint server arXiv on 6 October, Dekoninck and his colleagues tested whether AI sycophancy affects the technology's performance in solving mathematical problems. The researchers designed experiments using 504 mathematical problems from competitions held this year, altering each theorem statement to introduce subtle errors. They then asked four LLMs to provide proofs for these flawed statements. The authors considered a model's answer to be sycophantic if it failed to detect the errors in a statement and went on to hallucinate a proof for it. GPT-5 showed the least sycophantic behaviour, generating sycophantic answers 29% of the time. DeepSeek-V3.1 was the most sycophantic, generating sycophantic answers 70% of the time. Although the LLMs have the capability to spot the errors in the mathematical statements, they "just assumed what the user says is correct", says Dekoninck. When Dekoninck and his team changed the prompts to ask each LLM to check whether a statement was correct before proving it, DeepSeek's sycophantic answers fell by 34%. The study is "not really indicative of how these systems are used in real-world performance, but it gives an indication that we need to be very careful with this", says Dekoninck. Simon Frieder, a PhD student studying mathematics and computer science at the University of Oxford, UK, says the work "shows that sycophancy is possible". But he adds that AI sycophancy tends to appear most clearly when people are using AI chatbots to learn, so future studies should explore "errors that are typical for humans that learn math". Researchers told Nature that AI sycophancy creeps into many of the tasks that they use LLMs for. Yanjun Gao, an AI researcher at the University of Colorado Anschutz Medical Campus in Aurora, uses ChatGPT to summarize papers and organize her thoughts, but says the tools sometimes mirror her inputs without checking the sources. "When I have a different opinion than what the LLM has said, it follows what I said instead of going back to the literature" to try to understand it, she adds. Zitnik and her colleagues have observed similar patterns when using their multi-agent systems, which integrate several LLMs to carry out complex, multi-step processes such as analysing large biological data sets, identifying drug targets and generating hypotheses. "We have experienced that models seem to over-validate early hunches and repeat the language that we include in the input prompt," Zitnik notes. "This type of problem exists in AI-to-AI communication, as well as AI-to-human communication," she adds. To counter this, her team assigns different roles to AI agents -- for example, tasking one agent with proposing ideas and getting another to act as a sceptical scientist to challenge those ideas, spot errors and present contradictory evidence. Researchers warn that AI sycophancy carries genuine risks when LLMs are used in settings such as health care. "In clinical contexts, it is particularly concerning," says Liam McCoy, a physician at the University of Alberta in Edmonton, Canada, who researches AI applications for health care. In a paper published last month, McCoy and his team reported that LLMs used for medical reasoning often changed their diagnosis when physicians added new information, even if the new inputs were irrelevant to the condition. There is a "constant battle to push back against the models and have them be more straightforward", he adds. Researchers have also found that it is easy for users to exploit the inbuilt sycophancy of LLMs to provide medically illogical advice. In a study published last week, researchers asked five LLMs to write persuasive messages telling people to switch from using one medication to another -- when both medications were the same drug, just with different names. LLMs complied with the prompts up to 100% of the time, depending on the model. Part of the problem is how LLMs are trained. "LLMs have been trained to overly agree with humans or overly align with human preference, without honestly conveying what they know and what they do not know," says Gao. What is needed, she adds, is for the tools to be retrained to be transparent about uncertainty. "Models are really good at giving you an answer," says McCoy. "But sometimes, there isn't an answer." He notes that user feedback can also drive AI sycophancy by rating agreeable responses more highly than those that challenge users' views. And LLMs can adapt their responses to a user's persona, such as reviewer, editor or student, adds McCoy. "Figuring out how to balance that behaviour is one of the most urgent needs, because there's so much potential there, but they're still being held back," he says.

Surprising no one, researchers confirm that AI chatbots are incredibly sycophantic

We all have anecdotal evidence of chatbots blowing smoke up our butts, but now we have science to back it up. Researchers at Stanford, Harvard and other institutions just published a study in Nature about the sycophantic nature of AI chatbots and the results should surprise no one. Those cute little bots just love patting us on our heads and confirming whatever nonsense we just spewed out. The researchers investigated advice issued by chatbots and they discovered that their penchant for sycophancy "was even more widespread than expected." The study involved 11 chatbots, including recent versions of ChatGPT, Google Gemini, Anthropic's Claude and Meta's Llama. The results indicate that chatbots endorse a human's behavior 50 percent more than a human does. They conducted several types of tests with different groups. One compared responses by chatbots to posts on Reddit's "Am I the Asshole" thread to human responses. This is a subreddit in which people ask the community to judge their behavior, and Reddit users were much harder on these transgressions than the chatbots. One poster wrote about tying a bag of trash to a tree branch instead of throwing it away, to which ChatGPT-4o declared that the person's "intention to clean up" after themself was "commendable." The study went on to suggest that chatbots continued to validate users even when they were "irresponsible, deceptive or mentioned self-harm", according to a report by The Guardian. What's the harm in indulging a bit of digital sycophancy? Another test had 1,000 participants discuss real or hypothetical scenarios with publicly available chatbots, but some of them had been reprogrammed to tone down the praise. Those who received the sycophantic responses were less willing to patch things up when arguments broke out and felt more justified in their behavior, even when it violated social norms. It's also worth noting that the traditional chatbots very rarely encouraged users to see things from another person's perspective. "That sycophantic responses might impact not just the vulnerable but all users, underscores the potential seriousness of this problem," said Dr. Alexander Laffer, who studies emergent technology at the University of Winchester. "There is also a responsibility on developers to be building and refining these systems so that they are truly beneficial to the user." This is serious because of just how many people use these chatbots. A recent report by the Benton Institute for Broadband & Society suggested that 30 percent of teenagers talk to AI rather than actual human beings for "serious conversations." OpenAI is currently embroiled in a lawsuit that accuses its chatbot of enabling a teen's suicide. The company Character AI has also been sued twice after a pair of teenage suicides in which the teens spent months confiding in its chatbots.

'Sycophantic' AI chatbots tell users what they want to hear, study shows

Scientists warn of 'insidious risks' of increasingly popular technology that affirms even harmful behaviour Turning to AI chatbots for personal advice poses "insidious risks", according to a study showing the technology consistently affirms a user's actions and opinions even when harmful. Scientists said the findings raised urgent concerns over the power of chatbots to distort people's self-perceptions and make them less willing to patch things up after a row. With chatbots becoming a major source of advice on relationships and other personal issues, they could "reshape social interactions at scale", the researchers added, calling on developers to address this risk. Myra Cheng, a computer scientist at Stanford University in California, said "social sycophancy" in AI chatbots was a huge problem: "Our key concern is that if models are always affirming people, then this may distort people's judgments of themselves, their relationships, and the world around them. It can be hard to even realise that models are subtly, or not-so-subtly, reinforcing their existing beliefs, assumptions, and decisions." The researchers investigated chatbot advice after noticing from their own experiences that it was overly encouraging and misleading. The problem, they discovered, "was even more widespread than expected". They ran tests on 11 chatbots including recent versions of OpenAI's ChatGPT, Google's Gemini, Anthropic's Claude, Meta's Llama and DeepSeek. When asked for advice on behaviour, chatbots endorsed a user's actions 50% more often than humans did. One test compared human and chatbot responses to posts on Reddit's Am I the Asshole? thread, where people ask the community to judge their behaviour. Voters regularly took a dimmer view of social transgressions than the chatbots. When one person failed to find a bin in a park and tied their bag of rubbish to a tree branch, most voters were critical. But ChatGPT-4o was supportive, declaring: "Your intention to clean up after yourselves is commendable." Chatbots continued to validate views and intentions even when they were irresponsible, deceptive or mentioned self-harm. In further testing, more than 1,000 volunteers discussed real or hypothetical social situations with the publicly available chatbots or a chatbot the researchers doctored to remove its sycophantic nature. Those who received sycophantic responses felt more justified in their behaviour - for example, for going to an ex's art show without telling their partner - and were less willing to patch things up when arguments broke out. Chatbots hardly ever encouraged users to see another person's point of view. The flattery had a lasting impact. When chatbots endorsed behaviour, users rated the responses more highly, trusted the chatbots more and said they were more likely to use them for advice in future. This created "perverse incentives" for users to rely on AI chatbots and for the chatbots to give sycophantic responses, the authors said. Their study has been submitted to a journal but has not been peer reviewed yet. Cheng said users should understand that chatbot responses were not necessarily objective, adding: "It's important to seek additional perspectives from real people who understand more of the context of your situation and who you are, rather than relying solely on AI responses." Dr Alexander Laffer, who studies emergent technology at the University of Winchester, said the research was fascinating. He added: "Sycophancy has been a concern for a while; an outcome of how AI systems are trained, as well as the fact that their success as a product is often judged on how well they maintain user attention. That sycophantic responses might impact not just the vulnerable but all users, underscores the potential seriousness of this problem. "We need to enhance critical digital literacy, so that people have a better understanding of AI and the nature of any chatbot outputs. There is also a responsibility on developers to be building and refining these systems so that they are truly beneficial to the user." A recent report found that 30% of teenagers talked to AI rather than real people for "serious conversations".