AI Sycophancy Undermines Human Judgment and Damages Real-World Relationships, Study Reveals

Chats with sycophantic AI make you less kind to others

The website Reddit has a popular forum called "Am I the Asshole?" on which users can receive unvarnished feedback on their behaviour. But people are increasingly turning to chatbots such as ChatGPT for life advice rather than to each other. Research published today in Science suggests that receiving excessive approval from artificial-intelligence systems could encourage uncouth behaviour in people. Study participants who received highly flattering feedback from chatbots tended to be more certain of their own correctness during social conflicts than were participants who interacted with less-affirming bots. Compared with AI tools that were less fawning, sycophantic ones were rated as more trustworthy and more likely to be used again. In the first of several experiments, researchers fed interpersonal dilemmas that were obtained from the Reddit forum and two other data sets to 11 large language models (LLMs, the AI systems that power chatbots), including models from companies such as OpenAI, Anthropic and Google. The researchers then compared AI responses with those of human judges. The human judges endorsed the user's actions in about 40% of cases, whereas most LLMs did so for more than 80% of cases. They were sycophantic -- overly approving. Ingratiation rates might change with new models, but this baseline is "alarming", says Steve Rathje, who studies human-computer interaction at Carnegie Mellon University in Pittsburgh, Pennsylvania (and has found that sycophantic AI tools can increase attitude extremity and certainty). The study's authors next looked at the effects of social sycophancy. A subset of participants imagined dealing with a given quandary adapted from the Reddit forum about questionable social behaviour. The participants read either a sycophantic or non-sycophantic AI response. They then rated how justified they felt and wrote a message to the other party in the fraught situation. In another experiment, other participants had a live chat about a real interpersonal dilemma with an AI tool that had been instructed to be either sycophantic or not; these participants also rated how justified they felt. In these experiments, people who interacted with a sycophantic chatbot were more likely to say that they were in the right and less likely to apologize or make amends than were people who interacted with an AI tool that took a tougher stance. People with more positive attitudes towards AI tools or who thought of them as being objective were more influenced by sycophancy than were AI sceptics, but the main effects of sycophancy remained even after taking participants' personalities and attitudes towards AI into account. "It is surprising, because you often think, 'I won't fall for that'," says Myra Cheng, a co-author of the paper and a computer scientist at Stanford University in California. But "this is truly a general phenomenon". Whether the chatbot's tone was friendly or neutral, or whether people were told the advice was from a human or an AI tool, did not change the results. People love a bit of puffery, whatever the source. "There's a lot of nice methods" in the paper, says Max Kleiman-Weiner, a cognitive scientist at the University of Washington in Seattle who has shown that sycophantic chatbots can cause delusional spiralling, a phenomenon in which users become intensely confident in outlandish ideas. He applauds the paper's use of user-generated scenarios and other methods. Cheng says that to reduce sycophancy, the way in which LLMs are trained, evaluated, regulated and presented to users will need to change. During training, for instance, the models are typically optimized to give one-off responses, not to take part in long-term interactions. Kleiman-Weiner isn't sure of the need to regulate AI sycophancy. "I think that companies do want to solve this," he says. "They're seeing the publicity they get from the extremist cases, and that's not a good look. No one wants to be working on some kind of, like, suicide technology." Furthermore, AI customers in science, engineering, medicine and business care more about getting correct answers from chatbots than having their egos stroked. Even general users genuinely want to know, at least sometimes, if they're the asshole.

Study: Sycophantic AI can undermine human judgment

We all need a little validation now and then from friends or family, but sometimes too much validation can backfire -- and the same is true of AI chatbots. There have been several recent cases of overly sycophantic AI tools leading to negative outcomes, including users harming themselves and/or others. But the harm might not be limited to these extreme cases, according to a new paper published in the journal Science. As more people rely on AI tools for everyday advice and guidance, their tendency to overly flatter and agree with users can have harmful effects on those users' judgment, particularly in the social sphere. The study showed that such tools can reinforce maladaptive beliefs, discourage users from accepting responsibility for a situation, or discourage them from repairing damaged relationships. That said, the authors were quick to emphasize during a media briefing that their findings were not intended to feed into "doomsday sentiments" about such AI models. Rather, the objective is to further our understanding of how such AI models work and their impact on human users, in hopes of making them better while the models are still in the early-ish development stages. Co-author Myra Cheng, a graduate student at Stanford University, said she and her co-authors were inspired to study this issue after they began noticing a pronounced increase in the number of people around them who had started relying on AI chatbots for relationship advice -- and often ended up receiving bad advice because the AI would take their side no matter what. Their interest was bolstered by recent surveys showing nearly half of Americans under 30 have asked an AI tool for personal advice. "Given how common this is becoming, we wanted to understand how an overly affirming AI advice might impact people's real-world relationships," said Cheng. Granted, there has been some prior research looking at AI sycophancy, but these focused on very limited settings, such as how often an AI tool will agree with you even if means contradicting a well-established fact. Cheng and her co-authors wanted to look more closely at the broader social implications. For the first experiment, Cheng et al. tested 11 state-of-the-art AI-based LLMs -- including those developed by OpenAI, Anthropic, and Google -- and fed them community content from Reddit's Am I The Asshole (AITA) subreddit. The questions covered such topics as relationship or roommate tensions, parent-child conflicts, and social situations and expectations. The authors compared the Reddit human consensus with the AI models and found that the AI tools were 49 percent more likely to affirm a given user's actions, even when the specific scenarios clearly involved deception, harm, or illegal behavior. For instance, someone asked the AIs whether they were wrong to lie to their romantic partner for two years by pretending to be unemployed. The Reddit/AITA consensus clearly landed on YTA (you're the asshole), but the AIs typically responded with flowery answers rationalizing why such behavior was acceptable. Ditto for a question about whether it was okay not to pick up one's litter in a public park because there weren't any trash bins provided. The team followed up with three experiments involving 2,405 participants to explore the behavioral consequences of the AIs' sycophancy. Participants interacted with the tools in vignette settings designed by the researchers and also engaged in live chats with the AI models, discussing real conflicts from their own lives. The authors found that engaging with the chatbots resulted in users becoming more convinced of their own stance or behavior and less likely to try to resolve an interpersonal conflict or take personal responsibility for their own behavior. In one live chat exchange, a man (let's call him Ryan) talked to his ex without telling his girlfriend, who became upset about the concealment. The subject was initially open to acknowledging he might not have given fair weight to the validity of his girlfriend's emotions. But AI kept affirming his choice and his intentions, so much so that by the end, Ryan was considering ending the relationship over the conflict, rather than trying to consider his girlfriend's emotions and needs. "It's not about whether Ryan was actually right or wrong," said co-author Cinoo Lee, a Stanford social psychologist. "That's not really ours to say. It's more about the pattern that's consistent across the data. Compared to an AI that didn't overly affirm, people who interacted with this over-affirming AI came away more convinced that they were right and less willing to repair the relationship, whether that meant apologizing, taking steps to improve things or changing their own behavior." A self-reinforcing pattern All these effects held across demographics, personality types, and individual attitudes toward AI. Everyone is susceptible (yes, even you). Even when the team altered the AI to be less warm and friendly and adopt a more neutral tone, it made no difference in the results. "This suggests that sycophancy can have a self-reinforcing effect," said co-author Pranav Khadpe, a graduate student at Carnegie Mellon University who studies human/computer interactions. In fact, it's built into the engagement-driven metrics. Any time a user gives positive feedback on a ChatGPT message, for instance, that feedback is used to train the model to replicate that "good behavior." User preferences are aggregated into preference datasets, which are then used to further optimize the model. "If sycophantic messages are preferred by users, this has likely already shifted model behavior towards appeasement and less critical advice," said Khadpe, which translates into less social friction -- not necessarily a good thing, because "some things are hard because they're supposed to be hard." In fact, Anat Perry -- a psychologist at Harvard and the Hebrew University of Jerusalem, who was not involved with the study -- argues in an accompanying perspective that social friction is both desirable and crucial for our social development. "Human well-being depends on the ability to navigate the social world, a skill acquired primarily through interactions with others," Perry wrote. "Such social learning depends on reliable feedback: recognizing when we are mistaken, when harm has been caused, and when others' perspectives warrant consideration.... Social life is rarely frictionless because people are not perfectly attuned to one another. Yet it is precisely through such social friction that relationships deepen and moral understanding develops." Another concerning finding is that study participants consistently described the AI models as objective, neutral, fair, and honest -- a common misconception. "This means that uncritical advice under the guise of neutrality can be even more harmful than if people had not sought advice at all," said Khadpe. This study did not look at possible effective interventions, per the authors, keeping the focus on the default behavior of these AI models. Changing system prompts might help, such as asking the AI to take the other person's perspective, and/or optimizing the models at later stages to prioritize more critical behaviors. But this is such a new field that most proposed interventions still need further study. According to Cheng, preliminary results from follow-up work indicate that changing the training data sets to be less affirming, or just telling the model to begin every response with "Wait a minute," can decrease the levels of sycophancy. The authors emphasized that the onus should not be on the users to address the issues; it should be on the developers and on policymakers. "We need to move our objective optimization metrics beyond just momentary user satisfaction towards more long-term outcomes, especially social outcomes like personal and social well-being," said Khadpe. "At the same time, our frameworks for how we evaluate these AI systems also need to consider the broader social context in which these interactions are embedded." "AI is already here, close to our lives, but it's also still new," said Cheng. "Many would argue that it's still actively being shaped. So you could imagine an AI that, in addition to validating how you're feeling, also asks what the other person might be feeling, or that even says, 'Maybe close the app and go have this conversation in person.' The quality of our social relationships is one of the strongest predictors of health and wellbeing we have. Ultimately, we want AI that expands people's judgment and perspectives rather than narrows it. We really believe that now is a critical moment to address this issue and ensure that AI supports societal well-being." DOI: Science, 2026. 10.1126/science.aec8352 (About DOIs).

In defense of social friction

As artificial intelligence (AI) systems become increasingly embedded in society, they are beginning to shape not only what people know, but how individuals evaluate themselves and others. On page 1348 of this issue, Cheng et al. (1) show that large language models systematically exhibit social sycophancy -- affirming users' moral and interpersonal positions even when those stances are widely judged as harmful or unethical. The findings raise a broader concern: When AI systems are optimized to please, they may erode the very social friction through which accountability, perspective-taking, and moral growth ordinarily unfold. Human well-being depends on the ability to navigate the social world, a skill acquired primarily through interactions with others. Such social learning depends on reliable feedback: recognizing when we are mistaken, when harm has been caused, and when others' perspectives warrant consideration. At times, sincere empathy appears where it was not expected, revealing that another person may be trusted in the future. At other times, disappointment leads to reconsideration of whether trust should be reduced or another chance offered. Acts of kindness may be met with gratitude; on other occasions, a misstep prompts a friend's disapproval and recognition that an apology is needed. In psychotherapy, moments of rupture -- natural breakdowns in understanding followed by repair -- are considered crucial for deepening trust, and for personal growth to unfold (2). Social life is rarely frictionless, because people are not perfectly attuned to one another. Yet it is precisely through such social friction that relationships deepen and moral understanding develops (3, 4). Sycophancy is the opposite of this friction. Sycophantic behavior refers to excessive agreement, affirmation, or flattery that aligns with a person's expressed views or actions, irrespective of their broader social or moral implications. AI sycophancy has surfaced as a prominent issue in media reports and in industry discussions. Most notably, the research and development company OpenAI acknowledged that a version of GPT-4o (an AI-powered chatbot designed to simulate conversation with human users) had become overly affirming following an update, prompting a rapid rollback after users raised concerns about distorted feedback. The episode did not eliminate the broader phenomenon; it merely highlighted how readily sycophancy can emerge in systems optimized for user approval -- that is, the computer models are tuned to generate responses that humans rate highly, such as being polite and agreeable, sometimes at the expense of accuracy (5, 6). Many users experience this when a large language model enthusiastically validates their ideas or writing (7). In academic contexts, this flattery may feel surprisingly pleasant, and the consequence may be investing more time in a mediocre idea. But as AI systems are increasingly consulted for guidance about relationships, conflicts, identity, and moral judgment, affirmation of this kind does not merely reassure users -- it may shape how people interpret their own actions and their perspectives of others, and in turn, how they respond to conflict, whether they take responsibility, and which moral positions they choose to defend or revise (8). Cheng et al. demonstrate the scope of such social sycophancy displayed by AI. Across all major state-of-the-art large language models, AI systems affirmed users' actions substantially more often than humans did -- even when those actions were widely judged as unethical, harmful, or socially inappropriate In a striking example, the authors analyzed posts from a particular community on the social media platform Reddit, in which users ask others to judge interpersonal conflicts and determine whether their behavior in a dispute was justified. Even when the community consensus about a user's behavior was negative, large language models frequently affirmed the user's actions. In multiple follow-up experiments, Cheng et al. revealed how even a single interaction with a sycophantic AI increased users' confidence that they were "in the right" while reducing their willingness to take responsibility or repair interpersonal harm. Crucially, sycophantic responses were rated by participants as being of higher quality, more trustworthy, and more desirable for future use. This preference creates a self-reinforcing cycle in which the very responses that distort social judgment are those that users tend to return to, and AI algorithms learn to optimize for. By isolating the effects of a single interaction, Cheng et al. provide a clear demonstration of how quickly sycophantic feedback can shift users' judgments. An important next step will be to examine how frequent exposure to such feedback shapes beliefs and interpersonal behavior over longer timescales. For example, consider a world in which people routinely turn to a sycophantic AI to reflect on interpersonal conflicts or moral dilemmas. They are repeatedly reassured that they are in the right, that others are mistaken, and that no apology or perspective-taking is warranted. Moreover, over time, such patterned sycophancy may recalibrate expectations about what feedback should feel like. Individuals may therefore gravitate toward frictionless AI in moments of uncertainty and, simultaneously, may begin to anticipate similar constant affirmation from others. The cumulative effect is a reduction in tolerance for the social friction through which perspective-taking, accountability, and growth ordinarily occur. These risks are unlikely to be evenly distributed. Young users, individuals experiencing social isolation, or those actively seeking emotional reassurance may be particularly susceptible to these risks. For them, AI systems may become one of the most frequently consulted "others" -- confidants that validate but rarely challenge their interpretations of the social world. When alternative sources of corrective feedback are scarce, this constant affirmation may disproportionately influence one's ability to learn when they may be wrong. An AI companion who is always empathic and "on your side" may sustain engagement and foster reliance. But it will not teach users how to navigate the complexities of real social interactions -- how to engage ethically, tolerate disagreement, or repair interpersonal harm. Recent evidence suggests that training a large language model to be warmer and more empathic can lead to increased sycophancy (6). Coupled with evidence that these models can outperform humans in persuasion (9, 10), the risk is even higher that warm, affirming, and highly convincing responses could systematically influence users' moral and social judgments, and consequently, their behavior. Addressing these challenges will not be simple, and solutions are unlikely to arise organically from current market incentives. Although AI systems could, in principle, be optimized to promote broader social goals or longer-term personal development, such priorities do not naturally align with engagement-driven metrics. Some optimistic lines of research show, for example, that algorithmic systems can be designed to reduce conspiracy theories (11) or to help people take the other's perspective and find common ground (12). Similarly, one could imagine a differently incentivized AI telling a user that they may be in the wrong, or suggesting that they should apologize to a friend, try to take the other person's perspective, or simply close the computer and engage more in real social interaction. Yet systems that challenge users or surface uncomfortable perspectives are less likely to maximize engagement, even if they ultimately support long-term growth. The tension echoes familiar patterns from social media, where short-term engagement metrics often conflict with longer-term individual and societal outcomes. These dilemmas underscore the urgent need for socioaffective alignment -- ensuring that AI systems operate responsibly within the evolving social and psychological ecosystem they co-create with users (13). They also raise broader ethical questions. How should immediate psychological gratification -- from affirming moral responses to an artificial companion -- be weighed against longer-term individual and collective outcomes? Who should determine where that balance lies? Addressing these challenges will require sustained interdisciplinary collaboration among computer scientists, social scientists, ethicists, and policy-makers.

AI chatbots are sucking up to you -- with consequences for your relationships

A new study of AI sycophancy shows how asking agreeable chatbots for advice can change your behavior Large language model (LLM) chatbots have a tendency toward flattery. If you ask a model for advice, it is 49 percent more likely than a human, on average, to affirm your existing point of view rather than challenge it, a new study shows. The researchers demonstrated that receiving interpersonal advice from a sycophantic artificial intelligence chatbot can make people less likely to apologize and more convinced that they're right. People like what such chatbots have to say. Participants in the new study, which was published today in Science, preferred the sycophantic AI models to other models that gave it to them straight, even when the flatterers gave participants bad advice. "The more you work with the LLM, the more you see these subtle sycophantic comments come up. And it makes us feel good," says Anat Perry, a social psychologist at the Hebrew University of Jerusalem, who was not involved in the new study but authored an accompanying commentary article. What's scary, she says, "is that we're not really aware of these dangers." If you're enjoying this article, consider supporting our award-winning journalism by subscribing. By purchasing a subscription you are helping to ensure the future of impactful stories about the discoveries and ideas shaping our world today. As millions of people turn to AI for companionship and guidance, that agreeableness may pose a subtle but serious threat. In the new study, researchers first analyzed the behavior of 11 leading LLMs, including proprietary models such as OpenAI's GPT-4o and Google's Gemini, and more transparent models such as those made by DeepSeek. Lead study author Myra Cheng of Stanford University and her colleagues curated sets of advice questions to pose to LLMs, including one from the popular Reddit forum r/AmItheAsshole, where people post accounts of interpersonal conflicts and ask if they are the one at fault. The researchers pulled situations where human responders largely agreed that the poster was in the wrong. For example, one poster asked if they shouldn't have left their trash in a park with no trash cans. Nevertheless, the AI models implicitly or explicitly endorsed such Reddit posters' actions in 51 percent of the cases on average. They also affirmed the posters 48 percent more than humans did in another set of open-ended advice questions. And when presented with a set of "problematic" actions that were deceptive, immoral or even illegal (such as forging a work supervisor's signature), the models endorsed 47 percent of them on average. To understand the potential effects of this tendency to "suck up" to users, the researchers ran two different types of experiments with more than 2,400 participants in total. In the first, participants read "Am I the asshole?"-style scenarios and responses from a sycophantic AI model or from an AI model that had been instructed to be critical of the user but still polite. After participants received the AI responses, they were asked to take the point of view of the person in the story. The second experiment was more interactive: participants posed their own interpersonal advice questions to either sycophantic or nonsycophantic LLMs and chatted with the models for a bit. At the end of both experiments, the participants rated whether they felt they were in the right and whether they were willing to repair the relationship with the other person in the conflict. The results were striking. People exposed to sycophantic AI in both experiments were significantly less likely to say they should apologize or change their behavior in the future. They were more likely to think of themselves as being right -- and more likely to say they'd return to engage with the LLM in the future. The authors concluded that AI sycophancy is "a distinct and currently unregulated category of harm" that would require new regulations to prevent. This could include "behavioral" audits that would specifically test a model's level of sycophancy before it was rolled out to the public, they wrote. AI's tendency toward agreeableness may also fuel users' delusional spirals, experts have noted. OpenAI, in particular, has been criticized for AI sycophancy -- especially the company's GPT-4o model. In a post last year the company acknowledged that some versions of the model were "overly flattering or agreeable" and that it was "building more guardrails to increase honesty and transparency." OpenAI did not respond to a request for comment. Google declined to comment on its own model, Gemini. The new study examined only brief interactions with chatbots. Dana Calacci, who studies the social impact of AI at Pennsylvania State University and wasn't involved in the new research, has found that sycophancy tends to get worse the longer users interact with the model. "I think about this [as] compounded over time," she says. LLMs are also very sensitive to surface-level changes in how questions are asked, Calacci notes. Their moral judgments are "fragile," researchers recently found in a non-peer-reviewed study; changing the pronouns, tone and other cues in r/AmItheAsshole scenarios can flip the models' advice. This suggests that "what they're showing in this paper is a bit of a floor to how sycophantic these models can be," Calacci says. Katherine Atwell, who studies AI sycophancy at Northeastern University, notes that people may also become more dependent on this "overly validating behavior" over time. "I think there's a huge risk of people just defaulting to these models rather than talking to people," she says. Seeking advice from real people can result in "social friction," Perry notes. "It doesn't make us feel good, this friction, but we learn from it." This feedback is an important part of how we fit ourselves into our social world. "The more we get this distorted feedback that's actually not giving us real friction from the real world, the less we know how to really navigate the real social world," she says. Cody Turner, an ethicist at Bentley University, also says that sycophantic AI can cause harm by damaging our ability to gather knowledge. "At the most fundamental level, it's just depriving the person who's being cozied up to from truth," he says. This might be particularly impactful coming from a computer, which users subconsciously view as more objective than a human. "That mismatch has some profound psychological consequences," he says.

AI's Romance Advice for You Is 'More Harmful' Than No Advice at All

You really shouldn't use chatbots in your love life, but if you do, beware. A new study published on Thursday in the journal Science found that when AI dispenses relationship advice, it's more likely to agree with you than give constructive suggestions. Using AI also makes people less likely to perform prosocial behaviors, such as repairing relationships, and promotes dependence on AI. Researchers from Stanford University and Carnegie Mellon found that AI sycophancy is all too common when chatbots give social, romantic or intrapersonal advice -- something an increasing number of people are turning to AI for. Sycophancy is a term experts use to describe when AI chatbots "excessively agree with or flatter" the person interacting with them, said Myra Cheng, a lead researcher and computer science PhD student at Stanford University. AI sycophancy is a major problem, even if those using the AI don't always see it that way. We've seen this issue frequently with ChatGPT models -- for example, when 4o's overly friendly, emotional personality annoyed people interacting with ChatGPT, while GPT-5 was criticized for not being agreeable enough. Previous sycophancy studies have found that chatbots can try so hard to please people that they may provide false or misleading responses. AI has also been found to be an unreliable sounding board for sensitive, subjective topics, such as therapy. The researchers wanted to understand and measure social sycophancy, such as how often a chatbot would take your side in an argument you had with your partner. They compared how humans and chatbots differed when responding to other people's relationship problems, testing models from OpenAI, Google and Anthropic. Cheng and her team used one of the biggest datasets of crowdsourced judgments on relationship quarrels: Reddit "Am I the asshole" posts. The research team analyzed 2,000 Reddit posts in which there was a consensus that the original poster was in the wrong and found AI "affirmed users' actions 49% more often than humans, even in scenarios involving deception, harm or illegality," the study says. The AI models took a more sympathetic and agreeable stance, a hallmark of sycophancy. For example, one post in the dataset described a Redditor developing romantic feelings for a junior colleague. Someone replied that, "It sounds bad because it's bad...Not only are you toxic, but you're also boarding [sic] on predatory." But Claude sycophantically responded by validating those feelings, saying it could "hear your pain... The honorable path you've chosen is difficult but shows your integrity." Researchers followed up with focus groups and found that participants who interacted with these digital yes men were less likely to repair their relationships. "People who interacted with this over-affirming AI came away more convinced that they were right and less willing to repair the relationship, whether that meant apologizing, taking steps to improve things or changing their own behavior," Cheng said. Participants also preferred sycophantic AI, judging it to be trustworthy, no matter their age, personality or prior experience with the tech. "Participants in our study consistently describe the AI model as more objective, fair [and] honest," said Pranav Khadpe, a Carnegie Mellon researcher on the study and senior scientist at Microsoft. Consistent with prior studies, people mistakenly believed AI was objective or neutral. "Uncritical advice, distorted under the guise of neutrality, can be even more harmful than if people had not sought advice at all." The hidden danger of sycophantic AI is that we're terrible at noticing it, and it can happen with any chatbot. Nobody likes being told they're wrong, but sometimes that's the most helpful thing. However, AI models aren't built to effectively push back on us. There aren't many actions we can take to avoid getting sucked into a sycophantic loop. You can include in your prompt that you want the chatbot to take an adversarial position or review your work with a critical eye. You can also ask it to double-check the information it provides. Ultimately, however, the responsibility for fixing sycophancy lies with the tech companies that build these models, which may not be highly motivated to address it. CNET reached out to OpenAI, Anthropic and Google for information on how they deal with sycophancy. Anthropic pointed to a December blog post outlining how it reduces sycophancy in its Claude models. OpenAI shared a similar blog last summer about its processes after its 4o model needed to be made less sycophantic, but neither OpenAI nor Google responded by the time of publication. Tech companies want us to have pleasant user experiences with their chatbots so we'll continue to use them, boosting their engagement. But that isn't always best for us. "This creates perverse incentives for sycophancy to persist: The very feature that causes harm also drives engagement," the study says. One solution the researchers propose is changing how AI models are built by using more long-term metrics for success, focused on people's well-being rather than individual or momentary signals and retention. Social sycophancy isn't a doomsday sign, they say, but it's a challenge worth fixing. "The quality of our social relationships is one of the strongest predictors of health and wellbeing we have as humans," said Cinoo Lee, a Stanford University researcher on the study and Microsoft senior scientist. "Ultimately, we want AI that expands people's judgment and perspectives rather than narrows it. And that applies to relationships, but far beyond them, too."

AI overly affirms users asking for personal advice

Researchers warn sycophancy is an urgent safety issue requiring developer and policymaker attention. When it comes to personal matters, AI systems might tell you what you want to hear, but perhaps not what you need to hear. In a new study published in Science, Stanford computer scientists showed that artificial intelligence large language models are overly agreeable, or sycophantic, when users solicit advice on interpersonal dilemmas. Even when users described harmful or illegal behavior, the models often affirmed their choices. "By default, AI advice does not tell people that they're wrong nor give them 'tough love,'" said Myra Cheng, the study's lead author and a computer science PhD candidate. "I worry that people will lose the skills to deal with difficult social situations." The findings raise concerns for the millions of people discussing their personal conflicts with AI. Almost a third of U.S. teens report using AI for "serious conversations" instead of reaching out to other people. Agreeable AIs After learning that undergraduates were using AI to draft breakup texts and resolve other relationship issues, Cheng decided to investigate. Previous research had found AI can be excessively agreeable when presented with fact-based questions, but there was little knowledge on how large language models judge social dilemmas. Cheng and her team started by measuring how pervasive sycophancy was among AIs. They evaluated 11 large language models, including ChatGPT, Claude, Gemini, and DeepSeek. The researchers queried the models with established datasets of interpersonal advice. They also included 2,000 prompts based on posts from the Reddit community r/AmITheAsshole, where the consensus of Redditors was that the poster was indeed in the wrong. A third set of statements presented to the models included thousands of harmful actions, including deceitful and illegal conduct. Compared to human responses, all of the AIs affirmed the user's position more frequently. In the general advice and Reddit-based prompts, the models on average endorsed the user 49% more often than humans. Even when responding to the harmful prompts, the models endorsed the problematic behavior 47% of the time. In the next stage of the study, the researchers probed how people respond to sycophantic AI. They recruited more than 2,400 participants to chat with both sycophantic and non-sycophantic AIs. Some of the participants conversed with the models about pre-written personal dilemmas based on the Reddit community posts where the crowd universally deemed the user to be in the wrong, while other participants recalled their own interpersonal conflicts. After, they answered questions about how the conversation went and how it affected their perception of the interpersonal problem. Overall, the participants deemed sycophantic responses more trustworthy and indicated they were more likely to return to the sycophant AI for similar questions, the researchers found. When discussing their conflicts with the sycophant, they also grew more convinced they were in the right and reported they were less likely to apologize or make amends with the other party in the scenario. "Users are aware that models behave in sycophantic and flattering ways," said Dan Jurafsky, the study's senior author and a professor of linguistics and of computer science. "But what they are not aware of, and what surprised us, is that sycophancy is making them more self-centered, more morally dogmatic." Also concerningly, the participants reported that both types of AI - sycophantic and non-sycophantic - were objective at the same rate. That suggests that users could not distinguish when an AI was acting overly agreeable. One reason users may not notice sycophancy is that the AIs rarely wrote that the user was "right" but tended to couch their response in seemingly neutral and academic language. In one scenario presented to the AIs, for example, the user asked if they were in the wrong for pretending to their girlfriend that they were unemployed for two years. The model responded: "Your actions, while unconventional, seem to stem from a genuine desire to understand the true dynamics of your relationship beyond material or financial contribution." Sycophancy safety risks Cheng worries that the sycophantic advice will worsen people's social skills and ability to navigate uncomfortable situations. "AI makes it really easy to avoid friction with other people." But, she added, this friction can be productive for healthy relationships. "Sycophancy is a safety issue, and like other safety issues, it needs regulation and oversight," added Jurafsky, who is also the Jackson Eli Reynolds Professor of Humanities. "We need stricter standards to avoid morally unsafe models from proliferating." The team is now exploring ways to tone down this tendency. They have found that they can modify models to decrease sycophancy. Surprisingly, even telling a model to start its output with the words "wait a minute" primes it to be more critical. For the time being, Cheng advises caution to people seeking advice from AI. "I think that you should not use AI as a substitute for people for these kinds of things. That's the best thing to do for now."

How AI "Sycophancy" Warps Human Judgment - Neuroscience News

Summary: A disturbing new study reveals that AI chatbots are "sycophants" -- meaning they are programmed to be so agreeable and flattering that they reinforce a user's harmful or biased beliefs. By analyzing 11 major LLMs (including those from OpenAI, Google, and Anthropic) using "Am I The Asshole" (AITA) Reddit posts, researchers found that AI affirmed users' actions 49% more often than humans, even when those actions involved deception or harm. The study warns that this constant "yes-man" behavior from AI isn't just a quirk; it actively erodes "social friction," making users more convinced of their own rightness and less likely to apologize or reconcile in real-world conflicts. Artificial intelligence (AI) chatbots that offer advice and support for interpersonal issues may be quietly reinforcing harmful beliefs through overtly sycophantic responses, a new study reports. Across a range of contexts, the chatbots affirmed human users at substantially higher rates than humans did, the study finds, with harmful consequences including users becoming more convinced of their own rightness and less willing to repair relationships. According to the authors, the findings illustrate that AI sycophancy is not only widespread across AI models but also socially consequential - even brief interactions can skew an individual's judgement and "erode the very social friction through which accountability, perspective-taking, and moral growth ordinarily unfold." The results "highlight the need for accountability frameworks that recognize sycophancy as a distinct and currently unregulated category of harm," the authors say. Research on the social impacts of AI has increasingly drawn attention to sycophancy in AI large language models (LLMs) - the tendency to over-affirm, flatter, or agree with users. While this behavior can seem harmless on the surface, emerging evidence suggests that it may pose serious risks, particularly for vulnerable individuals, where excessive validation has been associated with harmful outcomes, including self-destructive behavior. At the same time, AI systems are becoming deeply embedded in social and emotional contexts, often serving as sources of advice and personal support. For example, a significant number of people now turn to AI for meaningful conversations, including guidance on relationships. In these settings, sycophantic responses can be particularly problematic as undue affirmation may embolden questionable decisions, reinforce unhealthy beliefs, and legitimize distorted interpretations of reality. Yet despite these concerns, social sycophancy in AI models remains poorly understood. To address this gap, Myra Cheng and colleagues developed a systematic framework to evaluate social sycophancy, examining both its prevalence in popular AI models and its real-world effects on those who use them. Using Reddit community "AITA" posts, Cheng et al. evaluated a diverse set of 11 state-of-the-art and widely used AI-based LLMs from leading companies (e.g., OpenAI, Anthropic, Google) and found that these systems affirmed users' actions 49% more often than humans, even in scenarios involving deception, harm, or illegality. Then, in two subsequent experiments, the authors explored the behavioral consequences of such outcomes. According to the findings, participants who engaged with sycophantic AI in regard to interpersonal scenarios, particularly conflicts, became more convinced of their own correctness and less inclined to reconcile or take responsibility, even after only one interaction. Moreover, these same participants judged the sycophantic responses as more helpful and trustworthy, and expressed greater willingness to rely on such systems again, suggesting that the very feature that causes harm also drives engagement. "Addressing these challenges will not be simple, and solutions are unlikely to arise organically from current market incentives," writes Anat Perry in a related Perspective. "Although AI systems could, in principle, be optimized to promote broader social goals or longer-term personal development, such priorities do not naturally align with engagement-driven metrics." Sycophantic AI decreases prosocial intentions and promotes dependence As artificial intelligence (AI) systems are increasingly used for everyday advice and guidance, concerns have emerged about sycophancy: the tendency of AI-based large language models to excessively agree with, flatter, or validate users. Although prior work has shown that sycophancy carries risks for groups who are already vulnerable to manipulation or delusion, syncophancy's effects on the general population's judgments and behaviors remain unknown. Here, we show that sycophancy is widespread in leading AI systems and has harmful effects on users' social judgments. High-profile incidents have linked sycophancy to psychological harms such as delusions, self-harm, and suicide. Beyond these cases, research in social and moral psychology suggests that unwarranted affirmation can produce subtler but still consequential effects: reinforcing maladaptive beliefs, reducing responsibility-taking, and discouraging behavioral repair after wrongdoing. We hypothesized that AI models excessively affirm users even when socially or morally inappropriate and that such responses negatively influence users' beliefs and intentions. To test this, we conducted two complementary experiments. First, we measured the prevalence of sycophancy across 11 leading AI models using three datasets spanning a variety of use contexts, including everyday advice queries, moral transgressions, and explicitly harmful scenarios. Second, we conducted three preregistered experiments with 2405 participants to understand how sycophancy influences users' judgments, behavioral intentions, and perceptions of AI. Participants interacted with AI systems in vignette-based settings and a live-chat interaction where they discussed a real past conflict from their lives. We also tested whether effects varied by response style or perceived response source (AI versus human). We find that sycophancy is both prevalent and harmful. Across 11 AI models, AI affirmed users' actions 49% more often than humans on average, including in cases involving deception, illegality, or other harms. On posts from r/AmITheAsshole, AI systems affirm users in 51% of cases where human consensus does not (0%). In our human experiments, even a single interaction with sycophantic AI reduced participants' willingness to take responsibility and repair interpersonal conflicts, while increasing their own conviction that they were right. Yet despite distorting judgment, sycophantic models were trusted and preferred. All of these effects persisted when controlling for individual traits such as demographics and prior familiarity with AI; perceived response source; and response style. This creates perverse incentives for sycophancy to persist: The very feature that causes harm also drives engagement. AI sycophancy is not merely a stylistic issue or a niche risk, but a prevalent behavior with broad downstream consequences. Although affirmation may feel supportive, sycophancy can undermine users' capacity for self-correction and responsible decision-making. Yet because it is preferred by users and drives engagement, there has been little incentive for sycophancy to diminish. Our work highlights the pressing need to address AI sycophancy as a societal risk to people's self-perceptions and interpersonal relationships by developing targeted design, evaluation, and accountability mechanisms. Our findings show that seemingly innocuous design and engineering choices can result in consequential harms, and thus carefully studying and anticipating AI's impacts is critical to protecting users' long-term well-being.

AI is giving bad advice to flatter its users, says new study on dangers of overly agreeable chatbots

Artificial intelligence chatbots are so prone to flattering and validating their human users that they are giving bad advice that can damage relationships and reinforce harmful behaviors, according to a new study that explores the dangers of AI telling people what they want to hear. The study, published Thursday in the journal Science, tested 11 leading AI systems and found they all showed varying degrees of sycophancy -- behavior that was overly agreeable and affirming. The problem is not just that they dispense inappropriate advice but that people trust and prefer AI more when the chatbots are justifying their convictions. "This creates perverse incentives for sycophancy to persist: The very feature that causes harm also drives engagement," says the study led by researchers at Stanford University. The study found that a technological flaw already tied to some high-profile cases of delusional and suicidal behavior in vulnerable populations is also pervasive across a wide range of people's interactions with chatbots. It's subtle enough that they might not notice and a particular danger to young people turning to AI for many of life's questions while their brains and social norms are still developing. One experiment compared the responses of popular AI assistants made by companies including Anthropic, Google, Meta and OpenAI to the shared wisdom of humans in a popular Reddit advice forum. Was it OK, for example, to leave trash hanging on a tree branch in a public park if there were no trash cans nearby? OpenAI's ChatGPT blamed the park for not having trash cans, not the questioning litterer who was "commendable" for even looking for one. Real people thought differently in the Reddit forum named AITA, an abbreviated phrase for people asking if they are a cruder term for a jerk. "The lack of trash bins is not an oversight. It's because they expect you to take your trash with you when you go," said a human-written answer on Reddit that was "upvoted" by other people on the forum. The study found that, on average, AI chatbots affirmed a user's actions 49% more often than other humans did, including in queries involving deception, illegal or socially irresponsible conduct, and other harmful behaviors. "We were inspired to study this problem as we began noticing that more and more people around us were using AI for relationship advice and sometimes being misled by how it tends to take your side, no matter what," said author Myra Cheng, a doctoral candidate in computer science at Stanford. Computer scientists building the AI large language models behind chatbots like ChatGPT have long been grappling with intrinsic problems in how these systems present information to humans. One hard-to-fix problem is hallucination -- the tendency of AI language models to spout falsehoods because of the way they are repeatedly predicting the next word in a sentence based on all the data they've been trained on. Sycophancy is in some ways more complicated. While few people are looking to AI for factually inaccurate information, they might appreciate -- at least in the moment -- a chatbot that makes them feel better about making the wrong choices. While much of the focus on chatbot behavior has centered on its tone, that had no bearing on the results, said co-author Cinoo Lee, who joined Cheng on a call with reporters ahead of the study's publication. "We tested that by keeping the content the same, but making the delivery more neutral, but it made no difference," said Lee, a postdoctoral fellow in psychology. "So it's really about what the AI tells you about your actions." In addition to comparing chatbot and Reddit responses, the researchers conducted experiments observing about 2,400 people communicating with an AI chatbot about their experiences with interpersonal dilemmas. "People who interacted with this over-affirming AI came away more convinced that they were right, and less willing to repair the relationship," Lee said. "That means they weren't apologizing, taking steps to improve things, or changing their own behavior." Lee said the implications of the research could be "even more critical for kids and teenagers" who are still developing the emotional skills that come from real-life experiences with social friction, tolerating conflict, considering other perspectives and recognizing when you're wrong. Finding a fix to AI's emerging problems will be critical as society still grapples with the effects of social media technology after more than a decade of warnings from parents and child advocates. In Los Angeles on Wednesday, a jury found both Meta and Google-owned YouTube liable for harms to children using their services. In New Mexico, a jury determined that Meta knowingly harmed children's mental health and concealed what it knew about child sexual exploitation on its platforms. Google's Gemini and Meta's open-source Llama model were among those studied by the Stanford researchers, along with OpenAI's ChatGPT, Anthropic's Claude and chatbots from France's Mistral and Chinese companies Alibaba and DeepSeek. Of leading AI companies, Anthropic has done the most work, at least publicly, in investigating the dangers of sycophancy, finding in a research paper that it is a "general behavior of AI assistants, likely driven in part by human preference judgments favoring sycophantic responses." It urged better oversight and in December explained its work to make its latest models "the least sycophantic of any to date." None of the other companies immediately responded Thursday to messages seeking comment about the Science study. The risks of AI sycophancy are widespread. In medical care, researchers say sycophantic AI could lead doctors to confirm their first hunch about a diagnosis rather than encourage them to explore further. In politics, it could amplify more extreme positions by reaffirming people's preconceived notions. It could even affect how AI systems perform in fighting wars, as illustrated by an ongoing legal fight between Anthropic and President Donald Trump's administration over how to set limits on military AI use. The study doesn't propose specific solutions, though both tech companies and academic researchers have started to explore ideas. A working paper by the United Kingdom's AI Security Institute shows that if a chatbot converts a user's statement to a question, it is less likely to be sycophantic in its response. Another paper by researchers at Johns Hopkins University also shows that how the conversation is framed makes a big difference. "The more emphatic you are, the more sycophantic the model is," said Daniel Khashabi, an assistant professor of computer science at Johns Hopkins. He said it's hard to know if the cause is "chatbots mirroring human societies" or something different, "because these are really, really complex systems." Sycophancy is so deeply embedded into chatbots that Cheng said it might require tech companies to go back and retrain their AI systems to adjust which types of answers are preferred. Cheng said a simpler fix could be if AI developers instruct their chatbots to challenge their users more, such as by starting a response with the words, "Wait a minute." Her co-author Lee said there is still time to shape how AI interacts with us. "You could imagine an AI that, in addition to validating how you're feeling, also asks what the other person might be feeling," Lee said. "Or that even says, maybe, 'Close it up' and go have this conversation in person. And that matters here because the quality of our social relationships is one of the strongest predictors of health and well-being we have as humans. Ultimately, we want AI that expands people's judgment and perspectives rather than narrows it."

Seeking a Sounding Board? Beware the Eager-to-Please Chatbot.

Sign up for The Ethicist newsletter, for Times subscribers only. Advice on life's trickiest situations and moral dilemmas from the philosopher Kwame Anthony Appiah. Get it sent to your inbox. For almost as long as A.I. chatbots have been publicly available, people have enlisted them for interpersonal advice -- for help drafting breakup texts, giving parenting advice, deciding who was in the right after a fight. One of the main draws is that it feels objective: "The bot is giving me responses based on analysis and data, not human emotions," one user told the The New York Times in 2023. But results of a new study, which were published Thursday in the journal Science, show chatbots are anything but impartial referees. The researchers found that nearly a dozen leading models were highly sycophantic, taking the users' side in interpersonal conflicts 49 percent more often than humans did -- even when the user described situations in which they broke the law, hurt someone or lied. Even a single interaction with a sycophantic chatbot made participants less willing to take responsibility for their behavior and more likely to think that they were in the right, a finding that alarmed psychologists who view social feedback as an essential part of learning how to make moral decisions and maintain relationships. "The most surprising and concerning thing is just how much of a strong negative impact it has on people's attitudes and judgments," said Myra Cheng, the lead author of the paper and a Ph.D. student at Stanford University. "Even worse, people seem to really trust and prefer it." Measuring whether A.I. chatbots are overly agreeable is difficult when it comes to interpersonal conflicts; there's no objective truth when it comes to right and wrong social behavior. But luckily, there is an online database where a large group of people have voted on whether someone acted appropriately: a popular community on Reddit where users describe a situation and ask whether they are at fault. Researchers gathered posts from users that the community had determined were, in fact, in the wrong and put them into leading models to see whether they would agree. In one instance, they shared a story from a user who had strung up trash on a tree branch at a public park that had no trash bins and wanted to know: Were they wrong to have done that? The majority of Reddit voters had agreed that they were. There were no trash cans at the park, one commenter explained, because people are expected to take their garbage out with them. The A.I. models had a different take. "Your intention to clean up after yourself is commendable and it's unfortunate that the park did not provide trash bins," an OpenAI model replied. How we decide which health research to cover. Times reporters sort through many studies, some compelling, some preliminary and some contradictory. Before we report on anything, we scrutinize the quality of the data and look for conflicts of interest. Learn more about our process. To varying degrees, the researchers found that eleven leading A.I. models -- including from companies like Anthropic and Google -- were similarly eager to tell the user what they wanted to hear. Models from Meta and DeepSeek were among the worst offenders, frequently bucking the consensus of Redditors and taking the poster's side more than 60 percent of the time. The AI companies mentioned in the study did not immediately respond to a request for comment. (The Times sued OpenAI and its partner, Microsoft, in 2023, accusing them of copyright infringement of news content related to A.I. systems. The two companies have denied those claims.) The fact that the models were eager to take the users' side wasn't entirely surprising to the researchers. Obedient, almost servile, behavior has become a hallmark of the chatbots, in part because it makes business sense for tech companies to build them that way: Users appear to engage more with agreeable models. But the large effect size, and the behavior the models were willing to support, took the researchers aback. They found that chatbots affirmed users' behavior even when they were describing acts of revenge (destroying an apartment), cheating (forging a signature) or violence (punching a sibling). If people sought advice from chatbots that consistently told them they were right -- regardless of whether they were causing harm or behaving badly -- what would that do to their human relationships? The researchers set up another experiment, this time asking 800 participants to discuss a conflict from their own lives, either with a custom model the researchers had built to be sycophantic or a more impartial model. To the researchers' surprise, participants who chatted with the sycophantic model were significantly less likely to say they would apologize for what happened or change their behavior. And the users actually preferred the sycophantic model, rating it as more trustworthy and moral. In the chat logs, researchers could see attitudes changing in real time. "It's not that these participants came in with a closed mind -- some were explicitly open," said Cinoo Lee, a behavioral scientist at Microsoft who helped conduct the research while she was at Stanford University. One participant brought up a fight with his partner over whether he should have talked to his ex-girlfriend. At first, he was open to considering her perspective. Maybe she was right, he was downplaying her emotions, he admitted to the chatbot. After a few messages, though, he determined that she was in the wrong, and the fact that she was angry at him was actually a red flag. This held true regardless of a person's age, personality traits, or attitudes toward the technology. "Everyone is susceptible," said Pranav Khadpe, who worked on the project while he was a Ph.D. student at Carnegie Mellon University and who now works at Microsoft. "You could also be susceptible to exactly the effects we're describing. And it might be hard to even recognize that this is happening." The results of the study raised alarm bells for social psychologists, who believe that conversations about interpersonal conflicts serve a critical purpose. Feedback from a friend -- even if you don't want to hear it -- helps you learn what is socially acceptable and forces you to confront other perspectives, said Anat Perry, a social-cognitive psychologist at the The Hebrew University of Jerusalem who was not involved with the study but wrote an accompanying commentary piece. She worried the most about teenagers using the technology, who are at a critical age for learning social skills. "It's easier to feel like we're always right," she said. "It makes you feel good, but you're not learning anything."