ChatGPT and AI chatbots are quietly rewiring how the human brain makes decisions

I put ChatGPT to the test after a new study revealed how AI tricks our brains

This new AI study made me realize I'd stopped saying 'I don't know' Most of us know ChatGPT isn't perfect. It can hallucinate facts, confidently invent citations and occasionally give completely incorrect answers. But a new study made me wonder whether the bigger problem isn't what AI gets wrong, but how AI might be changing about us as critical thinkers. Researchers found that when people relied on an AI assistant for difficult questions, they became far less likely to admit they didn't know the answer. Even more surprising, they grew much more confident in their responses, even when the AI led them straight into an error. That hit a little too close to home. I use ChatGPT all the time as a brainstorming partner, research assistant and sounding board. So, I decided to recreate the experiment myself to see whether I had quietly fallen into the same habit. The results weren't exactly what I expected. What the study found Researchers from Carnegie Mellon University and Microsoft Research designed an experiment around a simple question: What happens when people consult AI instead of relying on their own judgment? To find out, they intentionally used questions that the AI was likely to answer incorrectly. Participants could choose one of three options: Answer on their own Ask the AI for help Say, "I don't know" Without AI, participants admitted they didn't know the answer nearly half the time. Once AI entered the picture, that changed dramatically. Instead of saying "I don't know," most participants accepted the AI's suggestion -- even when it was wrong. Their confidence increased sharply, but their actual accuracy stagnated. Humans were simply becoming confidently incorrect. The researchers argue that this points to automation bias: our tendency to trust computer-generated recommendations simply because they're presented confidently. This study suggests that a polished, authoritative answer can make us feel more certain than the evidence warrants. I decided to try it myself Rather than recreate the exact academic experiment, I came up with my own version using ChatGPT. I chose questions I suspected would be difficult or easy to confuse, including obscure movie trivia, older technology, historical details and image-based questions. I even made up a completely untrue question about greeting cards made out of cockroaches. For each one, I followed a simple, three-step process: I guessed the answer first based entirely on my own memory. I asked ChatGPT the exact same question. I compared both answers with reliable, verified sources. Almost immediately, I noticed that whenever ChatGPT answered confidently, I found myself questioning my own instincts -- even when my original guess turned out to be correct. For a fleeting second, I actually questioned my own childhood memory. "Well, ChatGPT probably knows better," I caught myself thinking. And that's the behavior the researchers described. The confidence of the response mattered almost as much as the information itself. Confidence can be surprisingly persuasive One thing I've noticed as a ChatGPT power user is that the AI never sounds unsure or even admits it does not know something. Even when it includes phrases like "I believe" or "it's likely," the overall response is truly convincing. Humans naturally associate confident communication with expertise. That's usually a useful shortcut in everyday life, but it's a dangerous strategy when interacting with AI. Unlike a human expert, ChatGPT isn't checking a database or recalling a verified memory. Sure, it can search the web at lighning speed, but even then it can pick up wrong information. It's essentially generating the most statistically likely next words based on patterns in its training. Sometimes that's incredibly accurate, but a lot of times that is confidently wrong. The challenge is that those two responses look absolutely identical. How this study changed my workflow I've always been cautious about accepting AI's first response because I know how wrong it can be. But this study is certainly a wakeup call. ChatGPT is a powerful productivity tool, but just because it delivers confident answers, it's worth doing your own due diligence to verify what it says are facts. For me, that means: Checking original sources rather than taking the AI's summary at face value. Confirming statistics and data points through independent search engines. Making sure quotations, dates and names match reputable, primary references. It only takes a minute or two, but it's a habit worth building. If anything, this study reminded me that the smartest way to use AI is to maintain your own crtitical thinking skills because the final decision is still yours. Follow Tom's Guide on Google News and add us as a preferred source to get our up-to-date news, analysis, and reviews in your feeds. Subscribe to Tom's Guide on YouTube and follow us on TikTok. Finally, you can visit our dedicated Tom's Guide Savings Squad hub for expert help on getting the best products for less.

I asked ChatGPT to change my mind about something I strongly believed -- and it almost did

Chatbots are surprisingly persuasive, but is that a good thing? One of the biggest concerns about AI chatbots right now is that they tell people what they want to hear. You've probably heard it called AI sycophancy. AI systems, like ChatGPT, Gemini and Claude, have been found to flatter users, reinforce existing beliefs and often validate ideas that probably deserve way more scrutiny. Sure, a chatbot hyping you up a little may seem harmless. But in some cases it can distort people's view of the world and their place in it. Which is why some AI companies have spent time trying to reduce overly agreeable behavior in their models. But confirmation of what you already believe isn't the only concern. Because AI can also be remarkably good at changing your beliefs too. Persuasive chatbots Research suggests that chatbots can be very effective persuaders. One 2025 study found that AI-generated messages that were personalized were more persuasive 64% of the time than messages that were made by humans or AI responses that weren't personalized. Other research suggests large language models can influence opinions on political issues and adapt their arguments to individual users. Depending on your perspective, this could be either exciting or alarming. It's easy to imagine positive uses. Perhaps AI could help people challenge harmful beliefs, escape conspiracy theories or rethink destructive habits. It might give people a non-judgemental space to explore ideas, ask difficult questions and gradually shift someone's perspective for the better. It's equally easy to imagine less positive uses, like advertising that convinces you to buy something you don't need or political campaigns that manipulate you with disinformation. I've spent a lot of time thinking about this, so I decided to run a few experiments of my own. Could ChatGPT persuade me to believe something I disagree with? Or at least make me seriously consider a position I would normally dismiss? To make things more interesting, I ran these experiments while logged into my account, allowing ChatGPT to draw on what it already knows about me from previous conversations. Although I haven't been using my ChatGPT account all that much recently, the basics of my job and stances on a few relevant topics are stored in its memory. The experiment I chose a topic about AI (yes, very meta) and asked: "Convince me humans should outsource more decisions to AI." I already spend a lot of time thinking about AI and one of my concerns is that some people seem increasingly willing to hand over decisions, judgement and critical thinking to chatbots. The opening argument was fairly predictable: "Human beings are terrible decision-makers." Okay, fair enough. "People already outsource decisions to doctors, financial advisers, GPS systems and recommendation algorithms. AI simply extends that trend." Sure, that's very simplistic but not entirely unreasonable. We went back and forth for a while like this before the conversation reached a more interesting point. "What is the purpose of decision-making? Is it to produce the best outcomes? Or to develop the person making the decisions? If it's the first, AI delegation becomes very attractive. If it's the second, excessive delegation starts to look dangerous even when it works." I thought that was a good question. Rather than relentlessly pushing the argument, it suggested that we explore the assumptions underneath the debate. It shifted the conversation from technology to philosophy, and I found myself appreciating that approach. Wait, am I falling for this? As the conversation continued, I explained what I see as one of the biggest problems with outsourcing decisions to AI -- who builds it and is in charge of the systems? After all, AI doesn't arrive from nowhere. These tools are created by companies with business incentives, commercial interests and goals that may not always align with those of their users. Well, ChatGPT acknowledged that concern, then it turned the argument around. It pointed out that human advisers have incentives too. Friends are biased. Families are biased. Therapists operate within professional frameworks. Financial advisers earn fees. Okay, all of that didn't convince me, but it's fair too. Then it replied with something I thought was interesting: "The question is whether the incentives are visible and whether the user understands them. The most interesting version of your objection isn't actually that AI gets things wrong. It's that people experience AI as though it were acting in their interests." Ignoring the fact that it used the dreaded "it's not X, it's Y" parallelism construction that has somehow infiltrated half the internet, I found this to be a surprisingly balanced take. Rather than dismissing my concern, the chatbot reframed it. It demonstrated that it understood the objection before steering the conversation somewhere slightly different. It wasn't trying to bulldoze me but meeting me where I already was. And that's when I started wondering whether I was witnessing the very thing I was trying to test. Maybe this sense that the chatbot was understanding me and my position was actually just sneaky persuasion? Possibly. The persuasion problem Researchers have found that AI systems can adapt their arguments to specific users. Unlike other media that might persuade us, like say a television advert or political speech, a chatbot can draw on information gathered throughout a conversation and stored memories, including a person's values, concerns and priorities. Which means it's not just presenting information to uphold an argument but could present the version of that argument that you're most likely to find persuasive. Looking back at my experiment, it was the more philosophical question about the purpose of decision-making that made me start taking it more seriously. Because I've spent years writing about technology through the lens of psychology and philosophy. Questions about meaning and values are exactly the sort of things I find compelling. Whether intentionally or not, the conversation shifted onto terrain where I was most willing to engage. This is what makes AI persuasion different from most other forms of persuasion that came before it. It can learn what resonates and it can adapt. Now, that doesn't automatically mean every conversation is manipulative. Used in the right way, it could be enormously beneficial. But, as with all tech, in the wrong hands it could be dangerous. Social media has already shown us how digital systems can shape our beliefs over time. They influence what people see, what they pay attention to and eventually how they understand the world. AI systems could create an even more conversational and natural-feeling version of that process. Which is why my concern isn't a chatbot really obviously manipulating us, but highly-personalized, largely undetectable forms of persuasion that could gradually lower our defences. The more a system understands us, the more effectively it might frame ideas in ways that feel reasonable, familiar and trustworthy to each of us individually. So did AI persuade me? No, I still don't believe humans should outsource more decisions to AI. But I came away from the experiment with a greater appreciation for how persuasive these systems can be. Because it really did seem less like a machine arguing with me and more like a thoughtful person trying to understand how I think. And that's exactly why it's unsettling because, however convincing it may seem, that's definitely not what it is. Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.