AI Users Fall Into Reverse Dunning-Kruger Trap, Study Finds

When Using AI, Users Fall for the Dunning-Kruger Trap in Reverse - Neuroscience News

Summary: A new study reveals that when interacting with AI tools like ChatGPT, everyone -- regardless of skill level -- overestimates their performance. Researchers found that the usual Dunning-Kruger Effect disappears, and instead, AI-literate users show even greater overconfidence in their abilities. The study suggests that reliance on AI encourages "cognitive offloading," where users trust the system's output without reflection or double-checking. Experts say AI literacy alone isn't enough; people need platforms that foster metacognition and critical thinking to recognize when they might be wrong. When it comes to estimating how good we are at something, research consistently shows that we tend to rate ourselves as slightly better than average. This tendency is stronger in people who perform low on cognitive tests. It's known as the Dunning-Kruger Effect (DKE) -- the worse people are at something the more they tend to overestimate their abilities and the "smarter" they are, the less they realise their true abilities. However, a study led by Aalto University reveals that when it comes to AI, specifically, Large Language Models (LLMs), the DKE doesn't hold, with researchers finding that all users show a significant inability to assess their performance accurately when using ChatGPT. In fact, across the board, people overestimated their performance. On top of this, the researchers identified a reversal of the Dunning-Kruger Effect -- an identifiable tendency for those users who considered themselves more AI literate to assume their abilities were greater than they really were. 'We found that when it comes to AI, the DKE vanishes. In fact, what's really surprising is that higher AI literacy brings more overconfidence,' says Professor Robin Welsch. 'We would expect people who are AI literate to not only be a bit better at interacting with AI systems, but also at judging their performance with those systems - but this was not the case.' The finding adds to a rapidly growing volume of research indicating that blindly trusting AI output comes with risks like 'dumbing down' people's ability to source reliable information and even workforce de-skilling. While people did perform better when using ChatGPT, it's concerning that they all overestimated that performance. 'AI literacy is truly important nowadays, and therefore this is a very striking effect. AI literacy might be very technical, and it's not really helping people actually interact fruitfully with AI systems', saysWelsch. 'Current AI tools are not enough. They are not fostering metacognition [awareness of one's own thought processes] and we are not learning about our mistakes,' adds doctoral researcher Daniela da Silva Fernandes. 'We need to create platforms that encourage our reflection process.' The article was published on October 27th in the journal Computers in Human Behavior. Why a single prompt is not enough The researchers designed two experiments where some 500 participants used AI to complete logical reasoning tasks from the US's famous Law School Admission Test (LSAT). Half of the group used AI and half didn't. After each task, subjects were asked to monitor how well they performed -- and if they did that accurately, they were promised extra compensation. 'These tasks take a lot of cognitive effort. Now that people use AI daily, it's typical that you would give something like this to AI to solve, because it's so challenging', Welsch says. The data revealed that most users rarely prompted ChatGPT more than once per question. Often, they simply copied the question, put it in the AI system, and were happy with the AI's solution without checking or second-guessing. 'We looked at whether they truly reflected with the AI system and found that people just thought the AI would solve things for them. Usually there was just one single interaction to get the results, which means that users blindly trusted the system. It's what we call cognitive offloading, when all the processing is done by AI', Welsch explains. This shallow level of engagement may have limited the cues needed to calibrate confidence and allow for accurate self-monitoring. Therefore, it's plausible that encouraging or experimentally requiring multiple prompts could provide better feedback loops, enhancing users' metacognition, he says. So what's the practical solution for everyday AI users? 'AI could ask the users if they can explain their reasoning further. This would force the user to engage more with AI, to face their illusion of knowledge, and to promote critical thinking,' Fernandes says. AI makes you smarter but none the wiser: The disconnect between performance and metacognition Optimizing human-AI interaction requires users to reflect on their performance critically, yet little is known about generative AI systems' effect on users' metacognitive judgments. In two large-scale studies, we investigate how AI usage is associated with users' metacognitive monitoring and performance in logical reasoning tasks. Specifically, our paper examines whether people using AI to complete tasks can accurately monitor how well they perform. In Study 1, participants (N = 246) used AI to solve 20 logical reasoning problems from the Law School Admission Test. While their task performance improved by three points compared to a norm population, participants overestimated their task performance by four points. Interestingly, higher AI literacy correlated with lower metacognitive accuracy, suggesting that those with more technical knowledge of AI were more confident but less precise in judging their own performance. Using a computational model, we explored individual differences in metacognitive accuracy and found that the Dunning-Kruger effect, usually observed in this task, ceased to exist with AI use. Study 2 (N = 452) replicates these findings. We discuss how AI levels cognitive and metacognitive performance in human-AI interaction and consider the consequences of performance overestimation for designing interactive AI systems that foster accurate self-monitoring, avoid overreliance, and enhance cognitive performance.

AI use makes us overestimate our cognitive performance, study reveals

When it comes to estimating how good we are at something, research consistently shows that we tend to rate ourselves as slightly better than average. This tendency is stronger in people who perform low on cognitive tests. It's known as the Dunning-Kruger Effect (DKE): The worse people are at something, the more they tend to overestimate their abilities, and the "smarter" they are, the less they realize their true abilities. However, a study led by Aalto University reveals that when it comes to AI, specifically, large language models (LLMs), the DKE doesn't hold, with researchers finding that all users show a significant inability to assess their performance accurately when using ChatGPT. In fact, across the board, people overestimated their performance. On top of this, the researchers identified a reversal of the Dunning-Kruger Effect--an identifiable tendency for those users who considered themselves more AI literate to assume their abilities were greater than they really were. "We found that when it comes to AI, the DKE vanishes. In fact, what's really surprising is that higher AI literacy brings more overconfidence," says Professor Robin Welsch. "We would expect people who are AI literate to not only be a bit better at interacting with AI systems, but also at judging their performance with those systems -- but this was not the case." The finding adds to a rapidly growing volume of research indicating that blindly trusting AI output comes with risks like "dumbing down" people's ability to source reliable information and even workforce de-skilling. While people did perform better when using ChatGPT, it's concerning that they all overestimated that performance. "AI literacy is truly important nowadays, and therefore this is a very striking effect. AI literacy might be very technical, and it's not really helping people actually interact fruitfully with AI systems," says Welsch. "Current AI tools are not enough. They are not fostering metacognition [awareness of one's own thought processes] and we are not learning about our mistakes," adds doctoral researcher Daniela da Silva Fernandes. "We need to create platforms that encourage our reflection process." The article appears in Computers in Human Behavior. Why a single prompt is not enough The researchers designed two experiments where some 500 participants used AI to complete logical reasoning tasks from the US's famous Law School Admission Test (LSAT). Half of the group used AI and half didn't. After each task, subjects were asked to monitor how well they performed, and if they did that accurately, they were promised extra compensation. "These tasks take a lot of cognitive effort. Now that people use AI daily, it's typical that you would give something like this to AI to solve, because it's so challenging," Welsch says. The data revealed that most users rarely prompted ChatGPT more than once per question. Often, they simply copied the question, put it in the AI system, and were happy with the AI's solution without checking or second-guessing. "We looked at whether they truly reflected with the AI system and found that people just thought the AI would solve things for them. Usually there was just one single interaction to get the results, which means that users blindly trusted the system. It's what we call cognitive offloading, when all the processing is done by AI," Welsch explains. This shallow level of engagement may have limited the cues needed to calibrate confidence and allow for accurate self-monitoring. Therefore, it's plausible that encouraging or experimentally requiring multiple prompts could provide better feedback loops, enhancing users' metacognition, he says. So what's the practical solution for everyday AI users? "AI could ask the users if they can explain their reasoning further. This would force the user to engage more with AI, to face their illusion of knowledge, and to promote critical thinking," Fernandes says.