Google AI Overviews deliver millions of wrong answers every hour, analysis reveals

Google's AI Summaries Are Regularly Lying to You, Report Finds

AI hallucinations are nothing new, but a recent investigation found that Google's AI Overviews search results have an accuracy rate of 90%. Although that's a high margin, it also means tens of millions of search results every hour are potentially flat-out wrong. A report from The New York Times and AI startup Oumi found that one in 10 Google queries produced at least one summary with incorrect information, and in half of the cases where information was correct, there was a link to a source that didn't support the summary's claims. Things have improved, but not by much. Oumi's analysis found that of the 4,326 searches conducted in October 2025, Gemini 2 produced accurate responses around 85% of the time. In February, when the same test was conducted using Gemini 3, accuracy improved to 91%. Summary sourcing, however, degraded with Gemini 3. Oumi's data suggests that Gemini 2 produced erroneous source links 37% of the time last year, but now does it more than 56% of the time in 2026. Oumi suggests this may be because two of the most commonly cited sources by Gemini's AI summaries are Facebook and Reddit. The NYT's report also showed how a BBC journalist used a deliberately misleading article they had created to poison the AI. Google's summary bot took the bait and, within 24 hours, repeated phony information from the source article. Google disputes the results and notes that Oumi used the SimpleQA benchmark, an AI test developed by OpenAI that contains incorrect information in its own right. Google argues that the test doesn't reflect what people actually search for on Google and that summaries may differ for each search query. It also says Oumi uses its own AI systems to analyze the AI summaries, which could, in turn, lead to mistakes. Although those points could be valid, it's also a pot-kettle situation. If your argument is that a report about your AI being inaccurate is wrong, because the company is trying to show it used AI that could be inaccurate, it doesn't exactly raise confidence in the accuracy of your AI. Google's AI search summaries have been blamed for a downturn in publisher site traffic and job losses, which Google also disputes. More recently, the company has been using its AI to summarize headlines and news stories on Google Discover and Search, with questionable results.

Study: Google's AI Overviews show millions of wrong answers every hour

Aside from the hallucinations, immense energy requirements, and potentially negative mental health effects, generative artificial intelligence still has an issue with accuracy. However, that hasn't stopped major tech companies like Google from rolling out features like AI search summaries to its users. Most of the results seem fine at first glance and usually feature multiple source citations, but that doesn't mean the product works perfectly. A recent study reported by the New York Times found that Google's "AI Overview" offers correct and reputably sourced summaries 9 out of 10 times. But while 90 percent sounds like a passing grade, the failure rate adds up in a matter of minutes. Given that the company will process over five trillion searches in 2026, the ensuing math means AI Overview is churning out tens of millions of questionable answers each hour. That's hundreds of thousands of errors every minute. What's more, it's difficult to truly assess AI Overview's accuracy. An initially wrong response to a search query may transform into a correct summation when a user repeats the search a second time, making it basically impossible to anticipate. Google's decision to place the AI Overview tab at the top of most search result pages also means more people see it and can assume its reliability. Part of the issue relates to the specific sites treated as sources. The study's authors at the open-source AI company Oumi found that Facebook and Reddit were the second- and fourth-most-cited references for AI Overview. Accurate answers cited Facebook 5 percent of the time, while inaccurate responses cited the social media site 7 percent of the time. In other cases, AI Overview seems to misstate a reliable source when giving a wrong answer. Then there is the problem of bad actors. Anyone who understands these bugs in the system can potentially game AI Overview into giving inaccurate statements. Hypothetically, a person could author a series of blog posts asserting flat-out wrong historical information, then artificially boost traffic to their website. Google's AI Overview may include the site in its source-scouring, fail to flag its inaccuracies, then generate a wrong answer. "Our Search AI features are built on the same ranking and safety protections that block the overwhelming majority of spam from appearing in our results," Google spokesperson Ned Adriance told the NYT. "Most of these examples are unrealistic searches that people wouldn't actually do." At the very least, it's important to view tools like AI Overview with a heavy dose of skepticism for the conceivable future. One faulty response in every 10 answers may not seem too serious, but think about how many search queries you have already made today. Studies also already indicate that overreliance on this type of tech may not be great for overall cognitive abilities. But if there is one thing you can trust, it's AI Overview's fine-print disclaimer:

Google AI overviews might hallucinate tens of millions of times per hour

Serving tech enthusiasts for over 25 years. TechSpot means tech analysis and advice you can trust. Cutting corners: Most search engines now present users with AI-generated overviews by default, sparking controversy over concerns about accuracy and lost click-through traffic. While testing suggests that Google's AI overviews are accurate most of the time, the enormous volume of queries the search engine processes each day likely still results in millions of incorrect responses. According to The New York Times, testing suggests that approximately one in 10 Google AI search overviews contains false information. Given that the search engine processes roughly 5 trillion queries per year, users could be exposed to more than 57 million inaccurate answers each hour - nearly 1 million per minute. The figures come from AI startup Oumi, which the Times asked to evaluate Gemini's accuracy using SimpleQA, a widely used generative AI benchmark. After analyzing 4,326 Google searches, Oumi found that Google's AI assistant, Gemini version 2, produced accurate overviews 85 percent of the time in October. By February, Gemini 3 had improved that figure to 91 percent. However, Oumi can evaluate large volumes of results only by relying on AI tools, which may also introduce errors. In addition, Google sometimes generates different AI overviews for the same query, even when it is repeated seconds apart. A Google spokesperson called Oumi's testing flawed, arguing that it does not reflect real-world search behavior. The company's internal testing indicates that Gemini 3, when operating independently of Google Search, hallucinates 28 percent of the time. Sourcing presents another challenge. Google attempts to support its AI overview results with relevant links, but those sources often do not substantiate Gemini's claims - whether accurate or not. In some cases, an incorrect AI overview is immediately followed by a link containing correct information; in others, an accurate overview cites a source with inaccurate information; and sometimes the linked pages contain no relevant information at all. Notably, discrepancies between AI overviews and their sources increased after the February update, rising from 37 percent of searches with Gemini 2 to 56 percent with Gemini 3. Researchers also found that AI overviews are susceptible to manipulation. In one example, a BBC journalist published a blog post containing false information and later found that Google repeated those claims the following day. Tellingly, Google and other AI companies acknowledge the technology's tenuous relationship with the truth in the fine print. Microsoft's terms of service describe its Copilot AI tool as intended for entertainment purposes, not for making important decisions. Google's AI overviews advise users to double-check responses, while xAI acknowledges that hallucinations can occur.

Google's AI answers are wrong 1 in 10 times -- I looked closer and the real problem is even worse

Google's AI-powered search results are supposed to make finding answers faster and easier. Since it's almost impossible to ignore them, you'd think they would be fairly reliable. But a new analysis suggests they may also be getting things wrong -- more often than most people realize. According to a report highlighted by Ars Technica, Google's AI Overviews -- the summaries that now appear at the top of some search results -- were inaccurate about 10% of the time during testing. At first glance, that might not sound alarming. No system is perfect, but after digging into the findings, it's clear the real issue isn't just how often these answers are wrong -- it's how hard it is to tell when they are. Here's a look at what's going on with Google. The mistakes aren't obvious When people think about AI getting things wrong, they usually imagine bizarre answers like obvious hallucinations. Even ChatGPT is proven to be wrong 1 in 4 times. But that's not what's happening here. Most of the errors identified in Google's AI Overviews weren't outrageous -- they were subtle. In some cases, the summaries: * left out important context * simplified complex topics too aggressively * or presented partially correct information as fully accurate That makes them far more dangerous than obvious mistakes as billions of users rely on Google every day. Because if something sounds reasonable, most people won't question it. Why 10% is a bigger deal than it sounds Google handles billions of searches every day and even a small error rate at that scale can translate into millions of incorrect or misleading answers daily. Unlike traditional search results, AI Overviews often sit above all the links, meaning, users may never click through to verify. In other words, the AI answer becomes the "final" answer and context from original sources gets lost. Ultimately, the margin for error matters a lot more here. The confidence problem If you use AI even casually, you may have noticed its level of confidence is high. It can offer an answer that sounds so strong that you'd never think to double check. This adds another layer to this that doesn't get talked about enough. AI doesn't just summarize information -- it presents it confidently. Even when an answer is incomplete or slightly off, it can still sound polished, clear and authoritative. That creates a subtle psychological effect. Meaning, the cleaner the answer feels, the more we trust it. And that's exactly where things can go wrong. Bottom line So should you trust Google's AI answers? I reccomend no, at least not blindly. A 10% error rate might sound small -- until you realize those mistakes are often subtle, confident and easy to miss. However, you also don't need to ignore them completely. They can be useful for quick summaries, getting a general sense of a topic and speeding up basic research. But they shouldn't be your final answer -- especially when accuracy matters. Follow Tom's Guide on Google News and add us as a preferred source to get our up-to-date news, analysis, and reviews in your feeds.

Analysis Finds That Google's AI Overviews Are Providing Misinformation at a Scale Possibly Unprecedented in the History of Human Civilization

Can't-miss innovations from the bleeding edge of science and tech Google's AI Overviews are peddling misinformation on a scale that may be virtually unprecedented in human history. A recent analysis conducted by the AI startup Oumi at the behest of The New York Times found that the AI-generated summaries, which appear above Google search results, are accurate around 91 percent of the time. In a sense, that may sound like an impressive figure. But here's an even more impressive one: five trillion. That's roughly the number of search queries that Google processes every year, translating to tens of millions of wrong answers that the AI Overviews are providing every hour -- and hundreds of thousands every minute, the analysis calculated. In other words, Google has created a misinformation crisis. Studies have shown that people tend to trust what an AI tells them without question, with one report finding that only 8 percent of users actually double checked an AI's answer. Another experiment found that users still listened to AI when it gave them the wrong answer nearly 80 percent of the time -- a grim trend the researchers dubbed "cognitive surrender." Large language models adopt an authoritative tone and can confidently present fabricated information as fact when it can't immediately glean a straight answer. Add the convenience that Google's AI Overviews offer, and it's easy to imagine untold numbers of users taking its summaries at their word. Oumi conducted the analysis using a test called SimpleQA, a widely used benchmark for AI accuracy in the industry which was designed by OpenAI. The first round of tests, conducted in October, used a version of the AI Overviews powered by Google's Gemini 2 model. A follow-up conducted in February tested the feature after it was switched to Gemini 3, its much-hyped upgrade. Each round of tests involved 4,326 Google searches. Gemini 3 came out the more accurate model, giving a factually sound response 91 percent of the time. Gemini 2 performed significantly worse, at just 85 percent accurate. On the one hand, it shows that the models are improving. On the other, it shows that Google was willing to foist a model on its userbase that was even more prone to hallucinating, in an ongoing experiment that's still misinforming hundreds of millions of people. Google called the analysis flawed. "This study has serious holes," Ned Adriance, a Google spokesman, told the NYT in a statement. "It doesn't reflect what people are actually searching on Google." Yet Google's own tests paint a no less damning picture, the reporting notes. In an internal analysis of Gemini 3, it found that the AI model produced incorrect information 28 percent of the time. Google claims, however, that AI Overviews are more accurate because they draw on Google search results before answering. The improvement between Gemini 2 and Gemini 3 may be papering over a more serious flaw. In the Oumi analysis, Gemini 2 provided answers that were "ungrounded" 37 percent of the time, meaning the AI Overviews cited websites that didn't support the information they provided. But with Gemini 3, this jumped to 56 percent. On top of suggesting that the AI is pulling facts out of thin air, ungrounded responses make it difficult for users to verify the AI's claims.

Study claims nearly 1 in 10 Google AI answers contain errors

Google's AI Overviews face scrutiny over accuracy, with reports indicating that nearly one in 10 responses contains false information. The New York Times highlights a significant potential impact on users as Google processes approximately 5 trillion queries annually, leading to over 57 million inaccurate answers each hour. The findings, released by AI startup Oumi, suggest that while Google's Gemini version 2 provided accurate results 85 percent of the time in October, this figure improved to 91 percent with the February release of Gemini 3. However, Oumi's testing methodology relies on AI tools which may introduce their own errors. A Google spokesperson criticized Oumi's evaluation as flawed and unrepresentative of typical search behavior. Internal tests show that Gemini 3 produces false outputs, or "hallucinates," 28 percent of the time when used outside the framework of Google Search. Sourcing challenges add to the concerns surrounding these AI Overviews. Google attempts to provide relevant links to support its responses but often these sources fail to substantiate the claims made by Gemini. Significant discrepancies in responses were noted, increasing from 37 percent of searches for Gemini 2 to 56 percent for Gemini 3 after updates in February. Researchers have highlighted the vulnerability of AI Overviews to manipulation. In one instance, a BBC journalist's inaccurate claims were echoed by Google the following day. Google and other AI firms, including Microsoft, have endorsed the need for users to verify the information provided by AI systems, emphasizing that such tools are not fully reliable.

Google's AI Overviews Are Making Mistakes at Massive Scale. Here's What to Know

Starting in 2024, Google began putting AI-generated summaries -- called AI Overviews -- at the top of its search results. Instead of just pointing users to websites, Google started answering questions directly. At Google's scale, even small mistakes can quickly turn into much bigger problems, and as its AI rolled out, those errors started showing up almost immediately. Study Claims 90% Accuracy A recent analysis by AI startup Oumi found that AI Overviews were correct about 90 percent of the time. That sounds strong, until you consider that Google processes more than five trillion searches a year. At that volume, even a 10 percent error rate could translate into tens of millions of wrong answers every hour, The New York Times reported. In some cases, those errors resulted in real-world consequences.

Testing suggests that Google's AI Overviews have 90 % accuracy rate

Nowadays, when you write something on Google, it means confronting AI Overviews, the Gemini-powered search robot that appears at the top of the results page. As it is with everything new, the start was rough, but in time it's getting better and usually provides the right answer. As reported by Ars Technica, a new analysis from The New York Times suggests, that the accuracy of AI Overviews is 90 %. This means that 10 % of AI answers is wrong, and for Google, that means hundreds of thousands of lies going out every minute of the day. The report includes several examples of where AI Overviews went wrong, among these being the date on which Bob Marley's former home became a museum, and the date on which Yo Yo Ma was inducted into the classical music hall of fame. Google spokesperson Ned Adriance told The New York Times, that "this study has serious holes. It doesn't reflect what people are actually searching on Google". So what should we think about this? It's just like Google itself reminds you at the bottom of every overview: "AI can make mistakes, so double-check responses".

Google AI Overviews May Be Peddling Misinformation, says NYT-led Research

The issue of AI companies crawling through digital news websites without permission and using their copyright content for AI training created a major issue till publishers drew up financial deals with such companies to take care of their revenue losses A new survey commissioned by The New York Times suggests that some AI-generated summaries appearing by Google Search could be inaccurate, but given the scale of queries it processes, the search giant may be spreading unprecedented misinformation. As publishers, that sentence brought a modicum of calmness back in our lives, even as we begin our preparations to drown out our doubts and concerns about the future of mainstream and digital media... once again over yet another weekend. The research was conducted by AI startup Oumi following concerns expressed by The New York Times around the AI-generated summaries on Google search. The survey found that these are accurate around 91% of the time. Sounds impressive right? But, juxtapose the fat that Google processes 5 trillion search queries each year and that remaining 9% appears larger. When AI Summaries delivered a dud In short, this means Google could be serving up wrong information (or is it misinformation) in tens of millions of cases through its AI Overviews by the hour, the research noted. Must say though that Google does manage to fix some of these wrong answers over a period of time as this writer found post the release of Bollywood blockbuster Dhurandhar last December. A search for "Dhurandhar Bhatawdekar" had Google's AI summary talking nonsense, though as an avid movie buff, this author is aware that the query points to the name of a character played by the legendary actor Utpal Dutt in a Bollywood flick called "Rang Birangi" that released in April 1983. Can't blame Google if it didn't index movie pages from 43 years ago. But the movie does have a Wikipedia page (check here) and the name does get referenced. Close on the heels of this instance, media accidentally found reference a social media post made by actor Rakesh Bedi, who plays a key character in Dhurandhar. In the video, he is seen referring to himself as Dhurandhar Bhatawdekar, which was part of his popular TV series called "Qubool Hai" from over a decade ago. Now this viral clip and the media's assumption that Bedi had used the word Dhurandhar 12 years ago, probably caused Google to fix things. Now, the same search string has the AI summary correctly laying out the sequence of events from Rang Birangi down to the recent social media buzz. What's more, if one were to search for the name plus Rakesh Bedi, the AI Summary only gives you details of the actor's role in the movie Dhurandhar (both parts), while totally skipping reference to his video or the TV series. What exactly does this latest study say? Coming back to the study, the sheer volume of search queries means that Google might have unintentionally created a misinformation crisis. And this is not good news as another study quoted by Inc.com last July noted that only about 8% of users actually double-check an answer provided by AI. In fact, there is an even more serious statistic that emerged from yet another AI experiment quoted byFuturism and conducted by researchers in the University of Pennsylvania. It said that more than 80% of the users tended to take the output of ChatGPT at face value even when it gave them the incorrect answer. The research team called it the "cognitive surrender." All of the above proves that while LLMs adopt an authoritative tone and presents fabricated information as facts when it cannot find straight answers, the challenge today is that most people are taking AI offerings as the truth. Which is where the NYT-led Oumi research comes in handy through a very simple test called SimpleQA. Used to benchmark AI accuracy by the industry and designed by OpenAI, the latest research conducted its first round of tests in October on AI Overviews powered by Gemini 2. A follow-up happened three months later where the feature was again tested post the switchover to Gemini-3, the upgraded version. Each tests involved over 4,300 Google searches and Gemini 3 emerged as the more accurate model of the two. While it proves beyond doubt that the models are improving, there is still some concern over how search engines could foist answers on users that is prone to hallucinations. Of course, Google said the study had major problems with spokesman Ned Adriance telling the NYT in a statement that "It doesn't reflect what people are actually searching on Google." Of course, it is another matter that Google itself had reported that its latest AI model produced incorrect information quite a few times. In recent times, the company has gone to town claiming that it's summary accuracy steps from the high quality search indexing that it has done for over two decades now, a distinct advantage Gemini has over other AI models. From a publisher's perspective, this is somewhat good news, given that the advent of AI summaries caused our revenues to fall. In fact, the correlation between such summaries and the decay of the PPC model of digital ads resulted in publishers joining hands to call for a "content independence day". During an earnings call last November, People Inc. (formerly known as Dotdash Meredith) told investors that Google Search, which had accounted for 54% of their traffic two years ago, is now contributing only 24% of the traffic - as evidenced in the numbers from the immediate past quarter of the current year. CEO Neil Vogel, a vociferous opponent of web crawlers that train AI models without paying the publishers, said at the earnings call that the company's decision to block AI crawlers was "very effective" as it "bought almost everyone to the table." He also revealed a new licensing deal that People Inc. had signed with Microsoft, it's second after the OpenAI agreement. In fact, one of our columnists at CXO Today wrote a detailed article on how such a move could sound the death knell for small publishers in India. "Blocking AI bots feels empowering, but in India's ad-dependent media economy, it may accelerate collapse," the article had said. For India's digital media sector, the ability to block AI crawlers is less a weapon and more a "strategic siege." It is a defensive posture that feels good but does little to alter the stark economic reality. While conglomerates can afford to wall themselves off and wait for a payout, the vast majority of Indian publishers are walking into a trap. By blocking the bots, they risk invisibility. By letting them in, they risk obsolescence. Unlike their Western counterparts, they lack the leverage to demand a third option, says the author Alok Gurtu, himself a 20-year veteran from the space and currently advising startups on growth strategies.

Google's AI Overviews spew out millions of false answers per hour, bombshell study reveals

Google's AI-generated search results are spewing out tens of millions of inaccurate answers per hour - even as the tech giant siphons visitors and ad revenue from cash-strapped news outlets, according to a bombshell analysis. To test the accuracy of Google's AI Overviews, startup Oumi reviewed 4,326 Google search results generated by Google's Gemini 2 model and the same number of results generated by its more advanced Gemini 3 model. The analysis found that the models were accurate 85% and 91% of the time, respectively. With Google expected to handle more than 5 trillion searchers in 2026 alone, that means AI Overviews are spitting out fake news at a rate of hundreds of thousands of mistakes every single minute - with users left none the wiser. The New York Times was first to report on Oumi's analysis. "Google AI Overviews have been a disaster for publishers who rely on clicks to fund the production of quality journalism, but they also let down users looking for accurate information," said Danielle Coffey, president and CEO of the News/Media Alliance, a trade group that represents more than 2,000 news outlets including The Post. The wrong answers included several basic fumbles, such as misstating the year in which musician Bob Marley's home was converted into a museum, misstating the year that former MLB relief pitcher Dick Drago died, and claiming there was no record of Yo-Yo Ma being inducted into the Classical Music Hall of Fame even though he was in 2007, according to examples Oumi provided to the Times. AI Overviews have appeared at the top of Google search results since 2024, while the traditional set of blue links to news outlets are effectively buried out of sight. Publishers have long accused Google, led by CEO Sundar Pichai, of ripping off their work to "train" its AI model without proper credit or compensation. "Algorithmically-generated responses that pull in data from nearly every source on the internet simply cannot be trusted," Coffey said. "Publishers spend enormous amounts of time and money ensuring that the content they deliver to their readers is properly fact-checked, while Google's AI Overviews are produced with no oversight or accountability." AI Overviews also has a penchant for citing information from questionable or easily edited sources, such as Facebook pages, blog posts and Wikipedia entries, as though it is fact. The feature appears easy to trick into spewing fake news. The Times cited an example in which BBC podcast host Thomas Germain wrote up a blog post proclaiming himself as one of "The Best Tech Journalists at Eating Hot Dogs." Google's AI summaries had gobbled up the information within a day and began claiming Germain had "gained notoriety for their prowess at the 'news division' of competitive eating events." Oumi's analysis was conducted between October and February and utilized a well-known benchmark test called SimpleQA, which was developed by OpenAI and is used to assess the accuracy of AI models. While the accuracy improved in the jump from Gemini 2 and Gemini 3, Oumi's research showed that AI Overviews has gotten worse about correctly citing where it found information. The percentage of AI Overviews answers that were "ungrounded," or where the links provided by Google did not back up the information included in the AI summary, jumped from 37% in Gemini 2 to 51% in Gemini 3, the report said. A Google spokesperson said Oumi's study has "serious holes" - in part because the SimpleQA benchmark test includes inaccurate information within its own dataset. The company also questioned Oumi's reliance on its own in-house AI model, dubbed HallOumi, to conduct the analysis, despite the risk that it could also make errors. "It uses one AI to grade another on an old benchmark that is known for being full of errors, and it doesn't reflect what people are actually searching on Google," the spokesperson said. "AI Overviews are built on our Gemini models, which lead the industry in accuracy, and they clear the same high-quality bar that we have for all our Search features." As The Post has reported, AI Overviews has struggled to provide accurate information since its launch, previously advising users to add glue to their pizza sauce and touting the "health benefits" of tobacco for kids.