ChatGPT's GPT-5.2 model caught citing Grokipedia, sparking fears about AI misinformation loops

ChatGPT found to be sourcing data from AI-generated content -- popular LLM uses content from Grokipedia as source for more obscure queries

ChatGPT's latest model, GPT-5.2, has been found to be sourcing data from Grokipedia, xAI's all-AI-generated Wikipedia competitor. According to The Guardian, the AI LLM would sometimes use Elon Musk's AI-generated online encyclopedia for uncommon topics like Iranian politics, and details about British historian Sir Richard Evans. Issues like this were raised as problematic a few years ago in AI training, where some experts argued that training AI on AI-generated data would degrade quality and lead to a phenomenon called "model collapse." And while citing AI-generated data is different from using it for training, it still poses risks to people relying on AI for research. The biggest issue with this is that AI models are known to hallucinate or make up information that is wrong. For example, Anthropic attempted to run a business with its 'Claudius' AI -- it hallucinated several times during the experiment, with the AI even saying that it would hand-deliver drinks, in person. Even Nvidia CEO Jensen Huang admitted in 2024 that solving this issue is still "several years away" and requires a lot more computing power. Furthermore, many users trust that ChatGPT and other LLMs deliver accurate information, with only a few checking the actual sources used to answer a particular question. Because of this, ChatGPT repeating Grok's words can be problematic, especially as Grokipedia isn't edited directly by humans. Instead, it's completely AI-generated and people can only request changes to its content -- not write or edit the articles directly. Using another AI as a source creates a recursive loop, and we might eventually end up with LLMs citing content, which haven't been verified, from each other. This is no different from rumors and stories spreading between humans, with "someone else said it" being the source. This results in the illusory truth effect, where false information is deemed correct by many, despite having data saying otherwise, because it's been repeated by so many people. Human society was littered with myths and legends similarly, passed over hundreds of years through several generations. However, with AI churning through tons of data at infinitely faster speeds than humans, the use of AI sources risks the proliferation of digital folklore with every query entered into AI LLMs. What's more troubling is that various parties are already taking advantage of this. There have been reports of "LLM grooming," with The Guardian saying that some propaganda networks are "churning out massive volumes of disinformation in an effort to seed AI models with lies." This has raised concerns in the U.S., with Google's Gemini, for example, reportedly repeating the official party line of the Communist Party of China in 2024. This seems to have been addressed at the moment, but if LLMs start citing other AI-generated sources that haven't been vetted and fact-checked, then this is a new risk that people need to look out for. Follow Tom's Hardware on Google News, or add us as a preferred source, to get our latest news, analysis, & reviews in your feeds.

Latest ChatGPT model uses Elon Musk's Grokipedia as source, tests reveal

Guardian found OpenAI's platform cited Grokipedia on topics including Iran and Holocaust deniers The latest model of ChatGPT has begun to cite Elon Musk's Grokipedia as a source on a wide range of queries, including on Iranian conglomerates and Holocaust deniers, raising concerns about misinformation on the platform. In tests done by the Guardian, GPT-5.2 cited Grokipedia nine times in response to more than a dozen different questions. These included queries on political structures in Iran, such as salaries of the Basij paramilitary force and the ownership of the Mostazafan Foundation, and questions on the biography of Sir Richard Evans, a British historian and expert witness against Holocaust denier David Irving in his libel trial. Grokipedia, launched in October, is an AI-generated online encyclopedia that aims to compete with Wikipedia, and which has been criticised for propagating rightwing narratives on topics including gay marriage and the 6 January insurrection in the US. Unlike Wikipedia, it does not allow direct human editing, instead an AI model writes content and responds to requests for changes. ChatGPT did not cite Grokipedia when prompted directly to repeat misinformation about the insurrection, about media bias against Donald Trump, or about the HIV/Aids epidemic - areas where Grokipedia has been widely reported to promote falsehoods. Instead, Grokipedia's information filtered into the model's responses when it was prompted about more obscure topics. For instance, ChatGPT, citing Grokipedia, repeated stronger claims about the Iranian government's links to MTN-Irancell than are found on Wikipedia - such as asserting that the company has links to the office of Iran's supreme leader. ChatGPT also cited Grokipedia when repeating information that the Guardian has debunked, namely details about Sir Richard Evans' work as an expert witness in David Irving's trial. GPT-5.2 is not the only large language model (LLM) that appears to be citing Grokipedia; anecdotally, Anthropic's Claude has also referenced Musk's encyclopedia on topics from petroleum production to Scottish ales. An OpenAI spokesperson said the model's web search "aims to draw from a broad range of publicly available sources and viewpoints". "We apply safety filters to reduce the risk of surfacing links associated with high-severity harms, and ChatGPT clearly shows which sources informed a response through citations," they said, adding that they had ongoing programs to filter out low-credibility information and influence campaigns. Anthropic did not respond to a request for comment. But the fact that Grokipedia's information is filtering - at times very subtly - into LLM responses is a concern for disinformation researchers. Last spring, security experts raised concerns that malign actors, including Russian propaganda networks, were churning out massive volumes of disinformation in an effort to seed AI models with lies, a process called "LLM grooming". In June, concerns were raised in the US Congress that Google's Gemini repeated the Chinese government's position on human rights abuses in Xinjiang and China's Covid-19 policies. Nina Jankowicz, a disinformation researcher who has worked on LLM grooming, said ChatGPT's citing Grokipedia raised similar concerns. While Musk may not have intended to influence LLMs, Grokipedia entries she and colleagues had reviewed were "relying on sources that are untrustworthy at best, poorly sourced and deliberate disinformation at worst", she said. And the fact that LLMs cite sources such as Grokipedia or the Pravda network may, in turn, improve these sources' credibility in the eyes of readers. "They might say, 'oh, ChatGPT is citing it, these models are citing it, it must be a decent source, surely they've vetted it' - and they might go there and look for news about Ukraine," said Jankowicz. Bad information, once it has filtered into an AI chatbot, can be challenging to remove. Jankowicz recently found that a large news outlet had included a made-up quote from her in a story about disinformation. She wrote to the news outlet asking for the quote to be removed, and posted about the incident on social media. The news outlet removed the quote. However, AI models for some time continued to cite it as hers. "Most people won't do the work necessary to figure out where the truth actually lies," she said. When asked for comment, a spokesperson for xAI, the owner of Grokipedia, said: "Legacy media lies."

ChatGPT's new model is citing an AI-written encyclopedia, raising misinformation fears

ChatGPT cited Grokipedia multiple times while responding to a small set of questions, particularly on niche or complex topics. Amid the rising competition in the AI space, the latest ChatGPT model is facing scrutiny after researchers noticed it citing an unusual and controversial source for factual information. As per the reports, the test suggested that GPT-5.2 has begun referencing Grokipedia, an AI-generated online encyclopedia associated with Elon Musk. As per testing conducted by The Guardian, ChatGPT cited Grokipedia multiple times while responding to a small set of questions, particularly on niche or complex topics. These topics included explanations of Iran's political system, such as the role of the Basij paramilitary force and the control of influential foundations, as well as background details about British historian Sir Richard Evans. Also read: Apple may introduce AI powered Siri with iOS 26.4 next month: Eligible devices, features and more For the unversed, Grokipedia was introduced in October last year and positions itself as the biggest rival to Wikipedia. However, it operates on a very different model. Its entries are written and updated by an AI system rather than human editors, with users submitting requests instead of directly editing pages. Since its debut, the platform has faced criticism for allegedly promoting right-leaning narratives on issues such as same-sex marriage and the January 6 US Capitol attack. What caught researchers attention was the pattern of Grokipedia's appearance in ChatGPT responses. When the chatbot was asked about widely known misinformation topics, it tended to avoid the source. However, Grokipedia surfaced when queries involved less familiar or more technical subjects. In some cases, ChatGPT echoed claims found on Grokipedia that were stronger or more controversial than those presented on Wikipedia, including assertions that have previously been challenged or debunked. Also read: Google Pixel 10 price drops by Rs 12,000 on this platform: Check deal details here The report also added that the issue was not limited to ChatGPT. It also claims that other AI chatbots like Anthropic's Claude chatbot have also referenced Grokipedia while answering questions on topics ranging from global oil production to regional food and drink. ChatGPT maker has confirmed that the chatbot draws from a broad mix of publicly available sources and applies safety measures to limit harmful or misleading content, while aiming to clearly attribute information.