Microsoft Unveils AI Tool to Combat Hallucinations in AI-Generated Text

Microsoft's new AI tool wants to find and fix AI-generated text that's factually wrong

Experts warn hallucinations are here to stay, but Microsoft has a fix Microsoft has unveiled a new tool which looks to stop AI models from generating content that is not factually correct - more commonly known as hallucinations. The new Correction feature builds on Microsoft's existing 'groundedness detection', which essentially cross-references AI text to a supporting document input by the user. The tool will be available as part of Microsoft's Azure AI Safety API and can be used with any text generating AI model, like OpenAI's GPT-4o and Meta's Llama. The service will flag anything that might be an error, then fact check this by comparing the text to a source of truth via a grounding document (i.e. uploaded transcripts). This means users can tell AI what to view as fact in the form of grounding documents. Experts warn that whilst the existing state of play may be useful, it doesn't address the cause of hallucinations. AI doesn't actually 'know' anything, it only predicts what comes next based on the examples it is trained on. "Empowering our customers to both understand and take action on ungrounded content and hallucinations is crucial, especially as the demand for reliability and accuracy in AI-generated content continues to rise," Microsoft noted in its blog post. "Building on our existing Groundedness Detection feature, this groundbreaking capability allows Azure AI Content Safety to both identify and correct hallucinations in real-time before users of generative AI applications encounter them." The launch, available in preview now, comes as part of wider Microsoft efforts to make AI more trustworthy. Generative AI has struggled so far to gain the trust of the public, with deepfakes and misinformation damaging its image, so updated efforts to make the service more secure will be welcomed. Also part of the updates is 'Evaluations', a proactive risk assessment tool, as well as confidential inference. This will ensure that sensitive information remains secure and private during the inference process - which is when the model makes decisions and predictions based on new data. Microsoft and other tech giants have invested heavily in AI technologies and infrastructure and are set to continue to do so, with a new $30 billion investment recently announced.

Microsoft launches Correction Feature that fixes AI Hallucinations

Microsoft has released a new feature called "Correction" that identifies hallucinations within Azure AI and corrects them. The company argues that this feature will allow AI application developers who use Azure to both understand and take action against hallucinations. Microsoft claims that the correction feature will allow generative AI app developers to create applications for high-stakes domains like medicine and can also allow enterprises to deploy conversational interfaces to their customers with confidence. This feature builds on top of the groundedness detection feature Microsoft launched in March, which allowed AI app developers to detect hallucinations. For context, in large language models (LLMs) grounding is the process of tracking a model's claims back to their original sources. Once a developer enables the correction feature, it will detect any hallucinations and send them to the underlying AI model for assessment. The model compares the sentence to a grounding document. If the sentence lacks relevant content, the system may remove it entirely. However, if the sentence contains information from the grounding document, the model rewrites it to align more closely with the source material, thus ensuring accuracy and relevance. Efforts to combat hallucinations, like what Microsoft is doing, become relevant when we consider the fact that multiple parties have called out Microsoft's partner company OpenAI's models for hallucinations. In June last year, a radio host in the US filed a defamation lawsuit against OpenAI for damaging his reputation after ChatGPT named him in an embezzlement case. Similarly, in April this year, a group of newspapers filed a case against Microsoft and OpenAI where they argued that OpenAI was causing them "commercial and competitive injury" by misattributing hallucinations to them. More recently, a Nieman Journalism Lab investigation found that OpenAI was hallucinating links to news articles by its partner news organisations such as The Associated Press, The Wall Street Journal, The Verge etc. However, the real utility of this hallucination detection/correction system will depend on how accurate it is. When an AI model responds to a prompt, something like "Better late that...", it just tries to find the probability of the next possible word in the sequence, MediaNama's Editor Nikhil Pahwa explained at a roundtable discussion earlier this year. He mentioned that models do not have fixed answers to prompts, instead they change their responses based on what the user wants. So if a user is dissatisfied with one output, the model will generate an alternate one. This poses a challenge because it implies that developers cannot accurately predict what sort of output a model will generate. Microsoft also observed the shortcomings of blocking outputs/filtering content in its blog post about the correction feature. It argued that filtering certain kinds of outputs can result in a poor user experience because it can render output texts incoherent since the model no longer has access to related contexts.

Microsoft Tries Fixing AI Hallucinations -- With More AI

Microsoft wants to use AI to fix AI -- but will it work? Microsoft has unveiled a new tool for AI app deployers that uses other AIs to evaluate and scan the first AI's responses. If the hallucination correction tool believes a reply might be invented, it's supposed to fix it with an answer that's backed up with a source. This secondary AI moderator, known as the Content Safety tool, is a part of Microsoft's Azure AI platform. In a video explaining the correction tool, Microsoft refers to answers that have been pulled from a source as "grounded" and invented or hallucinated answers as "ungrounded." "When the system detects ungrounded content or hallucinations, it highlights the inaccurate portions. Next, it explains why the section has been flagged as ungrounded. Then, it corrects the ungrounded content, aligning it with connected data sources," the video explains. This process happens as a secondary step on the backend before any AI answer is delivered to the person who typed in the query. But this means AI apps using this moderation system might be slower in delivering answers to users -- and even vetted answers may not be correct. The correction feature uses small language models and large language models to scan AI responses for hallucinations and find data online that can support an answer, a Microsoft rep told TechCrunch. This solution means multiple generative AI tools must work together on the backend, where one looks for unbacked claims while the other revises them with new text. But an AI-powered solution to AI problems may not work, considering the fundamental flaw of hallucination lies at the core of all generative AI models. It could also frustrate users further if the AI app is checked with this secondary system intended to improve user trust but still produces errors that bypass or are caused by this verification system. While Microsoft acknowledges that hallucinations erode user trust in generative AI, it presents Content Safety as a solution but paradoxically admits that the tool doesn't actually solve for accuracy. That's where the "groundedness" comes in because the AI can't check the source to confirm it's accurate (but will still cite it anyway). And these fact-checking AIs could hallucinate, too, TechCrunch notes. "It is important to note that groundedness detection does not solve for 'accuracy,' but helps to align generative AI outputs with grounding documents," Microsoft said.

Microsoft claims its new tool can correct AI hallucinations, but experts advise caution | TechCrunch

AI is a notorious liar, and Microsoft now says it has a fix for that. Understandably, that's going to raise some eyebrows, but there's reason to be skeptical. Microsoft today revealed Correction, a service that attempts to automatically revise AI-generated text that's factually wrong. Correction first flags text that may be erroneous -- say, a summary of a company's quarterly earnings call that may have misattributed quotes -- then fact-checks it by comparing the text with a source of truth (e.g., transcripts). Correction, available as part of Microsoft's Azure AI Content Safety API, can be used with any text-generating AI model, including Meta's Llama and OpenAI's GPT-4o. "Correction is powered by a new process of utilizing small language models and large language models to align outputs with grounding documents," a Microsoft spokesperson told TechCrunch. "We hope this new feature supports builders and users of generative AI in fields such as medicine, where application developers determine the accuracy of responses to be of significant importance." Google introduced a similar feature this summer in Vertex AI, its AI development platform, to let customers "ground" models by using data from third-party providers, their own datasets, or Google Search. But experts caution that these grounding approaches don't address the root cause of hallucinations. "Trying to eliminate hallucinations from generative AI is like trying to eliminate hydrogen from water," said Os Keyes, a Ph.D. candidate at the University of Washington who studies the ethical impact of emerging tech. "It's an essential component of how the technology works." Text-generating models hallucinate because they don't actually "know" anything. They're statistical systems that identify patterns in a series of words and predict which words come next based on the countless examples they are trained on. It follows that a model's responses aren't answers, but merely predictions of how a question would be answered were it present in the training set. As a consequence, models tend to play fast and loose with the truth. One study found that OpenAI's ChatGPT gets medical questions wrong half the time. Microsoft's solution is a pair of cross-referencing, copy-editor-esque meta models designed to highlight and rewrite hallucinations. A classifier model looks for possibly incorrect, fabricated or irrelevant snippets of AI-generated text (hallucinations). If it detects hallucinations, the classifier ropes in a second model, a language model, that tries to correct for the hallucinations in accordance with specified "grounding documents." "Correction can significantly enhance the reliability and trustworthiness of AI-generated content by helping application developers reduce user dissatisfaction and potential reputational risks," the Microsoft spokesperson said. "It is important to note that groundedness detection does not solve for 'accuracy,' but helps to align generative AI outputs with grounding documents." Keyes has doubts about this. "It might reduce some problems," they said, "But it's also going to generate new ones. After all, Correction's hallucination detection library is also presumably capable of hallucinating." Asked for a backgrounder on the Correction models, the spokesperson pointed to a recent paper from a Microsoft research team describing the models' pre-production architectures. But the paper omits key details, like which data sets were used to train the models. Mike Cook, a research fellow at Queen Mary University specializing in AI, argued that even if Correction works as advertised, it threatens to compound the trust and explainability issues around AI. The service might catch some errors, but it could also lull users into a false sense of security -- into thinking models are being truthful more often than is actually the case. "Microsoft, like OpenAI and Google, have created this issue where models are being relied upon in scenarios where they are frequently wrong," he said. "What Microsoft is doing now is repeating the mistake at a higher level. Let's say this takes us from 90% safety to 99% safety -- the issue was never really in that 9%. It's always going to be in the 1% of mistakes we're not yet detecting." Cook added that there's also a cynical business angle to how Microsoft is bundling Correction. The feature is free on its own, but the "groundedness detection" required to detect hallucinations for Correction to revise is only free up to 5,000 "text records" per month. It costs 38 cents per 1,000 text records after that. Microsoft is certainly under pressure to prove to customers -- and shareholders -- that its AI is worth the investment. In Q2 alone, the tech giant ploughed nearly $19 billion in capital expenditures and equipment mostly related to AI. Yet, the company has yet to see significant revenue from AI. A Wall Street analyst this week downgraded the company's stock, citing doubts about its long-term AI strategy. According to a piece in The Information, many early adopters have paused deployments of Microsoft's flagship generative AI platform, Microsoft 365 Copilot, due to performance and cost concerns. For one client using Copilot for Microsoft Teams meetings, the AI reportedly invented attendees and implied that calls were about subjects that were never actually discussed. Accuracy, and the potential for hallucinations, are now among businesses' biggest concerns when piloting AI tools, according to a KPMG poll. "If this were a normal product lifecycle, generative AI would still be in academic R&D, and being worked on to improve it and understand its strengths and weaknesses," Cook said. "Instead, we've deployed it into a dozen industries. Microsoft and others have loaded everyone onto their exciting new rocket ship, and are deciding to build the landing gear and the parachutes while on the way to their destination."

Microsoft claims its AI safety tool not only finds errors but also fixes them

Microsoft is launching a new feature called "correction" that builds on the company's efforts to combat AI inaccuracies. Customers using Microsoft Azure to power their AI systems can now use the capability to automatically detect and rewrite incorrect content in AI outputs. The correction feature is available in preview as part of the Azure AI Studio -- a suite of safety tools designed to detect vulnerabilities, find "hallucinations," and block malicious prompts. Once enabled, the correction system will scan and identify inaccuracies in AI output by comparing it with a customer's source material. From there, it will highlight the mistake, provide information about why it's incorrect, and rewrite the content in question -- all "before the user is able to see" the inaccuracy. While this seems like a helpful way to address the nonsense often espoused by AI models, it might not be a fully reliable solution. Vertex AI, Google's cloud platform for companies developing AI systems, has a feature that "grounds" AI models by checking outputs against Google Search, a company's own data, and (soon) third-party datasets. In a statement to TechCrunch, a Microsoft spokesperson said the "correction" system uses "small language models and large language models to align outputs with grounding documents," which means it isn't immune to making errors, either. "It is important to note that groundedness detection does not solve for 'accuracy,' but helps to align generative AI outputs with grounding documents," Microsoft told TechCrunch.