Google's DataGemma: Pioneering Large-Scale AI with RAG to Combat Hallucinations

Google's DataGemma is the first large-scale Gen AI with RAG - why it matters

The increasingly popular generative artificial intelligence technique known as retrieval-augmented generation -- or RAG, for short -- has been a pet project of enterprises, but now it's coming to the AI main stage. Google last week unveiled DataGemma, which is a combination of Google's Gemma open-source large language models (LLMs) and its Data Commons project for publicly available data. DataGemma uses RAG approaches to fetch the data before giving an answer to a query prompt. The premise is to ground generative AI, to prevent "hallucinations," says Google, "by harnessing the knowledge of Data Commons to enhance LLM factuality and reasoning." Also: What are o1 and o1-mini? OpenAI's mystery AI models are finally here While RAG is becoming a popular approach for enabling enterprises to ground LLMs in their proprietary corporate data, using Data Commons represents the first implementation to date of RAG at the scale of cloud-based Gen AI. Data Commons is an open-source development framework that lets one build publicly available databases. It also gathers actual data from institutions such as the United Nations that have made their data available to the public. In connecting the two, Google notes, it is taking "two distinct approaches." The first approach is to use the publicly available statistical data of Data Commons to fact-check specific questions entered into the prompt, such as, "Has the use of renewables increased in the world?" Google's Gemma will respond to the prompt with an assertion that cites particular stats. Google refers to this as "retrieval-interleaved generation," or RIG. In the second approach, full-on RAG is used to cite sources of the data, "and enable more comprehensive and informative outputs," states Google. The Gemma AI model draws upon the "long-context window" of Google's closed-source model, Gemini 1.5. Context window represents the amount of input in tokens -- usually words -- that the AI model can store in temporary memory to act on. Also: Understanding RAG: How to integrate generative AI LLMs with your business knowledge Gemini advertises Gemini 1.5 at a context window of 128,000 tokens, though versions of it can juggle as much as a million tokens from input. Having a larger context window means that more data retrieved from Data Commons can be held in memory and perused by the model when preparing a response to the query prompt. "DataGemma retrieves relevant contextual information from Data Commons before the model initiates response generation," states Google, "thereby minimizing the risk of hallucinations and enhancing the accuracy of responses." The research is still in development; you can dig into the details in the formal research paper by Google researcher Prashanth Radhakrishnan and colleagues. Google says there's more testing and development to be done before DataGemma is made available publicly in Gemma and Google's closed-source model, Gemini. Already, claims Google, the RIG and RAG have lead to improvements in quality of output such that "users will experience fewer hallucinations for use cases across research, decision-making or simply satisfying curiosity." Also: First Gemini, now Gemma: Google's new, open AI models target developers DataGemma is the latest example of how Google and other dominant AI firms are building out their offerings with things that go beyond LLMs. OpenAI last week unveiled its project internally code-named "Strawberry" as two models that use a machine learning technique called "chain of thought," where the AI model is directed to spell out in statements the factors that go into a particular prediction it is making.

How Google's DataGemma uses RAG to combat AI hallucinations

Google has taken another significant step forward in the race to improve the accuracy and reliability of AI models with the introduction of DataGemma, an innovative approach that combines its Gemma large language models (LLMs) and the Data Commons project. The spotlight here is on a technique called retrieval-augmented generation (RAG), a method that has been gaining traction in enterprises, but now, with DataGemma, Google aims to bring it into the AI mainstream. At its core, RAG seeks to solve one of the biggest challenges faced by LLMs: the problem of hallucinations. In the world of generative AI, hallucinations refer to instances where the model generates information that sounds plausible but is factually incorrect. This is a common issue in AI systems, especially when they lack reliable grounding in factual data. Google's goal with DataGemma is to "harness the knowledge of Data Commons to enhance LLM factuality and reasoning," addressing this issue head-on. Retrieval-augmented generation is a game changer because it doesn't rely solely on pre-trained AI models to generate answers. Instead, it retrieves relevant data from an external source before generating a response. This approach allows AI to provide more accurate and contextually relevant answers by pulling real-world data from repositories. In the case of DataGemma, the source of this data is Google's Data Commons project, a publicly available resource that aggregates statistical data from reputable institutions like the United Nations. This move by Google to integrate Data Commons with its generative AI models represents the first large-scale cloud-based implementation of RAG. While many enterprises have used RAG to ground their AI models in proprietary data, using a public data resource like Data Commons takes things to a whole new level. It signals Google's intention to use verifiable, high-quality data to make AI more reliable and useful across a broad range of applications. According to Google, DataGemma takes "two distinct approaches" to integrate data retrieval with LLM output. The first method is called retrieval-interleaved generation (RIG). With RIG, the AI fetches specific statistical data to fact-check questions posed in the query prompt. For example, if a user asks, "Has the use of renewables increased in the world?" the system can pull in up-to-date statistics from Data Commons and cite them directly in its response. This not only improves the factual accuracy of the answer but also provides users with concrete sources for the information. The second method is more in line with the traditional RAG approach. Here, the model retrieves data to generate more comprehensive and detailed responses, citing the sources of the data to create a fuller picture. "DataGemma retrieves relevant contextual information from Data Commons before the model initiates response generation," Google states. This ensures that the AI has all the necessary facts at hand before it begins generating an answer, greatly reducing the likelihood of hallucinations. A key feature of DataGemma is the use of Google's Gemini 1.5 model, which boasts an impressive context window of up to 128,000 tokens. In AI terms, the context window refers to how much information the model can hold in memory while processing a query. The larger the window, the more data the model can take into account when generating a response. Gemini 1.5 can even scale up to a staggering 1 million tokens, allowing it to pull in massive amounts of data from Data Commons and use it to craft detailed, nuanced responses. This extended context window is critical because it allows DataGemma to "minimize the risk of hallucinations and enhance the accuracy of responses," according to Google. By holding more relevant information in memory, the model can cross-check its own output with real-world data, ensuring that the answers it provides are not only relevant but also factually grounded. While the integration of RAG techniques is exciting on its own, DataGemma also represents a broader shift in the AI landscape. It's no longer just about large language models generating text or answering questions based on what they've been trained on. The future of AI lies in its ability to integrate with real-time data sources, ensuring that its outputs are as accurate and up-to-date as possible. Google is not alone in this pursuit. Just last week, OpenAI unveiled its "Strawberry" project, which takes a different approach to improving AI reasoning. Strawberry uses a method known as "chain of thought", where the AI spells out the steps or factors it uses to arrive at a prediction or conclusion. While different from RAG, the goal is similar: make AI more transparent, reliable, and useful by providing insights into the reasoning behind its answers. For now, DataGemma remains a work in progress. Google acknowledges that more testing and development are needed before the system can be made widely available to the public. However, early results are promising. Google claims that both the RIG and RAG approaches have led to improvements in output quality, with "fewer hallucinations for use cases across research, decision-making, or simply satisfying curiosity." It's clear that Google, along with other leading AI companies, is moving beyond the basic capabilities of large language models. The future of AI lies in its ability to integrate with external data sources, whether they be public databases like Data Commons or proprietary corporate data. By doing so, AI can move beyond its limitations and become a more powerful tool for decision-making, research, and exploration.