Google Cloud Enhances Database and Analytics Tools with Generative AI Capabilities

Google Cloud expands gen AI power for database and data analytics tools

Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More Google Cloud is expanding the capabilities of its database and data analytics offerings with a series of updates announced today at the Google Cloud Next event in Tokyo. The announcements span across multiple services including the Spanner and Bigtable databases as well as the BigQuery data analytics and Looker business intelligence platforms. The overall goal is to integrate more flexibility into how data can be used and accessed, in an effort to help further accelerate generative AI deployments and adoption. Key announcements and update from Google include: "Organizations recognize that in order to get to incredible AI, they need to have incredible data," Gerrit Kazmaier, GM & VP of Data Analytics at Google Cloud said during a briefing with press and analysts. Google's data analytics platforms get a new 'look' with gen AI For data analytics, the big news is that Google's Gemini AI capabilities are now available in BigQuery and Looker. The integration of Gemini provides a long list of over 20 new features including code generation, explanation and intelligent recommenders that will help data analysts be more productive. Inside of BigQuery, Gemini will now also help to power advanced data preparation and analysis to accelerate time to value from data. "Data is messy," Kazmaier said. "One of the great benefits that we saw in building our specialized gen AI models is for actually reasoning about data and helping our customers to align and govern data much quicker." AI will also help to inform the new Data Canvas feature which Katzmaier described as, "...the perfect synergy between user experience AI and a data analyst." The key advantage of Data Canvas lies in its interactive and AI-assisted approach. It creates a self-reinforcing dynamic where users incrementally build their analysis path, and the system learns from this process. For Looker the AI updates have a focus on helping to make it easier to get at business intelligence insights. "We have focused our innovation on Looker on building customized agents who are really deep AI experts, which know how to select data, perform analysis and summarize it," Katzmaier said. Spanner database become even more multi-modal with vector and graph Though the Google Spanner database might not be familiar to everyone, it is in fact a technology that is used by almost everyone that uses Google. "Spanner is powering most of Google's if not all of Google's user products, whether that is Search, Gmail, YouTube and we had to build Spanner to really meet the level of scalability and availability that Google needed," Andi Gutmans said. "One of the exciting things about my job is I get the opportunity to externalize that innovation to our enterprise customers." One of the new innovations that Google is bringing to its enterprise customers is Graph database capabilities for Spanner. Graph provides a different way of making connections across data that can enable nuanced semantic relationships. Not only is Spanner getting graph support, it's also finally getting vector support as well. Google had previously announced a preview of vector support in Spanner back in February. Both vector and graph are useful at helping to enable gen AI applications. Vector in particular is commonly associated with Retrieval Augmented Generation (RAG). While there are many purpose-built native graph and vector databases in the market, Google's approach is to provide a multi-modal database. "It's not that customers have to move their data to get graph capabilities. they can take their enterprise data and start to build the graph capabilities on top of that," Gutmans said. The basic idea is that organizations are already relying on Spanner and trust it. The addition of graph and vector enable those organizations to extract even more utility from that data. "We've expanded Spanner now, from being primarily a relational database to really being a true multi-modal database," Gutmans said.

Google Cloud announces new data innovations to support AI applications - SiliconANGLE

Google Cloud announces new data innovations to support AI applications Google LLC's cloud division is rolling out new database and data cloud innovations to support customers building and scaling artificial intelligence applications to ensure that they are grounded in accurate and relevant enterprise information. Announced today at Google Cloud Next in Tokyo were several new capabilities for Spanner, Google Cloud's distributed SQL relational database management and storage service, that will make it easier for customers to build and deploy AI apps fueled by data from relationship graph networks, vector search and advanced full-text search. "Over the past year, we have been focused on helping developers build enterprise gen AI applications by providing industry-leading vector support and strong integration with Vertex AI and open-source LangChain," said Andi Gutmans, general manager and vice president of engineering and databases at Google Cloud. "But we've also heard from customers that in order to build intelligent AI applications, they want to reason about knowledge -- not just the data itself but how the data is interconnected." The newly announced capability, Spanner Graph, expands Spanner's ability to include graph processing to include industry-standard graph processing language, which allows for searching relationships between structured and unstructured data in a single query. Gutmans said that will allow developers to build AI applications based on graph-based retrieval-augmented generation, and implement smarter recommendation engines and financial services can serve fraud detection. GraphRAG can be used to improve the accuracy of AI applications by providing more contextually relevant answers to user queries using trusted enterprise real-time data sources. Spanner is also being upgraded with full-text search and vector search capabilities at scale. Developers can access both vector and full-text in a single query, receiving the power of both keyword search and context-aware semantic search from vector at the same time. "With Spanner Graph, full-text search and vector search, we have evolved Spanner from not only being the most available, globally consistent and scalable database, to a multi-model database with intelligent capabilities that seamlessly interoperate to enable you to deliver a new class of AI-enabled applications," Gutmans said. Bigtable, Google's high-performance NoSQL database service that can store large amounts of data in wide tables with thousands of columns and billions of rows, is receiving SQL query support. Now developers can use more than 100 SQL functions directly into Bigtable. Google also recently introduced Bigtable distributed counters that will enable developers to rapidly prototype and deploy real-time applications with real-time embedded analytics. Distributed counters are data types optimized for high throughput writes for processing high-speed events that can support AI and fast transactions at scale. To assist organizations in handling data, which is the lifeblood of AI applications, Google Cloud is rolling out data analytics products and AI data platform capabilities into general availability to support its customers. It begins with Gemini in BigQuery, which provides the assistance of Google's most powerful large language model for data engineering, data exploration and analysis, governance and security tasks. This adds new features such as code generation, completion and expiation of SQL and Python. "Google Cloud continues to strengthen its AI-ready data ecosystem," said Doug Henschen, vice president and principal analyst at Constellation Research Inc. "Gemini integration is an example of the gen AI augmentation we're seeing that will drive innovation and enhance use cases for data teams and information workers. Platform unification, like the innovations we're seeing with BigQuery, will make things simpler and easier for customers looking at data platform migrations." With access to Gemini in BigQuery, data engineers will be able to have the AI assist them with data preparation, cleansing, analysis and the entire data journey. It can also provide intelligent recommendations to enhance productivity and optimize costs. Gemini in Looker, now in preview for Google's business intelligence tool that will provide AI-powered assistance for building formulas, help explore data and create metrics from complex information and generate slides and presentations on the fly with conversational prompts. That means business users will be able to create calculation fields without having to remember complicated formulas, making their lives easier.

Google Cloud expands its database portfolio with new AI capabilities | TechCrunch

Google is hosting a version of its Cloud Next conference in Tokyo this week, and it's putting the focus squarely on tweaking its databases for AI workloads (because at this point in 2024, AI is the only thing these major tech companies want to talk about). These include updates to its Spanner SQL database, which now features graph and vector search support, as well as extended full-text search capabilities. This wouldn't be a Google announcement without some Gemini-powered features. These include Gemini in BigQuery and Looker to help users with data engineering and analysis, as well as governance and security tasks. Google argues that while the vast majority of enterprises think that generative AI will be critical to the success of their business, they also know that much of their data remains unmanaged, leaving it outside of the scope of their analytics and AI initiatives. "They have to really get out of all of their existing data silos and data islands, and get to a consolidated multimodal data platform, spanning structured and unstructured data -- [because] GenAI is terrific at analyzing unstructured data -- and combining data at rest with their data movement, so real-time data and data at rest processing," explained Gerrit Kazmeier, Google's VP and GM for database, Data Analytics and Looker. Activating this enterprise data flow, he argued, is what a lot of these new features are all about. Spanner powers most of Google's own products like Search, Gmail and YouTube and its customer list includes the likes of Home Depot, Uber, Walmart and others. And while Spanner can handle a massive volume of data, vector and graph databases are a necessity to bring enterprise data into GenAI applications and enrich existing foundation models. "What we're thinking about is what would it really take for us to take Spanner's availability, scale, relational model, and really expand that to be the best data platform for operational GenAI apps," said Andi Gutmans, Google's VP and GM for databases. Like so many database vendors, the first step here for Google is adding graph capabilities to Spanner, using the emerging GraphQL standard. Enterprises can then use this graph to augment their GenAI applications -- and the foundation models that power them -- using Retrieval Augmented Generation (RAG), which is currently the de facto standard for this use case. Also new in Spanner are full-text search and vector search, with the vector search capabilities backed by Google's ScaNN algorithm. "With Spanner Graph, full-text search and vector search, we have evolved Spanner from not only being the most available, globally consistent and scalable database, to a multi-model database with intelligent capabilities that seamlessly interoperate to enable you to deliver a new class of AI-enabled applications," Google says. In addition to these AI-centric updates, Spanner is getting a new, optional pricing structure. Dubbed "Spanner editions," the idea here is to offer a tier-based pricing model that offers them more flexibility. Currently, Google Cloud customers had to choose between a single-region offering and a multi-region version, which also offered a bundle of additional features like replication. Google also on Thursday announced a major update to Bigtable, Google's NoSQL database for unstructured data and latency-sensitive workloads. Bigtable now features SQL support (or more precisely, support for GoogleSQL, the company's own SQL dialect), making it significantly easier for virtually any developer to use the service. Previously, developers had to use the Bigtable API to query their databases. Currently, Bigtable supports roughly 100 SQL functions. For the Oracle database fans out there, Google will now allow them to host their Oracle Exadata and Autonomous database services right in the Google Cloud data centers -- and they can link their applications between Google Cloud and the Oracle Cloud. For Google, that means more workloads in its cloud and for Oracle, at least, it means these users are still paying their licensing fees, even if they aren't using the Oracle cloud. Also new in Google Cloud is support for open-source Apache Spark and Kafka for data streaming and processing, as well as real-time streaming from Analytics Hub (Google's service for securely sharing data between organizations).

Google introduces Bigtable SQL access and Spanner's new AI-ready features

On Thursday, Google announced a whole series of database and data analytics improvements to its cloud data architecture. In this article, we'll focus on the substantial improvements to Spanner and Bigtable (two of Google's cloud database offerings). These announcements substantially increase interoperability and open the door to additional AI implementations through the use of new features Google is showcasing. Also: Cost of data breach climbs 10%, but AI helping to limit some damage Spanner is Google's global cloud database. It excels in providing worldwide consistency (which is way harder to implement than it may seem) due to a plethora of time-related issues that Google has solved. It's also scalable, meaning the database can grow big and span countries and regions. It's multi-modal, meaning it supports media data and not just text. It's also all managed through SQL (Structured Query Language) queries. Bigtable is also hugely scalable (hence the "big" in Bigtable). Its focus is very wide columns that can be added on the fly and don't need to be uniformly defined across all rows. It also has very low latency and high throughput. Until now, it's been characterized as a NoSQL database, a term used to describe non-relational databases that allow for flexible schemas and data organization. Both of these tools provide support for giant enterprise databases. Spanner is generally a better choice for applications using a globally distributed database that requires robust and immediate consistency and complex transactions. Bigtable is better if high throughput is important. Bigtable has a form of consistency, but propagation delays mean that data will not immediately, but eventually, be consistent. Bigtable is primarily queried through API calls. One of the biggest and most game-changing features announced today is SQL queries for Bigtable. This is huge from a programming skills point of view. In a 2023 Stack Overflow survey of programming language use, SQL ranked fourth, with 48.66% of programmers using it. There was no mention of Bigtable in the Stack Overflow survey, so I turned to LinkedIn for some perspective. A quick search of jobs containing "SQL" resulted in 400,000+ results. Meanwhile, a search for "Bigtable" resulted in 1,561 results, less than 1% of the SQL number. Also: Google upgrades Search to combat deepfakes and demote sites posting them So, while any number of folks who know SQL could have learned how to make Bigtable API calls, SQL means that the learning curve has been flattened to nearly zero. Almost one out of every two developers can now use the new SQL interface to Bigtable to write queries whenever they need to. One note, though: this Bigtable upgrade doesn't support all of SQL. Google has, however, implemented more than 100 functions and promises more to come. Also on the Bigtable table is the introduction of distributed counters. Counters are features like sum, average, and other related math functions. Google is introducing the ability to get these data aggregations in real-time with a very high level of throughput and across multiple nodes in a Bigtable cluster, which lets them perform analysis and aggregation functions concurrently across sources. This lets you do things like calculate daily engagement, find max and minimum values from sensor readings, and so on. With Bigtable, you can deploy these on very large-scale projects that need rapid, real-time insights and that can't support bottlenecks normally coming from aggregating per node and then aggregating the nodes. It's big numbers, fast. Google has a number of big Spanner announcements that all move the database tool towards providing support for AI projects. The big one is the introduction of Spanner Graph, which adds graph database capabilities to the global distributed database functionality at the core of Spanner. Don't confuse "graph database" with "graphics." The term means the nodes and connections of the database can be illustrated as a graph. If you've ever heard the term "social graph" in reference to Facebook, you know what a graph database is. Think of the nodes as entities, like people, places, items, etc., and the connections (also called edges) as the relationships between the entities. Facebook's social graph of you, for example, contains all the people you have relationships with, and then all the people they have relationships with, and so on and so on. Spanner can now natively store and manage this type of data, which is big news for AI implementations. This gives AI implementations a global, highly consistent, region-free way to represent vast relationship information. This is powerful for traversal (finding a path or exploring a network), pattern matching (identifying groups that match a certain pattern), centrality analysis (determining which nodes are more important than the other nodes), and community detection (finding clusters of nodes that comprise a cluster of some sort, like a neighborhood). Also: OpenAI rolls out highly anticipated advanced Voice Mode, but there's a catch Along with the graph data representation, Spanner now supports GQL (Graph Query Language), an industry-standard language for performing powerful queries in graphs. It also works with SQL, which means that developers can use both SQL and GQL within the same query. This can be a big deal for applications that need to sift through row-and-column data and discern relationships in the same query. Google is also introducing two new search modalities to Spanner: full-text and vector. Full-text is something most folks are familiar with -- the ability to search within text like articles and documents for a given pattern. Vector search turns words (or even entire documents) into numbers that are mathematical representations of the data. These are called "vectors," and they essentially capture the intent, meaning, or essence of the original text. Queries are also turned into vectors (numerical representations), so when an application performs a lookup, it looks for other vectors that are mathematically close to each other -- essentially computing similarity. Vectors can be very powerful because matches no longer need to be exact. For example, an application querying "detective fiction" would know to search for "mystery novels," "home insurance" would also work for "property coverage," and "table lamps" would also work for "desk lighting." You can see how that sort of similarity matching would be beneficial for AI analysis. In Spanner's case, those similarity matches could work on data that's stored in different continents or server racks. According to Google's Data and AI Trends Report 2024, 52% of the non-technical users surveyed are already using generative AI to provide data insights. Almost two-thirds of the respondents believe that AI will cause a "democratization of access to insights," essentially allowing non-programmers to ask new questions about their data without requiring a programmer to build it into code. 84% believe that generative AI will provide those insights faster. I agree. I'm a technical user, but when I fed ChatGPT some raw data from my server, and the result was some powerfully helpful business analytics in minutes, without needing to write a line of code, I realized AI was a game-changer for my business. Also: The moment I realized ChatGPT Plus was a game-changer for my business Here's the problem. According to the survey, 66% of respondents report that at least half of their data is dark. What that means is that the data is there, somewhere, but not accessible for analysis. Some of that has to do with data governance issues, some has to do with the data format or a lack thereof, some of it has to do with the fact that the data can't be represented in rows and columns, and some of it has to do with a myriad of other issues. Essentially, even though AI systems may "democratize" access to data insights, that's only possible if the AI systems can get at the data. That brings us to the relevance of today's Google announcements. These features all increase the access to data, whether because of a new query mechanism, due to the ability of programmers to use existing skills like SQL, the ability of large databases to represent data relationships in new ways, or the ability of search queries to find similar data. They all open up what may have been previously dark data to analysis and insights.