VAST Data Launches Cosmos: A New AI Community to Accelerate Enterprise AI Adoption

Enterprise AI infrastructure and what's next for Vast - SiliconANGLE

Three insights you might have missed from the Vast Data "Enter the Cosmos" event Enterprise AI infrastructure faces unprecedented demands today. As AI-powered applications scale, the need for seamless data orchestration across hybrid environments is becoming critical. For Vast Data Inc., the goal has been to operationalize the entire AI pipeline. The company has emerged as a central player as companies seek infrastructure that can rapidly optimize data flows and quickly scale resources to support next-generation enterprise workloads. The company's new developments in enterprise AI infrastructure were a central focus of the Vast Data "Enter the Cosmos" event, which ran on October 1 with an exclusive broadcast with theCUBE, SiliconANGLE Media's livestream studio. There's an evolution of a new software layer pointing the way to the future of AI infrastructure, according to theCUBE Research's Dave Vellante. "Will Vast become the next great AI infrastructure company?" Vellante purported during a live analyst segment for the event. "Right now, Vast is in a good position to be just that, with a healthy balance sheet, strong growth and cash flow positivity -- which is exceedingly rare for a young company that's growing as fast as Vast." theCUBE Research's analysts explored the latest announcements during the event, talking with data platform experts, analysts and industry insiders about developments driving innovations in enterprise AI infrastructure and beyond. Here are three key insights you may have missed from the Vast Data "Enter the Cosmos" event: (* Disclosure below.) Last year, Vast introduced its data platform on theCUBE, which has four components: the DataStore, DataSpace, DataBase and DataEngine. It's a parallel transaction database that does analytics, Vellante noted. "We heard about this novel lock management. It's got full system observability, bridging the gap between structured and unstructured data, disaggregated shared everything," he said. "It's just really back to deep computer science roots for those of you who are interested in that kind of stuff, running anywhere. Vast calls this the [operating system] for the AI age." As a part of this year's "Enter the Cosmos" event, Vast announced the Vast InsightEngine in partnership with Nvidia Corp., advancing the capabilities of enterprise AI infrastructure, which speaks to the Cosmos community that Vast is building out, according to Rob Strechay, principal analyst at theCUBE Research. "[Vast is] aiming at AI and bringing it together," Strechay said. "Also, there was the engine for insights that they're building out, [which] is about how to move up the stack, and continue to move up the stack from a data perspective to be more of that data layer for people doing AI." Vast has origins as a storage company, with the goal of being a data company, a database company, and a key player in enterprise AI infrastructure, creating an operating system for AI. All told, Vast is indeed trying to move up the stack, reiterated Jack Gold, principal analyst at J.Gold Associates LLC. "They're trying to move up to a level that says, 'Hey, without us, you can't do AI from an enterprise perspective.' And that's actually probably a pretty good play for them because if they can move up the stack, one, they can charge more," Gold said. "And number two, the visibility also helps them as they try to go to an [initial public offering] in the not-too-distant future. That's certainly not a bad thing." Here's the complete video interview, part of SiliconANGLE's and theCUBE Research's coverage of Vast Data "Enter the Cosmos": There are two problems that hold enterprises back from using large-scale AI across the business, according to Jeff Denworth (pictured), co-founder of Vast. The first is that a model can't be trained with all of an organization's business. "I was with a Fortune 10 customer the other day. They said, 'We probably have about 100 petabytes of unique data.' GPT-3, just to give you a comparison, was trained on like 45 terabytes," Denworth said. "And so there's just more data sitting within an organization than what can be cost-effectively trained within a model." The second thing is that when one trains a model on any amount of data, the thing it can't do is preserve the security policies on that data that was trained on, according to Denworth. If two individuals get the same model and have different permissions at the source data level, it doesn't come through. "The way that we saw this is that there was an opportunity to make what's called retrieval-augmented generation, or AI retrieval, a lot more simple to deploy across the enterprise," Denworth said. "It starts with unstructured data, which has always been our specialization." There are different ways to approach retrieval. But at the end of the day, what's taking place is that file systems are evolving, according to Denworth. "If you think about it, you now have another form of metadata in the form of vectors that makes your data conversational," he said. "When you go down to your source data, you can get answers out of that data, not just lists of files or things like that. You've got a laptop here that presents all the files to you, but it doesn't say what's in those files. It doesn't allow you just run a [large language model] to ask questions to that data." For 40 years, there's been structured and unstructured data. The unstructured data is 20 times larger than the structured data market that sits in enterprise data warehouses and databases, according to Denworth. "Well, let's suppose that over the next couple [of] years, the unstructured data industry will disappear. Why is that? Because now you can query on all of your data, and I think it's just a massive opportunity for organizations to unlock their data," he said. Seeing a world where all data is accessible, Vast announced its Vast InsightEngine in what could be a significant step forward in enterprise AI infrastructure development. The platform merges the concepts of data storage and database to create a transactional data warehouse on top of an unstructured data store. Here's the complete video interview with Jeff Denworth, part of SiliconANGLE's and theCUBE Research's coverage of the Vast Data "Enter the Cosmos" event: It's no secret that many companies are excited about RAG. The problem for most is merely accessing the data to RAG the model, according to Gold. "It's hard to get the data in real-time, which is really what you need to do if you're going to use AI in any production environment," he said. "It's costly. You may not even know where the data is." Vast is trying to approach this problem. The company has a specific approach on its hands, Gold noted. "There's a huge focus in the [Vast] announcement on real-time and the difficult challenges of AI retrieval, and they've announced an in-architecture vector database," Vellante added. "They've got this globally because they talked about a consistent atomic file system, which eliminates the whole batch operations and is scalable." "What Vast is trying to do is: 'I don't care where your data is, just point us to the data [and] we'll be the consolidator. We'll bring it in, we'll put it into one format [and] we'll be able to do it in real-time,'" Gold further explained. "We're breaking down the silos of data into more of a plane if you will, a mesh, that's very valuable for RAG because that's what gets you to the real-time environment. If you can't do this in real-time, then RAG doesn't work for most people." Vast has announced that its InsightEngine with Nvidia will be generally available in early 2025, and pressure is high to pull it all off, according to Gold. "I'm assuming they can, or they probably wouldn't have announced it, or [Nvidia Chief Executive Officer Jensen Huang] certainly wouldn't have been on stage with them if he didn't think they could do it," he said. "I think it's going to be a game-changer for these guys." Here's the complete video playlist, part of SiliconANGLE's and theCUBE Research's coverage of the Vast Data "Enter the Cosmos" event.

The Vast Cosmos value proposition for AI infrastructures - SiliconANGLE

As new use cases mandate more robust data architectures, Vast Data Inc. is moving fast to bridge the gap between technology's past and its artificial intelligence-driven future. With ambitious initiatives such as the Vast Cosmos community, the company's AI-focused infrastructure aims to redefine how data is used to power intelligent innovation. "I think the two big things that they talked about was the Cosmos community that they're building out and aiming at AI and bringing it together," said Rob Strechay (pictured, right), principal analyst at theCUBE Research. "Also, there was the engine for insights that they're building out, which is about how to move up the stack and continue to move up the stack from a data perspective and be more of that data layer for people doing AI. Not to mention that they are going further with some of the database type stuff for streaming data and things like that with Kafka under the hood." Strechay was joined by fellow theCUBE Research analyst Dave Vellante and Jack Gold (left), principal analyst at J.Gold Associates LLC at the Vast Presents: Enter the Cosmos event, during an exclusive broadcast on theCUBE, SiliconANGLE Media's livestreaming studio. They discussed Vast Cosmos and the company's approach of integrating real-time capabilities with atomic file systems and partnerships with industry leaders such as Nvidia Corp. (* Disclosure below.) The Cosmos community is designed to connect innovators and help them leverage Vast's platform as a foundation for advanced AI and data processing projects. The end goal is to make Vast a capable data layer for enterprises using AI. By moving up the stack, Vast is attempting to provide the infrastructure and be at the forefront of AI innovation and development, according to Gold. This shift could also increase the company's visibility and position them well for an IPO in the near future. "The visibility also helps them as they try to go to an IPO in the not the distant future," Gold said. "That's certainly not a bad thing. I liked the idea of trying to move up the stack, being more embedded but visible to people as opposed to just being under the covers." Vast is also diverting a large chunk of resources to real-time data processing. While large language models have become more intelligent, they still struggle with real-time capabilities. Vast aims to fill that gap. AI requires immediate access to specialized and proprietary data, and Vast's focus on real-time AI retrieval, supported by a globally consistent atomic file system, aims to ensure data is always available and up-to-date, Gold added. "What Vast is trying to do is: 'I don't care where your data is, just point us to the data will be the consolidator. We'll bring it in, we'll put it into one format, we'll be able to do it in real time,'" he said. "We're breaking down the silos of data into more of a plane if you will, a mesh, that's very valuable for RAG because that's what gets you to the real-time environment. If you can't do this in real time, then RAG doesn't work for most people." Here's the complete video interview, part of SiliconANGLE's and theCUBE Research's coverage of the Vast Presents: Enter the Cosmos event:

Vast InsightEngine accelerates agentic AI development with Nvidia - SiliconANGLE

Vast Data announces InsightEngine, Cosmos community for faster AI adoption Vast Data Inc. sees a world in which all data is accessible, and the company took the next step toward making that a reality by announcing the Vast InsightEngine in partnership with Nvidia Corp. Vast Data is a platform that merges the concepts of data storage and database to create a transactional data warehouse on top of an unstructured data store. With Vast InsightEngine, the company now enables agentic AI to make autonomous decisions based on real-time data. "For 40 years you've had structured and unstructured data," said Jeff Denworth (pictured), co-founder of Vast. "The unstructured data is 20 times larger than the structured data market that sits in enterprise data warehouses and databases. Let's suppose that over the next couple years the unstructured data industry will disappear. Why is that? Because now you can query on all of your data, and I think it's just a massive opportunity for organizations to unlock their data." Denworth spoke with theCUBE Research's John Furrier at the Vast Presents: Enter the Cosmos event, during an exclusive broadcast on theCUBE, SiliconANGLE Media's livestreaming studio. They discussed Vast's partnership with Nvidia and the future of data architecture for AI. (* Disclosure below.) The Vast Data platform is built on a triggering system, where every piece of data that flows into it is instantaneously indexed and then updated into the vector database. The Vast InsightEngine will harness this infrastructure for AI-driven decision-making, according to Denworth. "The InsightEngine basically puts us together with Nvidia, where our software now supports deployment on GPU-based systems with what are called NIMS, which are Nvidia inference microservices," Denworth said. "You have NIMS for video, you have NIMS for text, you have NIMS for proteins and other bioinformatic data ... if you look at the exponential rate of improvement for all these new AI inference models, that all then just gets directed towards your enterprise data and allows you to get a lot smarter about what's inside of it." Vector embeddings are essentially metadata in a new format, Denworth explained. The platform now uses a Kafka broker to support real-time data streams from one's data warehouse. The speed of these retrievals is essential if businesses want to stay on the cutting edge of AI adoption. "Agents that are chaining with agents means that businesses are going to start working in GPU time," he said. "At that point, I don't think that you can abide a lot of stale data. So, if you think about the way that retrieval engines have been built today, or data warehouses, you always have some other source data that has to be periodically indexed back into where you're asking your questions to. That doesn't work for us. The way that we see it is we don't want AI agents to ever be exposed to any stale data or any stale permissions data so that CISOs are happy, you're always secure, but businesses are powered by real-time data all the time." Vast has also announced Cosmos, a tech community for sharing insights about agentic AI and AI adoption, with the goal of accelerating the development of an AI ecosystem. When it comes to the future of data storage for AI, Denworth believes the possibilities are limitless. "Our objective is to essentially be universal," he said. "We want to be on every platform and we want to be in every single data center, on-prem or in the cloud, that a customer may want to compute in. We want to minimize the amount of complexity that customers have to deal with so that they can essentially get this thinking machine's experience that we're trying to build towards. That is a ton of work that we have to do on all different levels in order to build out to what customers require for all their data, all their infrastructure, and all the places they want to compute." Here's the complete video interview, part of SiliconANGLE's and theCUBE Research's coverage of the Vast Presents: Enter the Cosmos event:

VAST Data Founds Cosmos, A Tech Community With A Mission To Simplify & Accelerate AI Adoption, Exploration And Discovery

AUCKLAND - 2 October 2024 - VAST Data, the AI data platform company, today announced the launch of Cosmos, an initiative designed to transform how organisations build and advance AI, by creating a comprehensive and supportive environment to nurture innovation, collaboration, growth, and economic prosperity. This community is built of AI practitioners and for AI practitioners - as researchers, technology partners, service providers, and solutions integrators all join together to help democratise the benefits of AI. Cosmos aims to streamline AI adoption for its members by offering a comprehensive, interconnected ecosystem that facilitates conversation, shares use cases, and provides learning opportunities through labs, vendor showcases, and general AI research news. Cosmos helps community members navigate available options and implement the most effective solutions for their unique needs. Artificial intelligence has the ability to revolutionise business productivity. However, today it can be difficult to build an AI-enabled workforce. As new technologies and methodologies are developed, participating in Cosmos is an ideal way to stay informed and be supported on this journey. Cosmos is unlocking innovation by helping to revolutionise the way organisations develop, deploy, and utilise AI technologies from this point forward. Composed of early participants including VAST Data, NVIDIA, xAI, Supermicro, Deloitte, WWT, Cisco, CoreWeave, Core42, NEA, Impetus, Run:AI, Dremio, and more, Cosmos is an AI community that serves to further AI progress by simplifying adoption and pioneering the next frontier of data-driven innovation. "Today marks a monumental step forward in our journey to redefine the future of data and AI," said Renen Hallak, Founder and CEO of VAST Data. "Since our inception, VAST Data has been dedicated to breaking down the barriers that have long confined the potential of data. With the VAST Data Platform at the centre of this comprehensive, interconnected AI ecosystem of technology leaders and AI practitioners, Cosmos will help accelerate discovery, empowering innovation, and enabling the transformation of entire industries. The Cosmos community is designed to accelerate a future where data is a catalyst for change, revolutionising the way data is synthesised and operationalised by: NVIDIA is participating in the VAST-led Cosmos community to support and foster ecosystem collaboration, helping to simplify AI adoption and making it easier for organisations to gain valuable insight from their data. Generative AI presents an unprecedented opportunity to transform industries and drive innovation globally," said Kevin Deierling, Senior Vice President at NVIDIA. "By collaborating as a technology ecosystem, NVIDIA and our partners can empower organisations to accelerate the next wave of AI-powered breakthroughs. Representing the agenda of foundation model builders and bringing model builders together with model users, xAI has also joined VAST as a founding member of Cosmos. "xAI is guided by our mission to accelerate human scientific discovery and advance our collective understanding of the universe with advanced AI systems that are truthful, competent, and maximally beneficial for all of humanity," said a spokesperson at xAI. "It will take many insights to truly maximise the potential of the AI models training on the world's data. We welcome all to the Cosmos, the first frontier for collaboration between AI model builders and users." Cosmos Members Across the Industry "We're in a new era. With the promise and the complexity of AI, data centres, both public and private, must be reimagined to meet the needs of these new AI workloads," said Jeetu Patel, Executive Vice President and Chief Product Officer at Cisco. "The scale of this change will only be possible if we collaborate across the technology stack. Cisco is working with VAST, NVIDIA and others to build modular infrastructure that allows organisations to quickly deploy these AI workloads, including the networks to support them." "The potential for AI is incredibly huge," said Vik Malyala, President and Managing Director EMEA, SVP, Technology and AI at Supermicro. "By working together and sharing our experiences, we can enhance efficiencies, accelerate insights, and ultimately make AI more powerful and safer through an open Cosmos community. Supermicro is committed to helping organisations realise this potential and bring the benefits of AI to different verticals, community, society, and the world." "At CoreWeave, we help customers push the limits of AI innovation by blending our industry expertise with our market-leading compute and AI infrastructure solutions," said Mike Intrator, CEO and Co-founder of CoreWeave. "We are proud to be a part of Cosmos, which affords us the opportunity to help build a community that is centred around providing solutions that power the future of AI." "Our clients are increasing their investments in AI because they have seen strong early value from Generative AI," said Stephen Brown, AI Factory Leader at Deloitte. "A strong data foundation is critical for successfully scaling AI and we look forward to collaborating with members of the Cosmos community to help clients extract tangible value from their GenAI initiatives." "Humans are at their best when they collaborate - offering up their passion, their competency and expertise from every corner of the world," said Lila Tretikov, Partner, Head of AI Strategy at NEA. "We need a global community for AI that can bring together a similar set of values and share in how we shape AI globally for the benefit of all. I think Cosmos can become that forum for everyone to participate and learn to develop something new and share ideas that will break down the barriers we face today and tomorrow." "We're excited for the launch of Cosmos, bringing together industry thought leaders who are moving the AI community forward," said Sendur Sellakumar, CEO at Dremio. "Dremio is happy to be a key enabler by providing AI-ready data through our lakehouse platform, which unifies data across sources, enhances access and quality, and ensures robust protection while managing large datasets efficiently. Leveraging data-as-code capabilities, Dremio simplifies data preparation, helping customers harness their data for advanced AI applications." "We are thrilled to join the Cosmos community, where innovation and collaboration will converge to push the boundaries of AI and data technology," said Rajeeva Gupta, CEO of Impetus Technologies. "Through our Impetus Innovation GenAI Labs and partnership with VAST Data, we're committed to help companies build intelligent enterprises powered by modern data platforms and enterprise AI. Cosmos provides the ideal vehicle to foster best practices around building and scaling AI models to enable the next generation of enterprise-class data architectures for superior business outcomes." "Achieving the full potential of AI requires new forms of industry collaboration, new methods of fostering innovation and the sharing of business domain and technology disciplines," said Mitch Ashley, Chief Technology Advisor, The Futurum Group. "Cosmos is a once-in-a-generation opportunity for industry and technology leaders to garner the once unimaginable benefits from AI, which would be unachievable if we go it alone. It's incumbent upon us to take bold steps like Cosmos that can reshape our future solutions possible with AI." To learn more about Cosmos, register for the World Tour and sign up to join the Cosmos community.

'Undigestible rate of change' sees vendors start 'OCP of AI'

It's called 'Cosmos' and Nvidia, Cisco, X, SuperMicro and VAST Data all think it will help - them and you A group of top enterprise vendors feel that AI is changing so fast it's "undigestible" to many, so they've created an org they hope will function like the Open Compute Project so you don't make a meal of it. The "undigestible rate of change" quote was offered to The Register by VAST Data co-founder Jeff Denworth, who told us "The pace of evolution of AI is nothing short of breathtaking." Users who try to deploy AI, he said, find it "hard to keep track." Denowrth lamented the fact that vendors each define their own approaches to AI, leaving the industry without a neutral "watercooler" at which to share experiences or resources. VAST Data founded an org called "Cosmos" to become that watercooler. Denworth said one model for the org is the Open Compute Project, the outfit that designs hardware and best practices derived from member organizations' own efforts and open sources them for anyone to share. Another model is the RSA conference that, despite bearing a vendor's name, has become a security event that touches on any and all infosec topics. Jonas Roslan, who has filled community manager roles at VMware, Rocky Linux, and Dell EMC, has been hired to make Cosmos work. Denworth said a code of conduct has been developed and recruitment of experts to lead content tracks has commenced. As an example of how Cosmos might work, Denworth told The Register an org working on large language models has been approached to manage an ethical AI track. Speaking for VAST, he said the vendor will "happily prioritize" features that track develops. Denworth admitted Cosmos is in a chicken-and-egg moment because it's yet to produce the content that proves its worth. But he is willing to wait. "This is not a sprint," he said, adding that the org has sufficient resources to have begun planning for events around the world. Plenty of the group's current membership roster - NVIDIA, xAI, Supermicro, Deloitte, WWT, Cisco, CoreWeave, Core42, NEA, Impetus, Run:AI, and Dremio - have decently deep pockets. Nvidia boss Jensen Huang helped to launch Cosmos, which won't hurt its credibility. Its structure might: Open Compute was founded around Facebook's hardware designs, which it shared to enlarge the ecosystem of possible suppliers and out of altruism. The members of Cosmos have something to sell you. And the org's full name is "Cosmos by Vast". Indeed, on the same day as it revealed Cosmos VAST Data announced new collaborations with Nvidia, Cisco, and Equinix that our sibling site Blocks and Files has explained here and here. ®