Curated by THEOUTPOST
On Wed, 20 Nov, 12:11 AM UTC
2 Sources
[1]
Zilliz debuts new release of its cloud-based vector database - SiliconANGLE
Zilliz debuts new release of its cloud-based vector database Startup Zilliz Inc. today debuted a new release of its flagship offering, a managed vector database called Zilliz Cloud that artificial intelligence models can use to hold information. Redwood Shores, California-based Zilliz is backed by more than $110 million in funding. It offers an open-source vector database called Milvus. Zilliz Cloud, the offering the company updated today, is a paid cloud-based version of Milvus that includes additional features and removes the need for customers to manage the underlying infrastructure. AI models don't process files such as documents in their original form, but first turn them into embeddings. Embeddings are a type of vector, a mathematical structure that lends itself well to storing information. Vector databases such as Zilliz Cloud are specifically optimized to hold AI models' embeddings. Zilliz Cloud ships with a search engine that enables an AI application to enter a piece of data and find all the similar records on file. A neural network optimized to generate shopping suggestions, for example, could enter a smartphone description provided by a shopper and have Zilliz Cloud return a list of matching devices. The new release of the platform that debuted today promises to process such queries up to 10 times faster than earlier versions. The upgraded search engine can process multiple types of queries. According to Zilliz, it's capable of running so-called single-vector searches in which an AI model enters one record and the database retrieves all the similar files it stores. The platform also supports more advanced use cases such as hybrid searches. Those are requests that specify the files Zilliz Cloud retrieves should not only be relevant to the user's query, but must also contain the same keywords. To speed up searches, developers often equip their databases with a so-called index. This is a collection of shortcuts that can be used by AI models to more quickly sift through contents of a database and find the record requested by the user. According to Zilliz, the new version of Zilliz Cloud includes a feature called AutoIndex that automates the index generation process to save time for customers. "As companies scale up similarity search for recommendation systems, RAG and image retrieval, they face a choice between overprovisioning hardware or spending months tuning vector indexes," said Zilliz founder and Chief Executive Officer Charles Xie.
[2]
Open source vector database vendor targets enterprise AI costs with cloud update
Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More As generative AI usage has grown dramatically in the last several years, vector databases have evolved from cutting-edge technology to essential enterprise infrastructure. With vector databases becoming more crucial, enterprises are taking an ever closer look at performance and cost. Zilliz, the company behind the open-source Milvus vector database, is announcing new features aimed at dramatically reducing costs and complexity for production deployments, addressing the growing demands of enterprise users who have moved beyond initial experiments to full-scale AI implementations. The timing is particularly relevant given the explosive growth in vector database adoption since late 2022, when OpenAI's ChatGPT catalyzed widespread interest in AI applications. The new features specifically target enterprises struggling with growing deployment sizes and the complexity of managing vector databases in production environments. In just two years, deployment scales have grown from millions to billions of vectors. Zilliz's largest implementation now manages 100 billion vectors. The technology is now deployed across diverse use cases including multimodal applications, recommendation systems, autonomous driving, drug discovery, fraud detection and cybersecurity. "In the past two years, we definitely saw that vector databases are moving from a cutting edge technology, to becoming a more mainstream technology," Charles Xie, founder and CEO of Zilliz, told VentureBeat. Enterprise AI vector database differentiation in a crowded market In 2024, vector database technology has become table stakes for enterprise AI deployment. Nearly every database vendor has some form of vector implementation, including Oracle, Microsoft, Google, DataStax and MongoDB among others. Milvus however is a bit different in that it is a purpose-built vector database. In that category, competition includes vendors like Pinecone. While there are certainly other open source vector database technologies, Milvus holds the somewhat unique distinction of being the only one that is part of the Linux Foundation's LF AI & DATA effort. Milvus being hosted under the Linux Foundation's AI & Data Foundation has enabled it to receive contributions from a wide ecosystem of participating institutions and organizations. Xie noted that among the organizations that have contributed code to the Milvus open source project are IBM, Nvidia, Apple, Salesforce and Intel. According to Xie, the combination of having an open source foundation, native vector database focus and most importantly having specialized features, help to differentiate his company's technology in the crowded market. Xie argued that being solely focused on vector database technology allows it to deliver more comprehensive and optimized solutions, than vendors that include vector as just yet another data type. This specialization has enabled Zilliz to develop features specifically tailored to enterprise vector search needs, including compliance, security and high availability capabilities that production environments demand. How Zilliz is improving its vector database for Enterprise AI production needs The Zillliz Cloud offering is built on top of the open source Milvus database. The offering provides a manages service for the database that makes its easier for organizations to consume and use. As part of the latest Zilliz Cloud update the company has added an automated indexing system that removes the need for manual parameter tuning. The new feature automatically picks the optimal indexing algorithms to provide the best performance, without the user having to manually configure the indexes. "Out of the box, you get the best performance," Xie said. The auto-indexing feature is part of Zilix Cloud's effort to provide an "autonomous driving mode" for vector databases, using machine learning algorithms to optimize performance behind the scenes. This helps reduce the total cost of ownership for customers, as they don't need to spend time and resources on manual index tuning. Algorithm optimization helps to improve specific Enterprise AI use cases Going a step further, Zilliz is now also integrating an algorithm optimizer. The optimization works with IVF (inverted file) as well as graph-based vector retrieval algorithms. Memory allocation as well as compute performance is also optimized for fast execution that the company claims provides up to 3X speedup over unoptimized implementations The algorithm optimizer works across different use cases, whether the organization is running a document search system, a recommendation engine, fraud detection, or any other vector-based application. Hybrid search and storage innovation helps lower enterprise AI cost The new release also introduces hybrid search functionality, combining vector similarity search with traditional keyword-based searching in a single system. The integration allows companies to consolidate their search infrastructure and reduce operational complexity. Xie explained that the keyword-based search component makes use of the industry-standard BM25 algorithm as well as a sparse index. To address growing storage costs, Zilliz has implemented a hierarchical storage system that makes its service more cost-effective than traditional in-memory vector databases. The multi-layer storage hierarchy allows most data to be stored on local disks and object storage, making it cheaper than a pure in-memory solution, according to Xie. Xie claims through the new set of innovations for performance and storage, Zilliz will be able to reduce vector database consumption costs for its users. Looking ahead, Zilliz has ambitious plans for further cost optimization. "I'm going to make a very bold prediction here, that in the next five years, the cost, the total cost of vector database solution, should be reduced by another 100 times," Xie stated.
Share
Share
Copy Link
Zilliz, the company behind the open-source Milvus vector database, has announced significant updates to its Zilliz Cloud offering, aiming to reduce costs and complexity for enterprise AI deployments while improving performance.
Zilliz, the startup behind the open-source Milvus vector database, has unveiled a significant update to its Zilliz Cloud offering, addressing the growing demands of enterprise AI deployments. The California-based company, backed by over $110 million in funding, aims to optimize performance and reduce costs for organizations scaling up their AI infrastructure 12.
The latest release of Zilliz Cloud boasts several key improvements:
Faster Query Processing: The platform now processes queries up to 10 times faster than previous versions, significantly enhancing the efficiency of AI applications 1.
AutoIndex Feature: A new automated indexing system eliminates the need for manual parameter tuning, automatically selecting optimal indexing algorithms for best performance 2.
Algorithm Optimizer: This feature optimizes IVF and graph-based vector retrieval algorithms, resulting in up to 3X speedup over unoptimized implementations 2.
Zilliz Cloud now offers more versatile search capabilities:
Hybrid Search: The platform combines vector similarity search with traditional keyword-based searching in a single system, allowing companies to consolidate their search infrastructure 2.
Multiple Query Types: Support for single-vector searches and more advanced use cases like hybrid searches that consider both relevance and specific keywords 1.
To address growing storage costs, Zilliz has implemented:
As vector databases transition from cutting-edge technology to essential enterprise infrastructure, Zilliz is positioning itself to meet the evolving needs of large-scale AI implementations:
Scalability: Zilliz's largest implementation now manages 100 billion vectors, reflecting the dramatic growth in deployment sizes over the past two years 2.
Diverse Use Cases: The technology is deployed across various applications, including multimodal systems, recommendation engines, autonomous driving, drug discovery, fraud detection, and cybersecurity 2.
Open-Source Foundation: As part of the Linux Foundation's LF AI & DATA effort, Milvus benefits from contributions from major tech companies like IBM, Nvidia, Apple, Salesforce, and Intel 2.
In an increasingly crowded market for vector databases, Zilliz differentiates itself through:
Specialized Focus: Unlike general-purpose database vendors, Zilliz concentrates solely on vector database technology, allowing for more comprehensive and optimized solutions 2.
Enterprise-Ready Features: The platform offers compliance, security, and high availability capabilities required for production environments 2.
As generative AI usage continues to grow, Zilliz's latest updates aim to address the critical challenges of performance, cost, and complexity faced by enterprises moving beyond initial experiments to full-scale AI implementations 2.
Reference
Vector databases are emerging as crucial tools in AI development, offering efficient storage and retrieval of high-dimensional data. Their impact spans various industries, from e-commerce to healthcare, revolutionizing how we handle complex information.
3 Sources
3 Sources
Aerospike Inc. has released an updated version of its Vector Search technology, featuring new indexing and storage innovations designed to enhance real-time accuracy, scalability, and ease of use for developers working with generative AI and machine learning applications.
3 Sources
3 Sources
Dutch AI database startup Weaviate introduces Weaviate Embeddings, an open-source tool designed to streamline data vectorization for AI applications, offering developers more flexibility and control over their AI development process.
2 Sources
2 Sources
Vectorize AI Inc. debuts its platform for optimizing retrieval-augmented generation (RAG) data preparation, backed by $3.6 million in seed funding led by True Ventures. The startup aims to streamline the process of transforming unstructured data for AI applications.
2 Sources
2 Sources
Pinecone introduces innovative features to its vector database, including inference capabilities and cascading retrieval, aiming to improve AI application development and accuracy. The update combines dense and sparse vector retrieval with reranking technologies.
2 Sources
2 Sources
The Outpost is a comprehensive collection of curated artificial intelligence software tools that cater to the needs of small business owners, bloggers, artists, musicians, entrepreneurs, marketers, writers, and researchers.
© 2025 TheOutpost.AI All rights reserved