AWS launches S3 Vectors with 90% cost savings claim, scales to 2 billion vectors per index

AWS claims 90% vector cost savings with S3 Vectors GA, calls it 'complementary' - analysts split on what that means for vector databases

Vector databases emerged as a must-have technology foundation at the beginning of the modern gen AI era. What has changed over the last year, however, is that vectors, the numerical representations of data used by LLMs, have increasingly become just another data type in all manner of different databases. Now, Amazon Web Services (AWS) is taking the next leap forward in the ubiquity of vectors with the general availability of Amazon S3 Vectors. Amazon S3 is the AWS cloud object storage service widely used by organizations of all sizes to store any and all types of data. More often than not, S3 is also used as a foundational component for data lake and lakehouse deployments. Amazon S3 Vectors now adds native vector storage and similarity search capabilities directly to S3 object storage. Instead of requiring a separate vector database, organizations can store vector embeddings in S3 and query them for semantic search, retrieval-augmented generation (RAG) applications and AI agent workflows without moving data to specialized infrastructure The service was first previewed in July with an initial capacity of 50 million vectors in a single index. With the GA release, AWS has scaled that up dramatically to 2 billion vectors in a single index and up to 20 trillion vectors per S3 storage bucket. According to AWS, customers created more than 250,000 vector indexes and ingested more than 40 billion vectors in the four months since the preview launch. The scale increase with the GA launch now allows organizations to consolidate entire vector datasets into single indexes rather than fragmenting them across infrastructure. The GA launch also shakes up the enterprise data landscape by providing a new production-ready approach for vectors that could potentially disrupt the market for purpose-built vector databases. Adding fuel to the competitive fires, AWS claims that the S3 Vector service can help organizations to "reduce the total cost of storing and querying vectors by up to 90% when compared to specialized vector database solutions." AWS positions S3 Vectors as complementary, not competitive to vector databases While Amazon S3 vectors provide a powerful set of vector capabilities, the answer to whether or not it replaces the need for a dedicated vector database is somewhat nuanced -- and depends on who you ask. Despite the aggressive cost claims and dramatic scale improvements, AWS is positioning S3 Vectors as a complementary storage tier rather than a direct replacement for specialized vector databases. "Customers pick whether they use S3 Vectors or a vector database based on what the application needs for latency," Mai-Lan Tomsen Bukovec, VP of technology at AWS, told VentureBeat. Bukovec noted that one way to think of it is as 'performance tiering' based on an organization's application needs. She noted that if the application requires super-fast low low-latency response times, a vector database like Amazon OpenSearch is a good option. "But for many types of operations, like creating a semantic layer of understanding on your existing data or extending agent memory with much more context, S3 Vectors is a great fit." The question of whether S3 and its low-cost cloud object storage will replace a database type isn't a new one for data professionals, either. Bukovec drew an analogy to how enterprises use data lakes today. "I expect that we will see vector storage evolve similarly to tabular data in data lakes, where customers keep on using transactional databases like Amazon Aurora for certain types of workloads and in parallel use S3 for application storage and analytics, because the performance profile works and they need the S3 traits of durability, scaleability, availability and cost economics due to data growth." How customer demand and requirements shaped the Amazon S3 Vector services Over the initial few months of preview, AWS learned what real enterprise customers really want and need from a vector data store. "We had a lot of very positive feedback from the preview, and customers told us that they wanted the capabilities, but at a much higher scale and with lower latency, so they could use S3 as a primary vector store for much of their rapidly expanding vector storage," Bukovec said. In addition to the improved scale, query latency improved to approximately 100 milliseconds or less for frequent queries, with infrequent queries completing in less than one second. AWS increased maximum search results per query from 30 to 100, and write performance now supports up to 1,000 PUT transactions per second for single-vector updates. Use cases gaining traction include hybrid search, agent memory extension and semantic layer creation over existing data. Bukovec noted that one preview customer, March Networks, uses S3 Vectors for large-scale video and photo intelligence. "The economics of vector storage and latency profile mean that March Networks can store billions of vector embeddings economically," she said. "Our built-in integration with Amazon Bedrock means that it makes it easy to incorporate vector storage in generative AI and video workflows." Vector database vendors highlight performance gaps Specialized vector database providers are highlighting significant performance gaps between their offerings and AWS's storage-centric approach. Purpose-built vector database providers, including Pinecone, Weaviate, Qdrant and Chroma, among others, have established production deployments with advanced indexing algorithms, real-time updates and purpose-built query optimization for latency-sensitive workloads. Pinecone, for one, doesn't see Amazon S3 Vectors as being a competitive challenge to its vector database. "Before Amazon S3 Vectors first launched, we were actually informed of the project and didn't consider the cost-performance to be directly competitive at massive scale," Jeff Zhu, VP of Product at Pinecone, told VentureBeat. "This is especially true now with our Dedicated Read Nodes, where, for example, a major e-commerce marketplace customer of ours recently benchmarked a recommendation use case with 1.4B vectors and achieved 5.7k QPS at 26ms p50 and 60ms p99." Analysts split on vector database future The launch revives the debate over whether vector search remains a standalone product category or becomes a feature that major cloud platforms commoditize through storage integration. "It's been clear for a while now that vector is a feature, not a product," Corey Quinn, chief cloud economist at The Duckbill Group, wrote in a message on X (formerly Twitter) in response to a query from VentureBeat. "Everything speaks it now; the rest will shortly." Constellation Research analyst Holger Mueller also sees Amazon S3 Vectors as a competitive threat to standalone vector database vendors. "It is now back to the vector vendors to make sure how they are ahead and better," Mueller told VentureBeat. "Suites always win in enterprise software." Mueller also highlighted the advantage of AWS's approach for eliminating data movement. He noted that vectors are the vehicle to make LLMs understand enterprise data. The real challenge is how to create vectors, which involves how data is moved and how often. By adding vector support to S3, where large amounts of enterprise data are already stored, the data movement challenge can be solved. "CxOs like the approach, as no data movement is needed to create the vectors," Mueller said. Gartner distinguished VP analyst Ed Anderson sees growth for AWS with the new services, but doesn't expect it will spell the end of vector databases. He noted that organizations using S3 for object storage can increase their use of S3 and possibly eliminate the need for dedicated vendor databases. This will increase value for S3 customers while increasing their dependence on S3 storage. Even with that growth potential for AWS, vector databases are still necessary, at least for now. "Amazon S3 Vectors will be valuable for customers, but won't eliminate the need for vector databases, particularly when use cases call for low latency, high-performance data services," Anderson told VentureBeat. AWS itself appears to embrace this complementary view while signaling continued performance improvements. "We are just getting started on both scale and performance for S3 Vectors," Bukovec said. "Just like we have improved the performance of reading and writing data into S3 for everything from video to Parquet files, we will do the same for vectors." What this means for enterprises Beyond the debate over whether vector databases survive as standalone products, enterprise architects face immediate decisions about how to deploy vector storage for production AI workloads. The performance tiering framework provides a clearer decision path for enterprise architects evaluating vector storage options. S3 Vectors works for workloads tolerating 100ms latency: Semantic search over large document collections, agent memory systems, batch analytics on vector embeddings and background RAG context-retrieval. The economics become compelling at scale for organizations already invested in AWS infrastructure. Specialized vector databases remain necessary for latency-sensitive use cases: Real-time recommendation engines, high-throughput search serving thousands of concurrent queries, interactive applications where users wait synchronously for results and workloads where performance consistency trumps cost. For organizations running both workload types, a hybrid approach mirrors how enterprises already use data lakes, deploying specialized vector databases for performance-critical queries while using S3 Vectors for large-scale storage and less time-sensitive operations. The key question is not whether to replace existing infrastructure, but how to architect vector storage across performance tiers based on workload requirements.

S3 Vectors aim to help enable faster AI workflows - SiliconANGLE

AWS bets on S3 Vectors as AI agents reshape the demands of storage Agents powered by artificial intelligence are evolving into powerful systems that depend on richer context and scalable data access, driving demand for technologies such as S3 Vectors, a native vector search capability. It may still be early in the agent era, but this is the first year that real, productive work has started to happen, according to Andy Warfield (pictured), vice president and distinguished engineer at Amazon Web Services Inc. This shift is bringing people back into hands-on development, fitting naturally into busy workflows and allowing deeper focus on finer details of development work. "I think that what the storage teams are really seeing is [that] on the generative AI side, we've got models that can write code, we've got models that can write docs," Warfield said. "At the end of the day, the big bridge that is on us to solve is the bridge between all of that kind of capability and the data that people have." Warfield spoke with theCUBE's John Furrier at AWS re:Invent, during an exclusive broadcast on theCUBE, SiliconANGLE Media's livestreaming studio. They discussed the industry's shift toward powerful AI agents and the rapid evolution of storage needed to support emerging AI workloads. On Tuesday, AWS announced that Amazon S3 Vectors is now generally available with increased scale and performance. By allowing customers to store and query vector embeddings directly in Amazon S3, the service can improve the relevance, latency and cost-efficiency of AI and retrieval-augmented applications. The launch is a major milestone that has generated a lot of enthusiasm across the organization, according to Warfield. "With so many mature customers building on top of S3, and so many emerging customers just starting with objects and then building out tables for structured data and vectors for search ... [Amazon S3 Vectors] is letting them build apps that are soup-to-nuts S3-based, which is super cool to see," Warfield said. The announcement quickly became the most popular S3 preview to date, according to Warfield. Use cases remain varied, especially in data-intensive fields like the life sciences. "We're seeing people use S3 Vectors to do drug development -- like radiology applications," he noted. "It's actually wild to see the kinds of things that people are using vectors for." Here's the complete video interview, part of SiliconANGLE's and theCUBE's coverage of AWS re:Invent: