New Alluxio Release Boosts GPU Utilization, Data Management Performance For AI/ML Applications

The new release of the Alluxio Enterprise AI data orchestration platform makes it easier to use GPU-based systems for training and operating AI applications and to provision AI/ML systems with data at HPC levels. Alluxio has launched a new release of its data management platform for AI tasks that can better utilize high-performance GPU systems, helping businesses and organizations turbocharge their data-intensive AI workloads. The new 3.2 release of Alluxio Enterprise AI also offers enhanced data I/O capabilities and improved performance with HPC (high performance computing) storage systems to boost AI processing. "This release is all about GPUs and, specifically, data loading performance and storage performance for different GPUs," said Adit Madan, Alluxio director of product, in an interview with CRN. [Related: Meeting The Exploding Demand For Data: The 2024 CRN Big Data 100] Alluxio, headquartered in San Mateo, Calif., develops its flagship Alluxio Enterprise Data platform for a range of big data management tasks. In October the company debuted Alluxio Enterprise AI, a new system built on the company's core data orchestration technology specifically for data-intensive AI and machine learning tasks. Madan noted that in provisioning AI and machine learning applications, many organizations have separate compute and storage infrastructure dedicated to developing and training AI/ML applications - including the use of deep learning frameworks such as PyTorch and TensorFlow - and for when those applications are running in production. So while an organization may use high-performance GPU-based systems for training AI/ML systems, the AI/ML applications may not be able to tap into that compute power once in production, Madan said. The Alluxio platform provides the needed flexibility to access those GPU resources as needed. "With the scarcity of GPUs, this is something that we've seen resonating with a lot of the early customers for this product," Madan said. The company says the 3.2 release provides more than 97-percent GPU utilization in large language model training benchmarks. The new release also provides enhanced data I/O performance, achieving up to 10GB/second throughput and 200K IOPS for AI/ML applications. The new software also delivers data storage performance the company says is comparable to HPC storage, based on MLPerf benchmarks, without the need for additional HPC storage infrastructure. Alluxio Enterprise AI also offers a new file system API, an FSSpec implementation, for Python applications, a move that expands Alluxio's interoperability within the Python development ecosystem. And new advanced cache management functionality provide administrators with more data management control. "By achieving comparable performance to HPC storage and enabling GPU utilization anywhere, we're not just solving today's challenges, we're future-proofing AI workloads for the next generation of innovations," Madan said.

CRN

Fri, 12 Jul, 10:00 PM UTC

Alluxio Enterprise AI 3.2 - DZone

In an era where AI and machine learning are pushing the boundaries of computational power, efficient GPU utilization, and data access have become critical bottlenecks. Alluxio, a pioneer in data orchestration for analytics and AI, has unveiled its latest offering, Alluxio Enterprise AI 3.2, to address these challenges head-on. This release promises to transform how organizations leverage their GPU resources and manage data for AI workloads, offering a blend of performance, flexibility, and ease of use that could reshape the landscape of AI infrastructure. One of the standout features of Alluxio Enterprise AI 3.2 is its ability to enable GPU utilization anywhere. This capability is a game-changer in a world where GPU resources are often scarce and distributed across various environments. Organizations can now run AI workloads wherever GPUs are available, whether on-premises, in the cloud, or in a hybrid setup. Adit Madan, Director of Product at Alluxio, emphasizes the significance of this feature, especially in hybrid and multi-cloud scenarios. "We're seeing a trend where companies have their primary data lakes in one of the major clouds but choose to train in a different location due to GPU availability," Madan explains. This flexibility allows organizations to optimize their AI workflows without data locality constraints. Alluxio Enterprise AI 3.2 offers flexibility and performance that matches or beats top-tier HPC storage vendors. The platform achieves an impressive 97% GPU utilization for language models by eliminating I/O bottlenecks. Regarding raw numbers, Alluxio Enterprise AI 3.2 can achieve up to 10GB/s throughput and 200K IOPS for a single client, representing 75% of the hardware limit. This performance boost translates to a 35% improvement for repeated access compared to the previous version. Madan proudly states, "On the MLPerf storage benchmarks, we've seen 97% GPU utilization for A100 GPUs. We can fully saturate the GPUs, comparable to all the high-performance vendors." Recognizing Python's central role in the data science ecosystem, Alluxio has introduced a new Python Filesystem API. This addition is a significant step towards simplifying integration for data scientists and AI engineers working with frameworks like Ray. Based on the FSSpec implementation, the API allows for seamless integration with Python applications. "Now, with the Python API being completely RESTful on the client side, there are no services to run. You just pip install Alluxio, and you can start communicating, eliminating the need for a person on the platform side," Madan explains. This streamlined approach addresses a common pain point in AI workflows. As Madan illustrates with a real-world example, "Speaking to someone in a large hedge fund, what they told me is, 'As a data scientist, as a consumer of data, I love your solution, but my only concern is that my platform guy is pushing back because he doesn't want to manage another service.' This new API helps solve that problem." Alluxio Enterprise AI 3.2 is a compelling alternative to traditional HPC storage for AI workloads. It allows organizations to leverage their existing data lake resources instead of investing in dedicated HPC storage solutions. Madan outlines the architectural considerations: "Let's say the primary data lake is in Amazon S3, but training is moving on-premises for various reasons. With Alluxio, you have your training cluster, Alluxio co-locates with training, and you simply point to the data. You must ensure the network pipe between the two is robust enough." This approach eliminates the need for data migration and the provisioning of separate high-performance storage systems. It offers a software-defined solution where performance can be dialed up as needs grow, all while utilizing the existing data infrastructure. Alluxio Enterprise AI 3.2 introduces advanced cache management features for organizations running large-scale AI projects in multi-tenant environments. These include a RESTful API for seamless cache management, an intelligent cache filter, and granular cache control. While these features primarily focus on authorization and governance rather than data segregation, they offer significant benefits in managing performance across tenants. Madan explains, "If one tenant is more important, as a platform admin, I can say, 'This is my MVP or most important tenant, and I want to make sure this person's performance is not affected by others.'" This level of control allows organizations to optimize resource utilization and ensure critical AI workloads receive the necessary priority and performance. An interesting aspect that emerged during the discussion with Madan was the concept of "GPU clouds." While prominent public cloud providers offer GPU resources, specialized GPU cloud providers like Lambda and Coreweave are gaining traction. Even Oracle is positioning itself as a significant supplier of GPUs. This diversification in the GPU cloud market aligns well with Alluxio's strategy of enabling GPU utilization anywhere. Organizations can now choose the most suitable GPU resources for their specific needs without being locked into a single provider or worrying about data movement and access. Alluxio Enterprise AI 3.2 represents a significant step forward in addressing the challenges of GPU utilization and data access for AI workloads. By offering a solution that combines high performance, flexibility, and ease of use, Alluxio empowers organizations to make the most of their AI investments. The platform's ability to achieve near-maximum GPU utilization, its Python integration, and advanced cache management features position it as a valuable tool for data scientists, AI engineers, and infrastructure architects. As AI workloads grow in complexity and scale, solutions like Alluxio Enterprise AI 3.2 will play a crucial role in shaping the future of AI infrastructure. For developers, engineers, and architects working on AI projects, Alluxio Enterprise AI 3.2 offers a pathway to optimizing GPU utilization, simplifying data access, and enhancing overall workflow efficiency. As the AI landscape evolves, tools that can bridge the gap between compute resources and data storage will become increasingly vital. Alluxio is positioning itself at the forefront of this transformation.

DZone

Mon, 15 Jul, 4:02 PM UTC

The art of AI architecture

There is a common misconception that one GPU cloud is very much like another. But that is simply not the case. They are built with different technologies and architectures, and they come with their own sets of challenges, advantages, and disadvantages. The most advanced AI cloud operators today are developing new blueprints for GPU datacenters that deploy NVIDIA H100s in Kubernetes or other virtualized environments to reach new levels of performance for AI processing. To the customer the specs are basically the same. AI cloud computing service providers boast about Nvidia HGX H100 arrays and the fact they have 3.2 terabytes of InfiniBand. But this is because they're all using the same network cards. If all clouds seem the same from a technical standpoint, customers will make decisions based on price. But technical specifications alone do not tell the whole story. You can buy a Toyota Corolla with 100 kilowatts of power, and you buy a Mercedes with 100 kilowatts of power, but they are not the same thing. The build quality is different, the cost is different, and the user experience is different. It is the same with data centers. If the head of finance oversaw architecting, it's likely we would have the Toyota Corolla of data centers, and that's fine for some, but given the choice, most organizations are going to pick the Mercedes. A data center built with cost savings at its heart might work for some customers, but it will be slower, and/or offer less cloud storage, and it may even be less secure. The construction of GPU clouds varies enormously between data centers. A common misconception is that AI infrastructure can just be built to the NVIDIA DGX reference architecture. But that is the easy bit and is the minimum viable baseline. How far organizations go beyond that is the differentiator. AI cloud providers are building highly differentiated solutions through the application of management and storage networks that can dramatically accelerate the productivity of AI computing. The deployment of GPU data centers as the infrastructure for AI is a complex and challenging task that requires deep understanding of the balancing technologies to maximize throughput. High quality management systems and security systems have an obvious impact on the customer experience. Another major factor governing the performance of AI clouds is the storage architecture. Using dynamically assigned WEKA architectures, NVMe (non-volatile memory express) disks and GPU direct storage can improve speed of execution by up to 100% for certain workloads such as Large Language Models (LLMs) used in machine learning. WEKA's data platform offers unmatched performance and scalability, particularly in feeding data to large-scale GPU environments. By transforming stagnant data silos into dynamic data pipelines, it effortlessly powers data-starved GPUs, enabling them to operate with up to 20 times greater efficiency and sustainability. How fast you access storage is critical in AI because you're dealing with very large data sets of probably small chunks of data. You could be looking at 100 billion pieces of data spread across a network. Compared with digital media where you're working with a few thousand assets at most, although they could be hundreds of gigabytes each, it's a very different profile. Traditional hard disks will provide good speeds for digital media. Whereas an AI workload is very random by comparison, you need to take one gig here and one gig there and do that millions of times a second. Another important difference to note with regards AI architecture versus traditional storage models is the absence of a requirement to cache data. Everything is done by direct request. The GPUs talk directly to the disks across the network, they don't go through the CPUs or the TCP IP stack. The GPUs are directly connected to the network fabric. They bypass most of the network layers and go directly to the storage. It removes network lag. AI infrastructure architecture needs to be designed to maximize compute power for the coming wave of AI workloads. Furthermore, network architectures should be designed to be completely uncontended. Lots of organizations will promise that, but you need a provider that has over provisioned to deliver that level of guarantee. Major AI users such as Tesla and Meta are designing cloud infrastructure to meet the needs of different applications where the AI cloud architectures can be dynamically optimized for specific workloads. But most cloud providers do not have the luxury of knowing exactly what they're building for. Returning to the automotive analogy, most modern transport networks in major cities around the world were not build with today's volumes of traffic in mind. Indeed, the issue with building a data center based on a current or even projected target in mind is that data canters will reach capacity sooner than you think. Clouds need to be both overprovisioned and extremely scalable. If you don't know exactly what you're building for, you just need to build the biggest, fastest, most secure, easy to use platform possible. To optimize throughput, data centers need highly distributed storage architecture, with hundreds of disks generating 10s of millions of Input/output operations per second throughout your servers. GPU clouds are also reliant on supporting infrastructure. For instance, if you're running Kubernetes, you need master nodes. You need coordination nodes, you need nodes to ingress the data, you need nodes to just log into so that you can have dashboards. The cloud provider needs to provision very substantial amounts of non-GPU compute in the same region. Building genuine clouds is neither easy nor cheap. A lot of data center providers call themselves 'cloud', but really it is more like a managed hardware environment. It is certainly less risky from a financial perspective to sign up organizations to x-year contracts, and then build out a facility that meets the demands of the contract. And there are some benefits particularly around security and performance. But it is not cloud. Cloud is self-service, it's API driven, you log in, click a button, and you've got access to the processing power you need for as long as you need it. There are a lot of organizations out there that don't have the resource or requirement for ongoing data center support, they might only need the processing power for a short time and cloud gives them that option. NexGen Cloud is democratizing AI by opening access to shared high performance architectures. A final consideration, and what that is becoming increasingly important, is energy consumption. Organizations of all sizes are being asked to not only monitor but also improve on their emissions. Not just from customers and society at large, but also from a regulatory perspective. Google and Microsoft recently announced an agreement with Nucor for a clean energy initiative to power data centers and ultimately reach Net Zero for AI processing. ESG performance is also proving to be a critical metric in terms of shareholder value and AI is incredibly power hungry. Ultimately, organisations should partner with a provider they can rely on. A partner that can offer guidance, provide engineering and support. Businesses using cloud infrastructure are doing so to concentrate on their own core differentiators. They're not in the business of running cloud AI infrastructure, they want convenience, security, and reliability, and genuine cloud provides all of that on-demand. We feature the best AI tools.

TechRadar

Tue, 23 Jul, 8:01 AM UTC

AI Infrastructure Innovations Reshape Computing Landscape at SC24

Weka, Nvidia, and partners showcase advancements in AI infrastructure at SC24, addressing challenges in scalability, efficiency, and sustainability for enterprise AI deployments.

7 Sources

Wed, 20 Nov, 8:02 AM UTC

Netweb Technologies Introduces Skylus.ai: Simplified and Easy Way to Set up a GPU-based AI Lab infrastructure

Netweb Technologies launches ai, an appliance for optimized GPU resource management across multi-vendor environments. ai enables flexible resource allocation, maximizing GPU utilization for AI workloads. The platform integrates with Tyrone HyperScale NAS -- ParallelStor, ensuring high throughput and low latency for large-scale AI applications. ai includes Kubyts, a workbench offering pre-tested, pre-built ready-to-deploy container images, simplifying application deployment. ai supports industries such as pharmaceuticals, finance, education, automotive design, or any organization looking to leverage the power of AI. It also facilitates Research as a Service (ReaaS) and Innovation as a Service (IaaS), enabling seamless AI adoption across industries and use cases. Netweb Technologies, the leading Indian OEM in high-end computing domain and innovative IT solutions, today announced the launch of Skylus.ai, a groundbreaking composable GPU aggregation-disaggregation appliance designed to optimize GPU resource management for AI including GenAI workloads. It addresses the critical challenges faced by organizations in utilizing multi-vendor GPU and CPU resources, offering a vendor-agnostic solution that drives faster ideation, fosters collaboration, and accelerates experimentation while optimizing resource utilization and total cost of ownership. Skylus.ai simplifies the integration of multi-vendor GPUs and CPUs, enabling the creation of tailored workspaces that align with specific workload needs. Whether utilizing CPUs, full GPUs, or slices of multi-instance GPUs, Skylus.ai ensures seamless and efficient resource allocation. The platform also uses Tyrone Kubyts, a workbench featuring a pre-tested, pre-built, optimized, and qualified set of curated container images in the form of an Image marketplace. This allows users to access packaged containers or VM disk images that are ready to deploy, saving time and reducing the complexity of application deployment from scratch. Skylus.ai resource composability transforms static resource allocation into a dynamic model, allowing for the optimal provisioning of resources based on unique workload requirements. For example, an enterprise GPU server with 8 GPUs can dynamically allocate slices of these GPUs to different teams, maximizing utilization and controlling costs. In addition to its advanced resource management capabilities, Skylus.ai integrates seamlessly with Tyrone HyperScale NAS -- ParallelStor, a hyperscale storage solution that supports file and object access, ensures throughput and latency, and manages large vol umes of small files with ease. This integration eliminates the need for multiple data copies and supports global data management across on-premises and cloud environments Mr. Sanjay Lodha, Chairman and Managing Director (CMD) of Netweb Technologies India Limited, said, "We are thrilled to unveil Skylus.ai, a revolutionary, Make-in-India AI technology platform poised to transform AI infrastructure and GPU resource management. AI workloads are expanding rapidly, creating major challenges in resource allocation, provisioning, and management. Businesses struggle with GPU wastage, inefficiencies, and infrastructure bottlenecks, slowing AI adoption and innovation. Skylus.ai addresses these pain points with an intelligent, scalable, and vendor-agnostic solution. It optimizes AI workloads, reduces GPU wastage, and optimizes resource management, enabling businesses to focus on AI innovation - not infrastructure issues. More than just a tool, Skylus.ai is a catalyst for AI-driven transformation, setting new benchmarks in efficiency, innovation, and performance. To top it , these innovations are all Made in India." About Netweb Technologies Netweb is a publicly listed Indian OEM in the space of high-end computing with fully integrated design, manufacturing, and integration capabilities. It designs, manufactures, integrates/assembles, and deploys high-end computing IT hardware and solutions comprising proprietary middleware solutions, end-user utilities, and pre-compiled application stack. Netweb's HCS products/solutions offerings include high-performance computing (Supercomputing/HPC) systems, private cloud and hyper-converged infrastructure (HCI), AI (Artificial Intelligence) systems, and enterprise workstations, high-performance storage (HPS/ Enterprise Storage System) solutions, data center servers, software and services for their HCS offerings, spare and others.

CXOToday

Thu, 20 Feb, 10:18 AM UTC

WEKA

Contact for Pricing

About WEKA

Related fields

Related News

Similar products

Weaviate

H2O AI

Knime

Weights & Biases

YData