Akamai Launches Cloud Inference: Revolutionizing AI Performance with Edge Computing

Akamai distributes AI inference across the globe, promising lower latency and higher throughput - SiliconANGLE

Akamai distributes AI inference across the globe, promising lower latency and higher throughput Akamai Technologies Inc. is expanding its developer-focused cloud infrastructure platform with the launch of Akamai Cloud Inference, a highly distributed foundation for running large language models closer to their real-world users. The new service is built atop Akamai Cloud, which Akamai claims is the world's most distributed cloud infrastructure platform, allowing it to overcome the problems associated with centralized, cloud-based models. According to Adam Karon, Akamai's chief operating officer and general manager of cloud technology, one of the main problems with running powerful LLMs in centralized cloud platforms like Google Cloud and Microsoft Azure is low latency. Simply put, the data those models feed on is located far away from where the model is hosted, and the latency means their performance can take a substantial hit. "Getting AI data closer to users and devices is hard and it's where legacy clouds struggle," Karon said. "While the heavy lifting of training LLMs will continue to happen in big hyperscale data centers, the actionable work of inference will take place at the edge." Karon said Akamai's unique network, which was built over the last two-and-a-half decades, will become "vital" for the future of AI, setting the company apart from every other cloud provider. Akamai's big advantage is that it began life as a content delivery network, building up a system of geographically distributed servers that host web content close to users. By caching websites closer to their users, it improves website load times and lowers latency, improving the overall experience. Having established itself as one of the world's leading CDNs, Akamai expanded into the cloud infrastructure business following its $900 million acquisition of the developer cloud company Linode LLC. Since then, it has rapidly built out its offerings to include services such as storage, compute and networking, competing with traditional cloud infrastructure firms such as Amazon Web Services Inc. and Microsoft Corp. Its cloud infrastructure is made up of more than 4,100 points of presence spread across ore than 1,200 networks in more than 130 countries, making it much more distributed than any of its traditional cloud computing rivals. The Akamai Cloud Inference service is hosted on the company's distributed CDN servers, providing tools for companies to build and run AI applications much closer to their users. Inference is the process where trained AI models generate outputs based on the user's prompts. It's often referred to as the operational phase of AI, where models apply what they have learned during their training. Karon likened LLM training to the process of creating a map. It's a slow and resource-intensive process that involves gathering data, analyzing the terrain and plotting routes. On the other hand, AI inference is more like using a GPS device, he said. "It's about instantly applying that knowledge, recalculating in real time and adapting to changes to get you where you need to go," he said. "Inference is the next frontier for AI." The company claims it can provide triple the throughput for AI inference and reduce latency by up to two-and-a-half times over traditional cloud infrastructures. It also provides significant cost savings on AI inference workloads, the company says. Akamai Cloud Inference consists of compute infrastructure resources ranging from traditional central processing units for low-cost, fine-tuned inference to powerful graphics processing units and tailored application-specific integrated circuits or ASICs, which provide additional horsepower for more demanding workloads. Its compute services are integrated with Nvidia Corp.'s AI Enterprise software suite to cater to developer's needs. The service also provides advanced data management capabilities thanks to Akamai's partnership with Vast Data Inc., which provides access to a modern "data fabric" that's customized for AI applications. Vast Data's storage infrastructure helps to provide rapid access to data as it's generated in real time, aided by Akamai's close geographic proximity to end users. Akamai's inference platform supports containerized workloads too, enabling demand-based autoscaling, multicloud portability and performance optimization for AI applications. These capabilities are delivered through the Linode Kubernetes Engine, which is an enterprise-grade cloud orchestration platform built for managing large-scale cloud workloads. Finally, Akamai Cloud Inference sets itself apart with its edge compute capabilities, enabling it to execute LLM inference on serverless infrastructure at the edge of the network, supporting latency-sensitive AI applications. Akamai believes its cloud inference services are going to be in big demand, citing a report by Gartner Inc. that predicts 75% of the world's data will be generated outside of centralized data centers and cloud regions by the end of the year. By processing this data closer to where it's created, AI applications can make decisions faster and provide real-time insights in the most remote environments. The company says this will be of immense value for applications such as autonomous vehicles, AI-powered crop management, retail shopping experiences and more.

Akamai transforms AI with faster and cheaper edge inference

Akamai Technology launches Akamai Cloud Inference, a new cloud service that enhances the efficiency of AI inference tasks. It delivers improved throughput, reduced latency, and lower costs than traditional hyperscale infrastructure. Akamai Cloud Inference runs on Akamai Cloud, the world's most distributed platform. This new technology is designed to address the limitations of centralized cloud models by processing AI data closer to the user and devices. Adam Karon, Chief Operating Officer and General Manager, Cloud Technology Group at Akamai, highlighted the challenge of distributing AI data efficiently. "Getting AI data closer to users and devices is hard, and it's where legacy clouds struggle," Karon stated. AI inference on Akamai Cloud enables platform engineers and developers to build and run AI applications closer to end users. This new solution offers 3x better throughput and up to 2.5x reduction in latency. The new tools empower businesses to save up to 86% on AI inference and agentic AI workloads compared to traditional hyperscaler infrastructure. Key features of Akamai Cloud Inference include: The scalable and distributed architecture of Akamai Cloud allows compute resources to be available globally -- from cloud to edge -- while accelerating application performance and increasing scalability. The platform spans 4,200 points of presence across 1,200 networks in over 130 countries. Polyhedra just made AI's honest secrets public Akasm reveals the shift from large language models (LLMs) training to AI inference, emphasizing the need for practical AI solutions. LLMs are effective for general-purpose tasks but often come with high costs and time-consuming requirements. Instead of investing heavily in LLMs, enterprises are moving to lighter AI models. These are optimised for specific business problems, offer a better return on investment today. Akamai Cloud Inference supports processing AI data closer to where it is generated, solving the demands for more distributed AI solutions. Akamai's new offering represents a notable move towards decentralized AI, solving the classic cloud computing conundrum of distance. Why? Because reduced latency directly translates to real, immediate savings and a better user experience, which is a tough combination for competitors to beat. One particularly savvy feature is the emphasis on containerization, ensuring the deployment of AI applications remains far easier and more secure than traditional setups. The use of Linode Kubernetes Engine (LKE)-Enterprise underlines Akamai's commitment to offering modern, efficient tools tailored for today's tech challenges.

Akamai Unveils Cloud Inference to Redefine AI Performance & Efficiency

Akamai unveiled Akamai Cloud Inference, to usher in a faster, more efficient wave of innovation for organizations looking to turn predictive and Large Language Models (LLMs) into real world action. Akamai Cloud Inference runs on Akamai Cloud, the world's most distributed network, to address escalating limitations of centralized cloud models. "Getting AI data closer to users and devices is hard and it's where legacy clouds struggle," said Adam Karon, Chief Operating Officer and General Manager, Cloud Technology Group, at Akamai. "While the heavy lifting of training LLMs will continue to happen in big hyperscale datacenters, the actionable work of inferencing will take place at the edge where the network Akamai has built over the past two and a half decades becomes vital for the future of AI and sets us apart from every other cloud provider in the market."

Akamai Sharpens Its AI Edge with Launch of Akamai Cloud Inference

New service gives companies the ability to realize a 3x improvement in throughput, 60% less latency, and 86% lower cost than traditional hyperscale infrastructure Akamai (NASDAQ: AKAM), the cybersecurity and cloud computing company that powers and protects business online, today unveiled Akamai Cloud Inference, to usher in a faster, more efficient wave of innovation for organizations looking to turn predictive and Large Language Models (LLMs) into real world action. Akamai Cloud Inference runs on Akamai Cloud, the world's most distributed network, to address escalating limitations of centralized cloud models. "Getting AI data closer to users and devices is hard and it's where legacy clouds struggle," said Adam Karon, Chief Operating Officer and General Manager, Cloud Technology Group, at Akamai. "While the heavy lifting of training LLMs will continue to happen in big hyperscale datacenters, the actionable work of inferencing will take place at the edge where the network Akamai has built over the past two and a half decades becomes vital for the future of AI and sets us apart from every other cloud provider in the market." AI Inference on Akamai Cloud Akamai's new solution provides tools for platform engineers and developers to build and run AI applications and data-intensive workloads closer to end-users, delivering 3x better throughput while reducing latency up to 2.5x. Using Akamai's solution, businesses can save up to 86% on AI inference and agentic AI workloads compared to traditional hyperscaler infrastructure. Akamai Cloud Inference includes: Compute: Akamai Cloud offers a versatile compute arsenal, from classic CPUs for fine-tuned inference, to powerful accelerated-compute options in GPUs, and tailored ASIC VPUs to provide the right horsepower for a spectrum of AI inference challenges. Akamai integrates with Nvidia's AI Enterprise ecosystem, leveraging Triton, Tao Toolkit, TensorRT, and NvFlare to optimize performance of AI inference on Nvidia GPUs. Data management: Akamai enables customers to unlock the full potential of AI inference with a cutting-edge data fabric purpose-built for modern AI workloads. Akamai has partnered with VAST Data to provide streamlined access to real-time data to accelerate inference-related tasks, essential to delivering relevant results and a responsive experience. This is complemented by highly scalable object storage to manage the volume and variety of datasets critical to AI applications, and integration with leading vector database vendors, including Aiven and Milvus, to enable retrieval augmented generation (RAG). With this data management stack, Akamai securely stores fine-tuned model data and training artifacts to deliver low latency AI inference at global scale. Containerization: Containerizing AI workloads enables demand-based autoscaling, improved application resilience, and hybrid/multicloud portability, while optimizing both performance and cost. With Kubernetes, Akamai delivers faster, cheaper, and more secure AI inference at petabyte-scale performance. Underpinned by Linode Kubernetes Engine - Enterprise, a new enterprise edition of Akamai Cloud's Kubernetes orchestration platform designed specifically for large-scale enterprise workloads, and the recently announced Akamai App Platform, Akamai Cloud Inference is able to quickly deploy an AI-ready platform of open-source Kubernetes projects, including Kserve, KubeFlow, and SpinKube, seamlessly integrated to streamline the deployment of AI models for inference. Edge compute: To simplify how developers build AI-powered applications, Akamai AI Inference includes WebAssembly (WASM) capabilities. Working with WASM providers like Fermyon, Akamai enables developers to execute inferencing for LLMs directly from serverless apps, enabling customers to execute lightweight code at the edge to enable latency-sensitive applications. Together, these tools create a powerful platform for low-latency, AI-powered applications that allows companies to deliver the experience their users demand. Akamai Cloud Inference runs on the company's massively distributed network capable of consistently delivering over one petabyte per second of throughput for data-intensive workloads. Comprising more than 4,100 points of presence across greater than 1,200 networks in over 130 countries worldwide, Akamai Cloud makes compute resources available from cloud to edge while accelerating application performance and increasing scalability. The Shift from Training to Inference As AI adoption matures, enterprises are recognizing that the hype around LLMs has created a distraction, drawing focus away from practical AI solutions better suited to solve specific business problems. LLMs excel at general-purpose tasks like summarization, translation, and customer service. These are very large models that are expensive and time-consuming to train. Many enterprises have found themselves constrained by architectural and cost requirements including datacenter and computational power, well-structured, secure, and scalable data systems, and the challenges that location and security requirements place on decision latency. Lightweight AI models, designed to address specific business problems, can be optimized for individual industries, leveraging proprietary data to create measurable outcomes, and represent a better return on investment for enterprises today. AI Inference Needs a More Distributed Cloud Gartner predicts that in 2025, 75% of data will be generated outside of centralized data centers or cloud regions. This shift is driving demand for AI solutions that leverage data generation closer to the point of origin. This fundamentally reshapes infrastructure needs as enterprises move beyond building and training LLMs, toward leveraging data for faster, smarter decisions and investing in more personalized experiences. Enterprises recognize that they can generate more value by leveraging AI to manage and improve their business operations and processes. Distributed cloud and edge architectures are emerging as preferable for operational intelligence use cases because they can provide real-time, actionable insights across distributed assets even in remote environments. Early customer examples on Akamai Cloud include in-car voice assistance, AI-powered crop management, image optimization for consumer product marketplaces, virtual garment visualization shopping experiences, automated product description generators, and customer feedback sentiment analyzers. "Training a LLM is like creating a map - requiring you to gather data, analyze terrain, and plot routes. It's slow and resource-intensive, but once built, it's highly useful. AI inference is like using a GPS, instantly applying that knowledge, recalculating in real time, and adapting to changes to get you where you need to go," explained Karon. "Inference is the next frontier for AI." # # # About Akamai Akamai is the cybersecurity and cloud computing company that powers and protects business online. Our market-leading security solutions, superior threat intelligence, and global operations team provide defense in depth to safeguard enterprise data and applications everywhere. Akamai's full-stack cloud computing solutions deliver performance and affordability on the world's most distributed platform. Global enterprises trust Akamai to provide the industry leading reliability, scale, and expertise they need to grow their business with confidence. Learn more at akamai.com and akamai.com/blog,or follow Akamai Technologies on X and LinkedIn.

Akamai Sharpens Its AI Edge with Launch of Akamai Cloud Inference

"Getting AI data closer to users and devices is hard, and it's where legacy clouds struggle," said Adam Karon, Chief Operating Officer and General Manager, Cloud Technology Group, at Akamai. "While the heavy lifting of training LLMs will continue to happen in big hyperscale datacenters, the actionable work of inferencing will take place at the edge, where the network Akamai has built over the past two and a half decades becomes vital for the future of AI and sets us apart from every other cloud provider in the market." Akamai's new solution provides tools for platform engineers and developers to build and run AI applications and data-intensive workloads closer to end-users, delivering 3x better throughput while reducing latency up to 2.5x. Using Akamai's solution, businesses can save up to 86% on AI inference and agentic AI workloads compared to traditional hyperscaler infrastructure. Akamai Cloud Inference includes: ● Compute: Akamai Cloud offers a versatile compute arsenal, from classic CPUs for fine-tuned inference to powerful accelerated-compute options in GPUs and tailored ASIC VPUs to provide the right horsepower for a spectrum of AI inference challenges. Akamai integrates with Nvidia's AI Enterprise ecosystem, leveraging Triton, Tao Toolkit, TensorRT, and NvFlare to optimize performance of AI inference on Nvidia GPUs. ● Data management: Akamai enables customers to unlock the full potential of AI inference with a cutting-edge data fabric purpose-built for modern AI workloads. Akamai has partnered with VAST Data to provide streamlined access to real-time data to accelerate inference-related tasks, essential to delivering relevant results and a responsive experience. This is complemented by highly scalable object storage to manage the volume and variety of datasets critical to AI applications, and integration with leading vector database vendors, including Aiven and Milvus, to enable retrieval-augmented generation (RAG). With this data management stack, Akamai securely stores fine-tuned model data and training artifacts to deliver low latency AI inference at global scale. ● Containerization: Containerizing AI workloads enables demand-based autoscaling, improved application resilience, and hybrid/multicloud portability, while optimizing both performance and cost. With Kubernetes, Akamai delivers faster, cheaper, and more secure AI inference at petabyte-scale performance. Underpinned by Linode Kubernetes Engine - Enterprise, a new enterprise edition of Akamai Cloud's Kubernetes orchestration platform designed specifically for large-scale enterprise workloads, and the , Akamai Cloud Inference is able to quickly deploy an AI-ready platform of open source Kubernetes projects, including Kserve, KubeFlow, and SpinKube, seamlessly integrated to streamline the deployment of AI models for inference. ● Edge compute: To simplify how developers build AI-powered applications, Akamai AI Inference includes WebAssembly (WASM) capabilities. Working with WASM providers like Fermyon, Akamai enables developers to execute inferencing for LLMs directly from serverless apps, enabling customers to execute lightweight code at the edge to enable latency-sensitive applications. Together, these tools create a powerful platform for low-latency, AI-powered applications that allows companies to deliver the experience their users demand. Akamai Cloud Inference runs on the company's massively distributed network capable of consistently delivering over one petabyte per second of throughput for data-intensive workloads. Comprising more than 4,100 points of presence across greater than 1,200 networks in over 130 countries worldwide, Akamai Cloud makes compute resources available from cloud to edge while accelerating application performance and increasing scalability.