IBM Cloud lands $240M deal with Together AI for open-source inference cluster

IBM Cloud Secures Major AI Infrastructure Deal

IBM Cloud1

1

and Together AI2

2

have signed a $240 million multi-year agreement to build a large-scale AI inference cluster on IBM Cloud using Nvidia HGX B300 systems3

3

. The partnership positions IBM to compete on economics rather than scale against hyperscalers like Amazon, Microsoft and Google. Together AI CEO Vipul Ved Prakash stated that enterprises want the performance of frontier models without the closed-model price tag, and this only works if the infrastructure underneath is fast and reliable at scale4

4

. The cluster is expected to come online in the first quarter of 20271

1

.

Open-Source AI Models Drive Infrastructure Demand

The cluster will provide AI inference for open-source AI models including DeepSeek, MiniMax and Kimi2

2

, which have gained traction as businesses seek to rein in AI costs and address cybersecurity concerns involving models from Anthropic, OpenAI and Meta2

2

. Together AI operates a platform that lets companies train and run AI workloads on open weights models at lower costs than closed systems2

2

. The San Francisco-based company was valued at $8.3 billion in July2

2

and reportedly serves around 400 trillion tokens monthly3

3

, giving a sense of how much raw inference now flows through these systems.

Nvidia B300 GPUs Power Cost-Effective AI Infrastructure

The deployment will use Nvidia HGX B300 systems built on Nvidia Blackwell architecture, paired with Spectrum-X networking gear4

4

. The HGX B300 platform contains eight Blackwell Ultra chips per system and can be deployed in traditional air-cooled datacenters1

1

. These 14-15 kW boxes interconnect GPUs using NVLink within the box and Nvidia's Spectrum-X Ethernet fabrics between them1

1

. IBM stated this marks its first dedicated large-scale cluster built for inference applications using B300 systems4

4

. The HGX B300 is a motherboard equipped with eight Blackwell Ultra chips that can provide 15 petaflops of performance when processing data in the NVFP4 format5

5

.

Strategic Positioning Against Cloud Giants

IBM's partnership represents a strategic bet that AI's real profits lie in cheap inference rather than ever-larger training runs3

3

. Together AI selected IBM and Nvidia based on their product roadmaps and ability to deliver GPU capacity at the pace required for AI scaling at low token cost4

4

. The move lets IBM compete on economics rather than size, positioning IBM Cloud as the place to run open models affordably3

3

. This dovetails with a wider European appetite for infrastructure that is not locked to a single American giant and supports data sovereignty concerns3

3

.

Inference Becomes Primary Driver of Compute Capacity

Inference has become one of the largest drivers of demand for computing capacity, prompting cloud providers and chipmakers to spend billions expanding AI infrastructure2

2

. Together AI will use the cluster to host AI workloads for customers through five different inference services including media generation models, Provisioned Throughput charged by token, serverless infrastructure, dedicated machines and batch processing5

5

. The platform processes about 400 trillion tokens per month for customers5

5

. This partnership follows Together AI's recent $800 million Series C funding round that included Nvidia among investors4

4

.

Broader IBM Commitment to Open-Source Ecosystem

The deal aligns with IBM's broader push into open-source AI infrastructure through initiatives like Project Lightwell, a $5 billion commitment with Red Hat to help enterprises secure open-source software using AI tools and more than 20,000 engineers4

4

. That initiative centers on a trusted enterprise clearinghouse using AI to vet and verify patches across the open-source ecosystem, with pilot participants including Bank of America, Goldman Sachs and JPMorgan Chase4

4

. The IBM and Together AI deal is part of a broader collaboration between IBM and Nvidia spanning GPU-native data analytics, unstructured data extraction, on-premises and cloud infrastructure, and consulting services4

4

. For banks and hospitals, the ability to keep sensitive data on infrastructure they control through fine-tuning and sovereign AI approaches is often the whole point3

3

.