DeepSeek Open-Sources DeepInfer Inference Framework to Slash AI Running Costs

DeepSeek Releases Production Inference Framework as Open Source

DeepSeek has open-sourced DeepInfer, the inference framework powering its large-scale production services, making it freely available on GitHub under the Apache 2.0 license1

1

2

2

. The Hangzhou-based Chinese AI startup announced the release on X, stating the goal is to help businesses reduce computational costs, lower energy consumption, and make AI model inference more affordable across the industry3

3

. DeepInfer currently supports the DeepSeek-V4.1-Flash model and runs on both Nvidia SM100 GPUs and Huawei Ascend 950 neural processing units4

4

. The company plans to add support for future models and additional hardware as they become available.

Cost Pressures Drive AI Infrastructure Innovation

The release comes as businesses face mounting pressure to control AI spending. Consulting giant Accenture recently capped employee Claude spending at $29 per month after previously pushing staff to maximize AI usage, according to 404 Media1

1

. Running AI models has become increasingly expensive as developers focus on boosting model agency and handling complex, multi-step tasks. A McKinsey & Company report found that by decade's end, the majority of data center compute will go toward inference rather than training new models1

1

. DeepSeek's V4.1-Flash is priced at just $0.035 per million input tokens and $0.12 per million output tokens, according to OpenRouter. In stark contrast, OpenAI's GPT-6 Astra costs $10 and $50 respectively1

1

. This dramatic price difference has created an opening for Chinese AI labs to capture market share from U.S. competitors.

Technical Capabilities and Performance Benchmarks

DeepInfer bundles optimized kernels with methods to distribute workloads across multiple chips efficiently2

2

. The framework employs speculative decoding through DSpark, where a draft model proposes five tokens simultaneously and a confidence head predicts which the main model will accept4

4

. This technique reduces draft tokens checked per step by approximately 30%, delivering a 4% to 7% throughput gain at production load4

4

. DeepSeek reports one cluster now serves 155 trillion tokens daily, generating revenue of $38 per GPU per hour at off-peak pricing4

4

. The framework also includes batch-invariant inference, deterministic sampling, and structured outputs, though some features work exclusively on Nvidia hardware3

3

. DeepSeek measured benchmarks using four SM100 GPUs for processing prompts and 64 for generating output, though no independent verification of these performance claims exists yet2

2

.

Built on Open-Source Foundations

DeepInfer was originally built primarily on vLLM, a widely used open-source inference engine, with additional inspiration from SGLang and Nvidia's TensorRT-LLM2

2

3

3

. The vLLM project thanked DeepSeek on X for crediting the open-source community and committed to improving its own support for V4.1-Flash using ideas from DeepInfer2

2

. DeepSeek also released deepseek-recipe alongside the framework, which converts requests into token sequences using the same logic as the company's official API4

4

5

5

. The release omits several components of a complete inference service, including KV cache management, prefill and decode disaggregation, and load balancing across data-parallel replicas3

3

. DeepSeek explained these systems are tightly coupled to internal infrastructure but published documentation on Zhihu explaining how they function4

4

.

Implications for Cloud Providers and Enterprises

Cloud and hosting companies running DeepSeek's models can now apply the same methods DeepSeek uses in production to cut their own costs3

3

. Companies deploying models on their own servers could also achieve significant savings, as the cost of serving a given model varies widely depending on the inference framework used3

3

. AI companies typically guard production techniques closely, making DeepSeek's decision to open-source its stack unusual3

3

. While the framework applies directly to DeepSeek's own models, outside developers can study the code and adapt techniques for their own systems3

3

. Open-weight models now account for 23% of spending on Vercel's AI Gateway, indicating growing adoption of alternatives to proprietary systems2

2

. Enterprises deploying generative AI solutions face inference costs reaching hundreds of millions annually, creating strong incentives to explore more efficient options5

5

.

Regulatory Response and Competitive Dynamics

The Department of Homeland Security issued an order Monday banning federal civilian agencies from acquiring DeepSeek products and requiring removal of the company's technology from government systems1

1

4

4

. The DHS ban cited DeepSeek as a privately owned company located in China presenting multiple threat vectors5

5

. However, the rule does not prohibit private companies or organizations from using DeepSeek's tools1

1

. DeepSeek's release intensifies competitive pressure on OpenAI and Anthropic to offer frontier AI capabilities without runaway costs that drive customers toward cheaper alternatives1

1

. Chinese AI labs can charge less than U.S. companies due to more efficient models and China's lower energy costs, according to the Financial Times5

5

. By Tuesday afternoon, the DeepInfer GitHub repository had accumulated more than 8,000 stars, signaling strong developer interest2

2

. Watch for how OpenAI and Anthropic respond to this escalating cost competition and whether more enterprises shift toward open-source alternatives despite federal restrictions.