From BigGo Finance · BigGo Finance · · 1m 40s
DeepSeek Open-Sources Inference Framework DeepInfer
DeepSeek releases open-source AI inference framework with $38 hourly revenue per GPU.
In brief
DeepSeek releases open-source AI inference framework with $38 hourly revenue per GPU. DeepSeek open-sources DeepInfer framework, achieving $38 hourly revenue per GPU at off-peak pricing for Nvidia and Huawei hardware. Originally reported by BigGo Finance.
DeepSeek Open-Sources Inference Framework
DeepSeek fully open-sourced its internal inference framework DeepInfer under Apache-2.0 license and token codec deepseek-recipe under MIT license.
Hardware Support and Pricing
DeepInfer supports Nvidia SM100 GPUs and Huawei Ascend 950 NPUs with CUDA 13 and CANN 9.2 environments.
Daily Token Processing
DeepSeek V4.1-Flash processes 155 trillion tokens per day on a single cluster.
Hourly Revenue per GPU
At off-peak pricing, DeepInfer generates $38 hourly revenue per GPU.
Gross Margin Estimates
Third-party researcher estimates hardware-level inference gross margins of 84.2% to 96.1%.
Open-Source Limitations
The open-source release covers only offline inference; online service requires KV cache, phase disaggregation and load-balancing.
vLLM Benchmark Results
vLLM benchmarks show decoding throughput within 6% of DeepInfer's published figures.
Hardware Performance Differences
All benchmarks published on Nvidia GPUs; Huawei Ascend 950 lacks performance data.
Revenue Calculation Basis
DeepSeek's pricing model shows input prices vary by cache hit or miss.
Industry Impact Analysis
Custom LPU hardware requires stacking boards for large models, increasing costs.
Cost-Reduction Debate
Software scheduling on GPUs and NPUs expands profit room for large-scale API services.
Deployment Barriers
Enterprise users can study but deployment requires additional components.
vLLM's Performance Update
vLLM achieves 1 million tokens per second on GB300 with $38.96 hourly revenue per GPU.
DeepSeek's Motivation
DeepSeek aims to help developers optimize inference throughput and reduce costs.
What matters most
DeepSeek open-sources DeepInfer framework, achieving $38 hourly revenue per GPU at off-peak pricing for Nvidia and Huawei hardware.
