From SiliconANGLE · Duncan Riley · · 1m
DeepSeek Open-Sources DeepInfer Framework
DeepSeek releases open-source inference framework for cost-efficient large model deployment.
In brief
DeepSeek releases open-source inference framework for cost-efficient large model deployment. DeepSeek open-sources DeepInfer, an inference framework that serves 155 trillion tokens daily at $38 per GPU hour. Originally reported by SiliconANGLE.
DeepSeek Open-Sources DeepInfer
DeepSeek open-sources DeepInfer, the inference framework behind its production services.
“DeepSeek calls reaching a massive user base at a reasonable price "the core problem an inference framework has to solve"”
155 Trillion Tokens Daily
One cluster serves 155 trillion tokens a day at $38 per gpu hour off-peak pricing.
DeepSeek-V4.1-Flash Model
DeepInfer supports DeepSeek-V4.1-Flash, the updated 1017 version released over the weekend.
Nvidia and Huawei Hardware
Nvidia SM100 Blackwell GPUs and Huawei Ascend 950 neural processing units support DeepInfer.
Speculative Decoding
Speculative decoding runs through DSpark, trimming draft tokens by 30% and boosting throughput 4-7%.
Prefill Optimization
DeepInfer nearly halves prefill computation by running only the last 128 tokens of each prompt.
U.S. DHS Order
U.S. Department of Homeland Security ordered federal agencies to remove DeepSeek products from systems this week.
In short
DeepSeek open-sources DeepInfer, an inference framework that serves 155 trillion tokens daily at $38 per GPU hour.
