In this article
  1. What happened
  2. Who can realistically run it
  3. Why it matters even if you never self-host
  4. What to do

TL;DR: On October 20, 2026, DeepSeek open-sourced DeepInfer, the inference framework it says powers DeepSeek-V4.1-Flash in production, under the Apache 2.0 license. Its announcement also highlighted deepseek-recipe, the code that turns API requests into model prompts and model output back into API responses, which DeepSeek says is aligned with the official DeepSeek API; it has been on PyPI since September 10, and version 0.2.0 came out October 20. DeepInfer supports only NVIDIA SM100 GPUs and Huawei Ascend 950 chips, and this release is offline inference only: DeepSeek lists a KV cache system and load balancing as “not included.” For most small teams, the DeepSeek API is still the practical way to use V4.1 Flash. The bigger win is for teams that host the model themselves or pick a provider that does.

What happened

DeepSeek announced the release in a four-post thread on X at 09:30 UTC on October 20 (5:30 p.m. Manila time), calling DeepInfer “our in-house inference framework, now fully open source” and saying it powers “DeepSeek-V4.1-Flash at scale.” The GitHub repository was created on October 16 and announced on October 20.

According to the DeepInfer README:

  • Model: DeepSeek-V4.1-Flash is the only supported model. The default checkpoint is DeepSeek-V4.1-Flash-1017 from Hugging Face, the same build that the deepseek-flash API runs today. DeepSeek’s thread says inference code for future DeepSeek models “will land here too.”
  • Hardware: NVIDIA SM100 GPUs or Huawei Ascend 950 NPUs. NVIDIA’s CUDA GPU list puts compute capability 10.0 on its B200 and GB200 data-center chips. The software needs Linux, GCC 13 or later, Python 3.12, and CUDA 13 (NVIDIA) or CANN 9.2 (Ascend). You build it from source.
  • Features: optimized kernels (FlashMLA, DeepGEMM, DeepEP and others), expert and data parallelism, speculative decoding with draft tokens, “SWA bounded replay” to cut prefill work, structured outputs (regex, JSON schema or EBNF grammar), and, on NVIDIA only, batch-invariant inference and deterministic sampling.
  • What’s missing: DeepSeek says a complete inference service also needs a KV cache system, VE/prefill/decode disaggregation (where “most required engine functionality is already implemented”) and “load balancing across data-parallel replicas,” and that these are “not included in this release.” It describes the release as “offline inference.”
  • Benchmarks: the README’s charts (the image above shows one of them) were measured on 4 SM100 GPUs for prefill and 64 for decode. These are DeepSeek’s numbers, and we haven’t reproduced them.

DeepSeek says DeepInfer “was originally built primarily on vLLM,” with ideas from SGLang and TensorRT-LLM.

deepseek-recipe is a set of Rust libraries with Python bindings (pip install deepseek-recipe, MIT license). It isn’t new: version 0.1.0 went up on PyPI on September 10, and version 0.2.0 was published on October 20. In DeepSeek’s words, it is “aligned with the processing logic behind the official DeepSeek API to bring the same API behavior to community deployments.” It converts Chat Completions, Responses and Messages requests into DeepSeek V4 and V4.1 prompts, then parses model output back into responses, including thinking content, tool calls, JSON and structured output. You still supply the model inference, the tool execution and the HTTP server yourself.

Who can realistically run it

Not most small teams. The DeepSeek-V4.1-Flash-1017 checkpoint files on Hugging Face add up to about 510 GB (our sum of the listed file sizes), DeepInfer’s example scripts default to spreading the model across 8 accelerators, and the supported chips are data-center hardware. The weights are MIT-licensed, so you’re allowed to self-host, but you need a multi-GPU Blackwell or Ascend 950 server, plus your own serving layer on top of this release.

Compare that with the API. DeepSeek’s pricing page lists deepseek-flash at $0.15 per million input tokens (cache miss) and $0.60 per million output tokens off-peak, and twice that during weekday peak hours (01:00-04:00 and 06:00-10:00 UTC, excluding Chinese public holidays). In our view, unless you have steady, high-volume traffic, paying per token will cost less than renting a multi-GPU server to run the model yourself; we haven’t priced GPU rentals for this story. Self-hosting makes more sense when the data has to stay on your own infrastructure, or when you already run GPU servers at high volume.

Why it matters even if you never self-host

Providers can match DeepSeek’s behavior. The V4.1 Flash model card says it doesn’t include a Jinja-format chat template, only a reference Python encoder, so each host builds its own request handling around the model. Small differences there can change how tool calls, thinking content or JSON output come back. deepseek-recipe is DeepSeek’s own production version of that layer. Its defaults show what “same behavior” means in practice: if a request doesn’t set max_tokens, it fills in 8,192 tokens outside thinking mode, 65,536 in thinking mode and 131,072 in thinking mode at max effort.

Deterministic runs help testing. On NVIDIA hardware, DeepInfer can give identical outputs for identical inputs and sampling settings. That makes before-and-after tests easier for teams that run their own evals.

What to do

  1. API users: nothing to do. DeepSeek’s announcement doesn’t change the deepseek-flash API, and its pricing page still lists V4.1-Flash-1017 at the rates above. We covered the current model and peak/off-peak prices in our story on DeepSeek’s move to V4.1-Flash-1017.
  2. If you use a third-party host for DeepSeek V4.1 Flash, ask whether it uses deepseek-recipe or DeepSeek’s own encoding code, then test the features you depend on (tool calls, JSON or schema output, thinking mode) against the official API before you switch traffic.
  3. If you plan to self-host, check that your hardware is on the supported list before you start, and plan to build or add the KV cache, load balancing and HTTP API layer that this release leaves out. If you serve images from user-supplied URLs with deepseek-recipe, note its warning: the image fetcher “connects to any address by default,” so install a DestinationFilter to block requests to internal addresses.
  4. If you want to run AI on a single machine instead, DeepInfer isn’t built for that. Smaller open models are, such as the one in our story on NVIDIA’s Nemotron 3.5 Bolt for local agents.

Sources & method

Sources

Method. Last checked October 20, 2026, at about 7:05 p.m. Manila time. Features, hardware, requirements, licenses, defaults and quotes come from DeepSeek’s X thread, GitHub READMEs, Hugging Face model card and API docs; the GPU model names come from NVIDIA’s list. The 510 GB figure is our sum of the file sizes Hugging Face lists for the checkpoint. We haven’t run DeepInfer or reproduced DeepSeek’s benchmarks. No affiliate links on this page. How we work.

Spotted an error? See our corrections policy and tell us.