DeepSeek-V4.1-Flash’s technical report (arXiv 2609.19969, September 17, 2026) has a title that names the bottleneck outright: "Pushing the Limits of KV Cache Compression". This is a paper about keeping long context cheaper, not about another row in a results table.
The starting point is familiar to anyone running an agent: the load has moved to the input side. Long-horizon agents push enormous contexts through the model, so the cost sits not only in compute (prefill is still expensive) but in how much room the KV cache takes in HBM and on SSD, and how much data has to move across that medium. Those three things together – compute, capacity and bandwidth – are what currently blocks further cuts in deployment cost.
Causal Encoder-Decoder: 8B active on prefill, 16B on decode
The model has 552B backbone parameters (Mixture-of-Experts) and supports contexts up to one million tokens. The architectural novelty is Causal Encoder-Decoder (CED): 40 Transformer layers split into a 20-layer causal encoder followed by a 20-layer decoder.
The key difference from a classic stack: the decoder’s global KV cache is projected from the final encoder hidden states, not derived from each decoder layer’s own hidden states. Two things follow. There is no separate set of KV tensors to keep per decoder layer, and the model activates 8B parameters per token during prefill and 16B during decode. For input-heavy agentic workloads that asymmetry is exactly where the money is.
CSA2: three attention modes and an indexer that does not track context length
Compressed Sparse Attention 2 assigns each attention layer one of three static modes: Full, Reindex or Reuse. Layers share the main KV and the indexer K, and Top-K sparse-attention indices are reused across them instead of being recomputed for every layer.
In the decoder a Hierarchical Sparse Indexer restricts later indexing layers to a candidate pool built by the first Full-mode layer. That bounds the cost of deeper indexers independently of context length – indexing no longer scales with the number of tokens in the window.
FP4 KV cache and 890 bytes per token
The main KV cache is stored in FP4 (E2M1) with one E4M3 scale per 16 channels. Together with cross-layer reuse this brings the global KV cache (always in HBM) to 890 bytes per token – about a quarter of DeepSeek-V4-Flash and roughly 437 times less than the first generation of the V4 line.
In comparable units: a million tokens of context means about 0.83 GiB of global KV cache, against roughly 3.3 GiB for the predecessor (using their reported 4:1 relation). At 1M context across many concurrent sessions, that is the difference between a handful and a dozen-plus requests per GPU.
SWA Bounded Replay: window state does not have to hit the disk
The second mechanism concerns persistent memory. Instead of persisting sliding-window-attention (SWA) KV states to SSD, the model reconstructs the missing states by replaying only the most recent nwin tokens – hence the name. The persistent KV cache, on SSD or in host memory, drops to roughly 1/8 of what V4-Flash uses.
That is a deliberate trade: replaying a narrow window is cheaper than keeping its state and shuttling it between HBM, RAM and disk. When sizing bandwidth, remember the model is converting memory footprint into compute.
The rest of the architecture
- Single-Pass mHC – revised residual-stream mixing with an efficient Mega-mHC kernel.
- Engram – conditional memory of 196B parameters, sparsely accessed through token-based lookup rather than densely.
- DSpark – speculative decoding with semi-autoregressive draft generation and confidence-scheduled verification.
- MoE – 1 shared expert and 384 routed experts per layer, with 6 routed experts active per token.
- Vision – a DeepSeek-ViT encoder trained from scratch (2D-RoPE, 3×3 pixel-unshuffle downsampling) plus a two-layer MLP projector; images join pretraining from the start.
Pre-training: 45T tokens of multimodal corpus, sparse attention trained at a 64K sequence length, context extended to 1M tokens at 34T tokens. Post-training is the standard SFT, then RL, then on-policy distillation, with no algorithmic changes – the work went into the data pipeline, meaning large-scale automated synthesis of agent tasks and environments. The model also exposes continuously controllable reasoning effort (an integer from 1 to 100) to trade inference cost for accuracy.
Results
Base model against its predecessors:
| Benchmark | V4-Flash-Base (284B) | V4-Pro-Base (1.6T) | V4.1-Flash-Base (552B) |
|---|---|---|---|
| MMLU-Pro (5-shot) | 68.3 | 73.5 | 74.1 |
| SuperGPQA (5-shot) | 46.5 | 53.9 | 53.1 |
| SimpleQA-Verified | 30.1 | 55.2 | 42.3 |
| HumanEval (Pass@1) | 69.5 | 76.8 | 79.4 |
| BigCodeBench (Pass@1) | 56.8 | 59.2 | 60.6 |
| GSM8K (8-shot) | 90.8 | 92.6 | 93.0 |
| MATH (4-shot) | 57.4 | 64.5 | 61.1 |
| LongBench-V2 | 44.7 | 51.5 | 45.2 |
| MMMU-Pro (4-shot) | – | – | 56.5 |
| DocVQA (LLM-Judge) | – | – | 95.6 |
The takeaway: a 552B-parameter model beats a 1.6T one on several reasoning and code benchmarks (MMLU-Pro, HumanEval, GSM8K, BigCodeBench) but loses where broad factual knowledge dominates – SimpleQA-Verified 42.3 against 55.2, SuperGPQA 53.1 against 53.9, MATH 61.1 against 64.5. Knowledge still costs parameters, and those were deliberately not spent here.
After post-training, at maximum reasoning effort, the picture looks like this:
| Benchmark | Opus-5.0 | GPT-5.6 Sol | K3 | GLM-5.3 | V4-Pro | V4-Flash | V4.1-Flash |
|---|---|---|---|---|---|---|---|
| Terminal-Bench 2.1 | 89.1 | 88.8 | 88.3 | 88.2 | 87.9 | 82.7 | 90.6 |
| DeepSWE v1.1 | 74.0 | 73.0 | 67.5 | 66.9 | 62.7 | 54.4 | 74.2 |
| CyberGym | – | 84.5 | 80.0 | 84.5 | 83.3 | 76.7 | 88.1 |
| Codeforces (rating) | – | – | – | – | 3348 | 3289 | 3471 |
| AutomationBench | 50.3 | 45.8 | 46.7 | 48.8 | 43.2 | 37.7 | 54.8 |
| Agent’s Last Exam | 28.6 | 26.7 | 27.6 | 28.5 | 25.7 | 25.2 | 31.8 |
| HLE with tools | 63.6 | – | 59.8 | 62.5 | 60.0 | 51.5 | 63.9 |
The jump over V4-Flash is large: Terminal-Bench 2.1 from 82.7 to 90.6, DeepSWE from 54.4 to 74.2, CyberGym from 76.7 to 88.1. There are clear weak spots too: HLE 36.8 against Opus at 56.3 (39.1 on the text-only subset), Terminal-Bench 3.0 30.0 against 43.3, Terminal-Bench 4.0 31.2 against 51.8, ExploitGym 15.3 against 33.7 for GPT-5.6 Sol. On the newer, harder agentic sets the frontier still pulls away, and the authors themselves footnote that part of the HLE numbers covers text only.
One table deserves a look of its own: results by agent scaffold, because it shows how much the harness is worth. Terminal-Bench 2.1: DSH Minimal 90.6, mini-SWE 90.3, Claude Code 88.0, Pi 86.1, OpenCode 85.0, Codex 84.1. DeepSWE v1.1: mini-SWE 74.2, DSH Minimal 72.6, DSH Standard 70.5, Claude Code 69.8. The spread across scaffolds is several points – roughly the same gap that separates neighbouring models in the leaderboard table.
What an engineer should take from this
- Track bytes of KV cache per token, not just price per token. With long context and many sessions, that number, not the per-million rate in the price list, drives real cost.
- The prefill/decode asymmetry (8B against 16B) favours agentic workloads. If you pay for cache hits, it is worth re-running your own traffic mix through the new cost split.
- The weights are MIT licensed, the Hugging Face repo has passed 570k downloads, and the model card lists FP8 quantization – this can be run locally.
- Deployment caveat: the release ships no Jinja chat template. Instead there is a reference Python implementation (the
encoding/directory) and a set of Rust libraries called deepseek-recipe for prompt encoding. Recommended sampling: temperature 1.0, top_p 0.95, at least 256K output tokens.
We already wrote about this model in August, when V4-Flash-0731 came out – that post was about the numbers and how the smaller model caught up with Pro. Now we have the technical report that shows where those numbers come from architecturally, which is the more interesting part.
Note the direction here. The rest of the market races on benchmark points while DeepSeek works on what context costs to hold. For anyone running an agent on long context, that matters more today than another row in the results table.
This content was generated using artificial intelligence and reviewed by a human editor.
Sources
- DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compression, arXiv:2609.19969 (September 17, 2026) – https://arxiv.org/abs/2609.19969
- DeepSeek-V4.1-Flash model card (MIT license, evaluation results, architecture details) – https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash
- Technical report PDF – https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash/blob/main/DeepSeek_V41_Tech_Report.pdf
- Earlier post on the same line: DeepSeek-V4-Flash-0731: The Smaller Model That Beat Pro on Agentic Benchmarks
- China AI Weekly Brief, edition September 13-20, 2026 (job 3d03e2d4fae9)
Polish version: [PL] DeepSeek-V4.1-Flash: KV cache ścięty do 890 bajtów na token
Leave a Reply
You must be logged in to post a comment.