From Github · · 4m
DeepSeek-V4.1 Flash: Extreme KV Cache Compression
DeepSeek's new model achieves 4x KV cache compression through architectural innovations, enabling faster long-context processing for AI agents.
In brief
DeepSeek's new model achieves 4x KV cache compression through architectural innovations, enabling faster long-context processing for AI agents. DeepSeek-V4.1 Flash uses a Causal Encoder-Decoder architecture combined with cross-layer cache sharing to compress KV cache by 4x while maintaining performance, making…
DeepSeek-V4.1 Flash arrives
DeepSeek released V4.1 Flash achieving 420 tokens per second. The technical report reveals it's a major architectural redesign, not just a post-training update, warranting comparison to a V5 release.
The KV cache problem
Long-context agent workflows create massive KV cache storage demands in memory and SSD. Tool calls add heavy prefill computation pressure, making scaling impossible without compression breakthroughs.
Four times smaller cache
V4.1 Flash compresses KV cache by 4x at runtime and 8x for persistent storage compared to V4-Flash, while maintaining high task completion quality. Storage grows only 890 bytes per token.
Model specs at a glance
V4.1 Flash has 552 billion parameters, 40 layers, 64 attention heads, and supports up to 1 million token contexts. It activates 8B parameters during prefill and 16B during decoding, optimizing for input-heavy agent tasks.
Three compression dimensions
Compression happens across channel dimension using 512-dimensional latent vectors, sequence dimension by merging adjacent positions, and layer dimension by sharing cache across multiple layers.
Causal Encoder-Decoder explained
The model splits into two 20-layer halves. The Encoder processes sequences efficiently, then the Decoder reuses the Encoder's KV cache via projection, eliminating redundant computation for long prompts.
Prefill computation halved
During prefill, most prompt positions only pass through the first 20 Encoder layers. The Decoder processes only recent tokens, reducing total prefill complexity from O(n²) to roughly O(n·d/2).
Bounded replay solves a problem
When new tokens arrive, the Decoder needs local KV from previous layers. Bounded replay reconstructs only recent positions instead of tracing back through the entire history, keeping overhead fixed.
CSA2 compression strategy
CSA2 combines sliding window attention with sparse retrieval and cross-layer cache reuse. It selects top-512 entries per query from compressed KV, balancing coverage and efficiency.
Three reuse modes
Full mode computes new KV and indices. Reindex mode reuses KV but computes new indices. Reuse mode reuses both KV and indices. This flexibility lets different layers share cache while varying attention patterns.
Recursive Transformer view
The architecture can be understood as recursive: each layer modifies queries while reusing the same KV cache, similar to iterating through a computation rather than stacking independent layers.
Vision processing pipeline
Images are split into 14-pixel patches, processed by a 32-layer ViT, then spatially rearranged to reduce tokens by 9x. Maximum 1024 visual tokens per image, supporting up to 1344x1344 resolution.
Multimodal load balancing
Image and text tokens have different representation distributions and routing preferences in the MoE. DeepSeek maintains separate expert correction biases for each modality to balance load within each type.
Engram memory injection
Engram injects n-gram patterns at layers 1 and 14, encoding repeated phrases and local regularities. It reads 2-gram, 3-gram, and 4-gram patterns from 16 million entry tables, gated by context.
DSpark speculative decoding
Three additional SWA-128 draft blocks use a smaller MoE with top-3 of 128 experts. A Markov head models dependencies and a confidence head decides verification length, accelerating generation.
Linear scaling within 1M tokens
As context grows to 1 million tokens, V4.1 Flash's computation grows almost linearly, far more efficiently than previous generations. This enables practical long-context agent deployments.
FP4 quantization for cache
KV cache uses FP4 precision instead of higher bit depths, further reducing storage without significant quality loss. Combined with structural compression, this achieves the 4x reduction target.
Encoder block structure
Each Encoder block has one Full mode CSA2 layer followed by five Reuse mode layers. This 1+5 pattern allows all five layers to share the same KV and indices while computing different queries.
Decoder block structure
The Decoder has five blocks: one Full mode block followed by four Reindex mode blocks. All use compression ratio 1 to maintain per-token addressability for generation, unlike the Encoder's 2x compression.
What matters most
DeepSeek-V4.1 Flash uses a Causal Encoder-Decoder architecture combined with cross-layer cache sharing to compress KV cache by 4x while maintaining performance, making long-context agent workflows practical.
