DeepSeek-V4.1-Flash's KV-Cache Compression, Checked Down to the Byte: 890 Bytes per Token Is a Design, Not a Miracle

Long-horizon agents broke the economics of transformer serving in a specific, mechanical way: they are input-heavy. A coding agent that reads a repository, calls tools, and iterates prefills hundreds of thousands of tokens and generates comparatively few — so the cost per served session is dominated by KV-cache bytes stored, moved, and kept warm across turns. DeepSeek's answer is DeepSeek-V4.1-Flash¹: a 552B-backbone (plus 196B memory-module) multimodal MoE with 1M-token context whose headline number is a global KV cache of 890 bytes per token — roughly 1/4 of DeepSeek-V4-Flash at the same sequence length in HBM, and roughly 1/8 in persistent storage. The blog frames it as a cost story for agents; the paper frames it as a systems co-design story². Both framings are true, and neither is the interesting part. The interesting part is that 890 bytes is not one trick — it is the product of four independent compression axes (a restructured encoder-decoder split, three-way cross-layer attention reuse, 4-bit quantization with a norm argument, and a deployment trade that stops persisting sliding-window state entirely), each of which is individually known prior work, composed with unusual discipline and trained-in from the first token. This guide takes each axis apart down to the arithmetic, checks whether the published configuration actually reproduces the headline number, and identifies the places where the composition could break. This article ships with an interactive architecture diagram of the full inference stack.

1. Why KV bytes price agents

Decode in a transformer is a bandwidth-bound problem: each generated token requires re-reading the KV cache from HBM, and the FLOPs that the read enables are trivially few compared to the bytes moved. Cutting cache bytes is therefore cutting decode latency almost linearly — but for agents the bigger constraint is capacity, not just bandwidth. Weights for this model are large in aggregate (552B backbone parameters) but only 8B activate per token in prefill and 16B in decode; the per-request KV cache is what multiplies with concurrency. At the claimed 890 B/token, a full 1M-token context costs roughly 933 MB of HBM (890 B x 1,048,576 tokens — our computation in cell 2 below); the V4-Flash-equivalent baseline at 4x would want roughly 3.7 GB per context. The difference is the difference between tens and hundreds of concurrent million-token sessions per node — the number that determines whether agent workloads are servable at all outside of a lab.

Persistence is the second half of the economics. A multi-turn agent reuses its prefix across turns, so the cache has to survive between turns: in V4, global KV and sliding-window (SWA) KV were both persisted, governed by LRU eviction, with residency configured north of 72 hours — and SWA KV alone accounted for nearly half of the persistent cache. For state whose actual reuse window is minutes inside a live session, billing it SSD residency on a 72-hour timescale is a mismatch; V4's alternative ("Zero SWA Caching", exact recomputation) was, per the paper's own admission, too expensive in production, because exact recovery of SWA state across L layers requires replaying L x n_win tokens. V4.1's move — which we price in section 5 — is to stop persisting SWA KV entirely, hold it in a host-DRAM pool provisioned from 10% of host memory with a TTL of minutes, and make misses survivable with a bounded approximate replay. The design lives or dies on that replay being both cheap and harmless; both properties are checkable, and the paper itself flags the second one as not mathematically guaranteed.

One deflation before the mechanics start. The 1/4 and 1/8 comparisons are against DeepSeek's own V4-Flash — an already heavily cache-optimized baseline — not against a generic dense transformer, for which the ratio would look far more dramatic (the paper separately cites a 437x per-token global-KV reduction versus V1 in its Figure 1, which is cross-generational marketing-adjacent arithmetic, not a controlled comparison). And the 890 counts global KV only: per-layer SWA KV in FP8 sits on top of it, bounded by the window rather than by sequence length, which is real but is not the same as "890 bytes and nothing else." Our computations in this guide, beyond the 890 itself, all sit under an explicit assumption label.

2. CED: prefill becomes half a model, and SWA refuses to cooperate

The first axis is architectural. V4.1-Flash splits its 40 causal transformer layers into a 20-layer causal encoder and a 20-layer decoder (first two layers are SWA-only; the remaining 18 encoder layers and all 20 decoder layers carry the sparse global branch). The decisive move, inherited in spirit from YOCO³: the decoder's global KV entries are not derived from decoder hidden states at all. Each decoder layer projects its global K and V from the final encoder hidden state with layer-dependent weights — C_l = H_(L/2) W_KV^l and Z_l = H_(L/2) W_Z^l for layers above L/2, where the paper's C are the main KV entries and Z their compression weights. One shared representation, 20 cheap per-layer projections.

The payoff is prefill asymmetry. Because the decoder-half's global cache is a projection of a quantity the encoder-half already computed, prefill complexity drops from O(N x L) to O(N x L/2 + n_win x L/2), which the paper rounds to "effectively halving the overall computation." That is also where the 8B-prefill / 16B-decode activation split comes from: prefill mostly runs the encoder half, decode runs everything. For input-heavy agent workloads — prefill tokens outnumbering generated tokens by orders of magnitude — that is the correct side to make cheap, and it is the single biggest cost asymmetry in the design.

What does not cooperate is the SWA branch. CED keeps sliding-window attention conventional and layer-wise across all layers: each layer's local K/V derive from that layer's own hidden state. This is a deliberate choice (it preserves the computational depth of local KV generation, and stacked local windows are what give SWA its effective multi-layer receptive field), but it has a structural consequence: decoder SWA KV cannot be projected from the encoder — it requires running the decoder layers themselves, which means the encoder shortcut does not close the loop. Prefilling decoder SWA KV exactly requires processing an additional n_win x L/2 tokens, and exact reconstruction of SWA state after a cache miss requires replaying L x n_win tokens — the term that made V4's Zero SWA Caching uneconomical. This is the hook for everything in section 5. The paper leans on a result by Chen et al. (2025) — that SWA's effective receptive field is much smaller than the theoretical n_win x L/2 — to justify bounding that replay; note the lineage: the approximation is licensed by prior measurement, not by anything in V4.1's own training.

3. CSA2: three multiplicative dimensions, three modes, one cadence

The second axis is CSA2, and the clearest framing of its contribution is this: every axis is prior work. Entry-size compression is GQA and MLA; sequence compression (m tokens per entry) is V4's own CSA; layer compression is cross-layer attention. The paper's own related-work section names the ancestors: IndexCache reuses Top-K indices across layers⁴, YOCO shares caches across decoder halves, CLA-based work (Brandon et al., 2024) shares KV across layers. What was missing, and what CSA2 actually claims, is all three dimensions jointly — with cache sharing and index reuse decoupled, and statically assigned so deployment knows exactly what to prefetch.

Each CSA2 layer operates in one of three modes:

• Full Mode computes its own main KV and indexer Q, projects indexer K from that main KV (one less compression path than V4's CSA, which derived indexer K from hidden states separately), and produces fresh Top-K indices.
• Reindex Mode reuses the preceding layer's main KV and indexer K, computes its own indexer Q, rescores, and produces fresh Top-K.
• Reuse Mode reuses main KV and the latest Top-K indices and performs no indexer computation at all.

Every layer, in every mode, keeps its own global Q and SWA KV — the shared quantities are the expensive persistent ones. The static assignment cadences from the config table: the 18 encoder CSA2 layers run at compression m=2 (two tokens per entry) in three groups of six — one Full, five Reuse. The 20 decoder layers run at m=1 in five groups of four: the first group is Full + 3 Reuse, the remaining four are Reindex + 3 Reuse. The decoder cadence is where the residual risk concentrates — selection enters the cache at exactly five layers (one Full, four Reindex), and every Reuse layer downstream consumes it unverified. Compared to V4, the simplifications are real: the overlapping source entries and absolute positional embedding in the CSA compressor are gone, the indexer K projection is unified, and V4's CSA-HCA hybrid is replaced by pure CSA2.

The remaining indexer cost is bounded by the Hierarchical Sparse Indexer (decoder only, applied identically in training and inference). The first Full layer scores the full causally visible range and does blockwise candidate selection — a maximum of 2,048 blocks at 8 positions each, a pool of up to 16,384 candidate positions. Later Reindex layers score only that pool, so their per-query indexer cost is constant, independent of context length. Priced at the limit:

# Cell 4: Hierarchical Sparse Indexer cost bound at 1M context
H, D = 32, 128            # indexer query heads, indexer head dim
S = 1_048_576             # 1M-context visible range
POOL = 2048*8             # 2048 blocks x 8 positions
full_mac = H*D*S
pool_mac = H*D*POOL
print(f"full-range indexer scoring @1M: 32 heads x 128 dims x {S:,} positions = {full_mac:,} MACs")
print(f"candidate-pool scoring:         32 heads x 128 dims x {POOL:,} pool     = {pool_mac:,} MACs")
print(f"pool is {100*POOL/S:.2f}% of the full range -> later Reindex indexers cost is")
print(f"constant per query, independent of context ({full_mac/pool_mac:.1f}x cheaper than full scan)")
print(f"the first Full layer still scans all {S:,} positions: {full_mac:,} MACs, unavoidable")

full-range indexer scoring @1M: 32 heads x 128 dims x 1,048,576 positions = 4,294,967,296 MACs
candidate-pool scoring:         32 heads x 128 dims x 16,384 pool     = 67,108,864 MACs
pool is 1.56% of the full range -> later Reindex indexers cost is
constant per query, independent of context (64.0x cheaper than full scan)
the first Full layer still scans all 1,048,576 positions: 4,294,967,296 MACs, unavoidable

(Our computation, /usr/bin/python3, from the published indexer config: 32 heads, head dim 128, top-k 512, pool 2,048 x 8.) The 1.56% figure is the entire trick: the pool is a fixed budget, so deeper indexers stop caring whether the context is 4K or 1M. The cost does not vanish — it concentrates in the first Full layer, which at 4.29 billion MACs per decoder-side query set at 1M context is itself far from free — and the pool's quality is now a single point of failure for every Reindex layer above it. This is worth the 5th cell of skepticism: hierarchical pooling makes the average query cheaper by making the pool-selection query mandatory and unverifiable.

4. FP4: the norm argument, and hunting the 890-byte ledger

The third axis is storage precision. V4.1-Flash stores its main KV cache in MXFP4 — the OCP-standard format, E2M1 payloads with one E4M3 scale per 16 channels, following NVFP4's scheme but deliberately omitting NVFP4's second-level global scale⁵. The justification is a norm argument worth writing out in full, because it is the rare quantization decision that is derived rather than benchmarked:

1. The largest trained RMSNorm weight magnitude in the model is approximately 1.
2. After RMS normalization, the L2 norm of the 512-channel KV latent is therefore bounded by sqrt(512) ≈ 22.6.
3. RoPE is a rotation; rotations preserve L2 norms — so the maximum absolute channel value after rotation is also bounded by ~22.6.
4. The format represents magnitudes up to E2M1's 448 times E4M3's largest scale 6 = 2688.
5. 2688 / 22.6 ≈ 118.8x headroom against the norm bound — and 2688 / 10 ≈ 268.8x against the ~10 maximum actually observed during training. No global scale needed; the format has range to spare.

# Cell 3: FP4 range analysis (MXFP4 E2M1 + per-16 E4M3 scale, no global scale)
e2m1_max, e4m3_scale_max = 448.0, 6.0
quant_max = e2m1_max*e4m3_scale_max
print(f"representable max magnitude: E2M1 448 x E4M3 scale 6 = {quant_max:.0f}")
import math
norm_bound = math.sqrt(512)
print(f"L2 norm bound of 512-ch latent after RMS norm (largest weight ~1): sqrt(512) = {norm_bound:.3f}")
print(f"RoPE preserves L2 norm -> max |value| after rotation <= {norm_bound:.3f}")
print(f"headroom vs norm bound: {quant_max/norm_bound:.1f}x")
print(f"headroom vs observed max magnitude ~10 during training: {quant_max/10:.1f}x")
print()
bits_fp4 = 4 + 8/16      # E2M1 4 bits + one E4M3 scale byte per 16 channels
bits_fp8 = 8
print(f"effective bits/element: FP4 {bits_fp4} vs FP8 {bits_fp8} -> {bits_fp8/bits_fp4:.2f}x ('nearly halves')")
ch = 512
per_tok_fp4 = ch*bits_fp4/8 ; per_tok_fp8 = ch*bits_fp8/8
print(f"per-token main K (or V), 512 ch: FP4 {per_tok_fp4:.1f} B vs FP8 {per_tok_fp8:.0f} B")
print(f"per-token K+V: FP4 {2*per_tok_fp4:.1f} B vs FP8 {2*per_tok_fp8:.0f} B ({2*per_tok_fp8/(2*per_tok_fp4):.2f}x)")

representable max magnitude: E2M1 448 x E4M3 scale 6 = 2688
L2 norm bound of 512-ch latent after RMS norm (largest weight ~1): sqrt(512) = 22.627
RoPE preserves L2 norm -> max |value| after rotation <= 22.627
headroom vs norm bound: 118.8x
headroom vs observed max magnitude ~10 during training: 268.8x
 
effective bits/element: FP4 4.5 vs FP8 8 -> 1.78x ('nearly halves')
per-token main K (or V), 512 ch: FP4 288.0 B vs FP8 512 B
per-token K+V: FP4 576.0 B vs FP8 1024 B (1.78x)

(Our computation, /usr/bin/python3, from the paper's Section 2.4.4 statements.) Three deployment-relevant details the abstract does not carry: FP4 here is a storage format, not a compute format — values are dequantized before attention, so no native FP4 matmul is required and the format choice does not bind the accelerator roadmap; quantization happens after RoPE because pre-RoPE quantization bought only marginal accuracy while adding decode overhead; and the SWA KV cache stays FP8, explicitly because of its quantization sensitivity. The norm argument is therefore a claim about the global branch only — and it is exactly the kind of argument that fails if a future training run produces RMSNorm weights meaningfully above 1, which would tighten the bound the whole format rests on.

Now the adversarial question: does the config actually reproduce 890 bytes per token? The paper publishes the per-component dims but no byte ledger for the 890 — the number appears in the abstract and Figure 1 with no decomposition. We can, however, enumerate candidate accountings from the config and check which comes close. Per token-side (K or V) of one uncompressed 512-channel layer, MXFP4 costs 0.5 B/channel plus one scale byte per 16 channels: 512 x 0.5 + 512 x 1/16 = 288 B, matching cell 3's output. From there:

# Cell 1: hunting for a byte ledger consistent with the paper's 890 B/token global KV
FP4_B, SCALE_B, CH = 0.5, 1.0/16, 512    # E2M1: 0.5 B/channel + one E4M3 scale byte per 16 ch
side = CH*FP4_B + CH*SCALE_B             # one token-side (K or V) of ONE uncompressed layer
print(f"per token-side (512 ch), FP4 MXFP4 incl. scales: {side:.0f} B")
pair = 2*side                            # K + V of one uncompressed layer
print(f"K+V of one uncompressed layer: {pair:.0f} B")
 
# encoder: 18 CSA2 layers, m=2 -> entries cover 2 tokens; 3 groups of 6 share one main KV each
enc_group = pair/2                       # m=2 halves per-token cost of the shared tensor
enc = 3*enc_group
print(f"encoder: 3 groups x {enc_group:.0f} B = {enc:.0f} B/token")
 
# candidate A: encoder-only accounting (all decoder global KV somehow amortized/shared)
print(f"candidate A (encoder only): {enc:.0f} B vs paper 890 B -> {890/enc*100-100:+.1f}% off")
 
# candidate B: encoder + ONE decoder main KV (Full layer, m=1, group 1)
decB = enc + pair
print(f"candidate B (enc + 1 decoder main KV): {decB:.0f} B -> {890/decB*100-100:+.1f}% off")
 
# candidate C: encoder + decoder group-1 main KV at FP4 with K/V sharing one latent (512 ch total)
latC = CH*FP4_B + CH*SCALE_B             # 512-ch latent shared by K and V post-projection
decC = enc + latC
print(f"candidate C (enc + 1 decoder 512-ch latent): {decC:.0f} B -> {890/decC*100-100:+.1f}% off")
 
# candidate D: 3 enc groups + 1 shared decoder latent + shared indexer K
# indexer: 32 heads x 128 dim = 4096 ch, FP4, per-16 scales, shared cross-layer
idx = 4096*FP4_B + 4096*SCALE_B
decD = enc + latC
print(f"indexer K, uncompressed FP4 (32x128): {idx:.0f} B -> too large alone; skip in D")
print(f"candidate D (= C): {decD:.0f} B")
print()
print("closest consistent read: 864 B (encoder 3 x 288) is 3.0% below 890;")
print("the residual 26 B/token is not attributable from published config alone.")
print("The paper publishes no per-tensor byte ledger for the 890; all of the")
print("above are OUR assumption-dependent computations from the config table.")

per token-side (512 ch), FP4 MXFP4 incl. scales: 288 B
K+V of one uncompressed layer: 576 B
encoder: 3 groups x 288 B = 864 B/token
candidate A (encoder only): 864 B vs paper 890 B -> +3.0% off
candidate B (enc + 1 decoder main KV): 1440 B -> -38.2% off
candidate C (enc + 1 decoder 512-ch latent): 1152 B -> -22.7% off
indexer K, uncompressed FP4 (32x128): 2304 B -> too large alone; skip in D
candidate D (= C): 1152 B
 
closest consistent read: 864 B (encoder 3 x 288) is 3.0% below 890;
the residual 26 B/token is not attributable from published config alone.
The paper publishes no per-tensor byte ledger for the 890; all of the
above are OUR assumption-dependent computations from the config table.

(Our computation, /usr/bin/python3; all candidates assumption-dependent.) The honest summary: an encoder-only reading — three cross-layer-shared main-KV tensors at m=2, 288 B each — lands at 864 B/token, 3.0% below the published 890; the residual 26 B/token plausibly belongs to some decoder-side contribution or compressed indexer share that the config table does not let us pin down, and any of the straightforward decompositions that keep a full uncompressed decoder tensor overshoot badly (1,152-1,440 B). So we can say the 890 is consistent with the architecture as described, within a few percent, under a cross-layer-sharing read of the encoder — but the exact ledger is underdetermined by the paper, and anyone budgeting an HBM pool off that number should carry a ±5% margin and an SWA-KV surcharge on top. We are not aware of any other published attempt at this decomposition.

5. SWA Bounded Replay: turning a catastrophic miss into a cheap one

The fourth axis is a deployment trade, and it is the one most likely to be underestimated. SWA dependencies accumulate across layers: exactly reconstructing the SWA KV of L layers after a miss requires replaying L x n_win tokens — with L = 40 layers and n_win = 128, that is 5,120 tokens of full-depth recompute. V4's own Zero SWA Caching proposal attempted exactly this recovery and, per the paper, proved too expensive in production. Bounded Replay gives up exactness: replay only the most recent n_win = 128 tokens, truncating SWA to the replay segment — a query at position i attends to SWA keys in [max(s, i-W+1), i] for a replay started at s. The arithmetic:

# Cell 2: SWA Bounded Replay arithmetic (L=40 layers, n_win=128)
L, W = 40, 128
exact, bounded = L*W, W
print(f"exact SWA reconstruction: L x n_win = {L} x {W} = {exact:,} tokens")
print(f"bounded replay:            n_win    = {W} tokens   ({exact/bounded:.0f}x cut)")
dec_exact = (L//2)*W
print(f"decoder-half exact:        (L/2) x n_win = {L//2} x {W} = {dec_exact:,} -> bounded {W} ({dec_exact/W:.0f}x cut)")
print()
print("replay overhead as % of prefill compute (our computation, naive token-count model):")
for P in (1_000, 8_000, 65_536, 524_288, 1_048_576):
    print(f"  prompt {P:>9,}: bounded {100*W/P:6.3f}%   V4-exact {100*exact/P:7.2f}%")
print()
# cache sizes at 1M tokens
B = 890
mb = B*1_048_576/1e6 ; mib = B*1_048_576/2**20
print(f"global KV @1M tokens: {B} B/token x 1,048,576 = {B*1_048_576:,} B = {mb:.0f} MB = {mib:.1f} MiB")
print(f"V4-Flash-equivalent global KV (4x): {4*mb:.0f} MB ({4*mib:.1f} MiB)")
print()
# how many 1M contexts fit in an 80-GiB HBM remainder (our hardware assumption)
for inst in (80.0, 60.0, 40.0):
    cap = inst*2**30
    n = cap/(B*1_048_576)
    print(f"80→{inst:.0f} GiB HBM pool (weights/activations already resident): {n:.1f} million-token contexts")

exact SWA reconstruction: L x n_win = 40 x 128 = 5,120 tokens
bounded replay:            n_win    = 128 tokens   (40x cut)
decoder-half exact:        (L/2) x n_win = 20 x 128 = 2,560 -> bounded 128 (20x cut)
 
replay overhead as % of prefill compute (our computation, naive token-count model):
  prompt     1,000: bounded 12.800%   V4-exact  512.00%
  prompt     8,000: bounded  1.600%   V4-exact   64.00%
  prompt    65,536: bounded  0.195%   V4-exact    7.81%
  prompt   524,288: bounded  0.024%   V4-exact    0.98%
  prompt 1,048,576: bounded  0.012%   V4-exact    0.49%
 
global KV @1M tokens: 890 B/token x 1,048,576 = 933,232,640 B = 933 MB = 890.0 MiB
V4-Flash-equivalent global KV (4x): 3733 MB (3560.0 MiB)
 
80→80 GiB HBM pool (weights/activations already resident): 92.0 million-token contexts
80→60 GiB HBM pool (weights/activations already resident): 69.0 million-token contexts
80→40 GiB HBM pool (weights/activations already resident): 46.0 million-token contexts

(Our computation, /usr/bin/python3; the GiB rows assume an 80-GiB-class HBM node with 40-80 GiB left after weights, activations, and Engram shards — that hardware split is our assumption, not the paper's.) Two reads fall out. First, the trade inversion: under exact recovery, replay cost scaled with the architecture (5,120 tokens, 512% of a 1K prompt's prefill — hence "prohibitively expensive"); under bounded replay it is a constant 128 tokens, which is 12.8% of a 1K prompt's prefill but 0.012% at 1M. That is a new storage-computation trade-off point in the design space, and it is what makes the pool deletion financeable. Second, the capacity arithmetic from section 1 repeats here concretely: at 933 MB per million-token context, 92 concurrent such contexts fit in a nominal 80-GiB HBM pool versus roughly 23 at the V4-Flash-equivalent 3.7 GB — the concurrency limiter moves by a factor of four.

The deployment shape has three moving parts. Encoder SWA Bounded Replay makes prefix caching depend only on global KV — on a global-hit/SWA-miss, the last 128 tokens of the cached prefix are replayed, regenerating only SWA KV while the cached global KV is reused without recomputation. Decoder SWA Bounded Replay bounds the decoder forward to 128 tokens, feeding the replay tokens' encoder outputs through the decoder under the same truncation — the paper states this "nearly halves total prefill computation", consistent with the CED O(N x L/2) analysis. And the DRAM pool: SWA KV leaves persistence entirely, living in a distributed host-memory pool provisioned from 10% of host DRAM per machine, with a TTL of minutes and immediate recycling for new sessions, while global KV keeps its guaranteed 72-hour persistent lifetime. The paper's own words deserve the last slot here: this bounded replay is the cornerstone — it turns a catastrophic miss into a graceful, inexpensive degradation.

The caveat, also the paper's own: the replayed state is approximate, and the global and SWA KV computed for the uncached suffix depend on the cache-hit position — the same prompt resumed at different cache positions produces states that are not mathematically identical. The paper reports experimental evidence that degradation is negligible, and it simulated the replay during post-training so the model adapted to it. That is an empirical license for a mathematical approximation — exactly the kind of thing that holds until it does not, which is the subject of section 7.

6. The supporting cast

Four further components complete the efficiency story, each worth one paragraph of engineering respect:

• Single-Pass mHC. The multi-head-convolution input-mixing design is upgraded so the input-mixing coefficients shift by one block — each block reuses the previous block's coefficients for its input while predicting its own for the next. That shift makes the whole pipeline fusible: the Mega-mHC deployment kernel fuses residual update, input mixing, and coefficient prediction into one pass with (2n+2)d activation reads/writes versus (3n+2)d for the mHC form — halving activation memory traffic versus the original four-kernel implementation while pretraining keeps the multi-kernel path. The mHC expansion factor is 4, with 20 Sinkhorn-Knopp iterations.
• Engram. A 196B-parameter sparse conditional memory module (two modules at layers 1 and 14, zero-indexed), with 8 hash heads, N-gram orders 2/3/4, total embedding dimension 2048 per order, ~16M-entry tables at distinct prime sizes, FP8 tables and projections. The short causal convolution from the original design is omitted as not worth the complexity at inference; deterministic addressing lets embeddings be prefetched from host memory over RDMA, overlapping the first module's prefetch with the first transformer block. It decouples memorization from computation — and lands 196B of its own parameters outside the 552B backbone.
• DSpark. Speculative decoding with three transformer blocks drafting under a 128-token sliding window, five draft positions per pass in parallel, a Markov head for draft-token dependencies, and a confidence head predicting per-position acceptance. The scheduler picks verification length from profiled engine-throughput curves to maximize system-wide token throughput under load. Trained in a dedicated post-pretraining stage with the backbone frozen, and kept aligned during post-training without DSpark gradients flowing into the backbone — replacing V3's jointly-trained MTP module. Note what the confidence scheduling actually is: speculative decoding retuned as a cluster-level throughput policy, not just a per-request latency trick.
• Kernel and topology accounting. Reuse-Mode layers execute with only 15 kernels in prefill and 11 in decode — in memory-bound execution, launch counts and fused reads are latency. MoE is 1 shared + 384 routed experts, 6 active per token, expert intermediate dim 2304, SwiGLU clamped at 10, with modality-specific auxiliary-loss-free balancing (separate text/image correction biases, update speed 0.001, small sequence-level balance loss at 0.0001). Deployment is EPD-disaggregated — encoder, prefill, and decode pools scale independently. The vision stack (DeepSeek-ViT trained from scratch, 2D-RoPE, linear patch projection for Muon compatibility, pixel-unshuffle reducing visual tokens by a factor of 9, 32 layers, hidden 1024, patch 14, plus a 2-layer MLP projector at hidden 5120, multimodal from the start of pretraining) feeds the same cache machinery.

Pretraining context, for the training-in-thesis check: 45T multimodal tokens, sparse attention from scratch at 64K with no dense warmup, extended to 1M at 34T, batch size fixed at 100.6M tokens, LR 2.6e-4 until 28T, cosine decay to 2.6e-5 between 28T and 40T, held to 45T. The compression is not bolted on after the fact — the model never knew a dense KV cache. (Self-hosting cost math for this stack is deliberately out of scope here; we cover that separately.)

7. What could break

The falsification signals, in the order a skeptical operator should test them:

1. CSA2 single-point selection. In the decoder, Top-K selection is computed at exactly one Full layer and four Reindex layers; every Reuse layer (15 of the 20 decoder layers) consumes those indices unverified. A systematic selection error at the single Full layer poisons the entire downstream stack, and the pool reuse in Reindex compounds it. The paper's limitations section names this explicitly: "potential selection errors in CSA2... may still cause capability degradation in untested boundary cases." The reproducible probe: needle-retrieval tasks whose targets sit just below the top-k boundary (k=512) at 512K-1M context, run across many seeds — a distribution of misses rather than graceful degradation is the failure signature.
2. Bounded Replay position-dependence. The paper states — not burying it, which deserves credit — that replayed states depend on the cache-hit position and are not mathematically identical across positions. This is directly probeable: same prompt, two different cache resumption points, compare output divergence. A vendor whose product is deterministic-seeming agent sessions is shipping a system whose outputs are only statistically stable across cache schedules. That divergence, should it appear in the wild on some input class, is the design's most unsweepable vulnerability, because the replay is structural — there is no config flag that restores exactness without re-paying L x n_win.
3. FP4 error at small magnitudes. E2M1 has a 1-bit mantissa; its relative error is largest where values are small, and small magnitudes are exactly where long-context attention logits and needle retrieval live. The norm argument in section 4 is a range argument — it certifies nothing clips, and clipping was never the risk. The evidence for harmlessness is QAT plus observed training magnitudes, and the SWA branch's own FP8 sensitivity is the in-family evidence that the team knows where the cliff is. A 512K-1M needle benchmark with FP4 versus an FP8-KV ablation is the test; the paper's limitations paragraph names "sparse retrieval over long contexts and SWA state reconstruction at cache-resumption boundaries" — precisely these two classes — as its own stress priorities.

8. Verdict and scope honesty

What the design actually is: four known compression axes (YOCO-style asymmetric prefill, three-dimension cross-layer reuse in the IndexCache/CLA lineage, NVFP4-style storage quantization, and a replay-vs-persistence trade), composed multiplicatively and trained in from scratch, wrapped in deployment machinery — a memory tier, a drafter, fused kernels, EPD disaggregation — that assumes the compression rather than tolerating it. The 890 B/token is consistent with the architecture (our closest decomposition: 864, within 3.0%) but not fully derivable from the published config; the 1/4 and 1/8 are against the vendor's own optimized previous generation; the benchmark parity claims are the vendor's self-report, alongside admitted gaps (science-oriented Terminal-Bench 4.0, multimodal versus giant closed-source models) — and "over 95% of real-world tasks" is a claim about their task distribution, which nobody outside can sample. What survives skepticism: the norm-derived FP4 format choice, the constant-cost indexer pool, and the bounded-replay trade arithmetic — all mechanism, all checkable, all ours to re-verify above. What to watch: whether the single-Full-layer selection, the position-dependent replay state, and the 1-bit-mantissa long-context behavior hold on inputs that look nothing like the evaluation suite. The paper names all three as untested boundaries; that is the correct last sentence of any honest review.

9. Sources