DeepSeek-V4.1-Flash-Ternary
«- back to models- - for complex coding and research agents
- - ~92% intelligence retained vs FP4
- - 142 GB pack, 28% of 510 GB base
- - ~12.1x speedup (11.8 → 143 tok/s @ c1)
Performance (Measured on 4× RTX 6000 Pro Max-Q)
Agentic coding throughput
Replay of 10 SWE-bench Pro / DeepSWE mini-swe-agent conversations × 6 turns each. Full chat history + tool observations fed back each turn; prefix cache on.
| Concurrency | out tok/s | accept length | TTFT ms | ITL ms |
|---|---|---|---|---|
| 1 | 194 | 3.78 | 425 | 3.5 |
| 4 | 309 | 4.08 | 474 | 9.8 |
Cold prefill and long-context decode
Same stack with --disable-radix-cache and --flush-cache between lengths. sglang.benchmark.serving random, 8 prompts, concurrency 1.
| prompt | decode tok/s | prefill tok/s | TTFT mean / p50 / p90 / p99 (ms) | ITL mean / p50 / p90 / p99 (ms) | accept |
|---|---|---|---|---|---|
| 1024 | 75 | ~0.9k | 1080 / 1085 / 1982 / 2828 | 5.8 / 4.1 / 16.0 / 16.6 | 2.83 |
| 4096 | 99 | ~6.5k | 629 / 600 / 984 / 1000 | 5.7 / 4.1 / 8.5 / 16.8 | 2.88 |
| 16384 | 27 | ~3.7k | 4376 / 2203 / 8701 / 20339 | 5.8 / 4.1 / 8.6 / 16.6 | 2.88 |
| 32768 | 32 | ~9.1k | 3617 / 3301 / 6158 / 7080 | 5.5 / 4.1 / 8.3 / 16.6 | 3.02 |
Decode slows as context grows; ITL stays flat.
Optimisation
- - Hadamard-rotated ternary GPTQ expert overlay (group 128, 40 layers, w123 layout)
- - Attention, router, shared experts, Engram, and DSpark draft stay in the base checkpoint
- - DSpark speculative decoding (block size 5) from base draft MoE
- - Goethe ternary_runtime kernels for ternary MoE decode (TP=EP=4)
- - Full CUDA graphs on decode; unused experts fall back to FP4 from base