⚡ DeepSeek V4 Flash on AMD Strix Halo Up to 32 tok/s decode — a tuned gfx1151 fork of antirez/ds4 with ROCmFPX tooling, SSD expert streaming, and native DeepSeek-V4-Flash-0731 support
Run a 284B-parameter model at up to 32 tok/s on a single AMD APU — no cloud, no discrete GPU, no per-token bill. This fork turns a Strix Halo mini-PC (Ryzen AI MAX+ 395 / Radeon 8060S) into a local DeepSeek V4 Flash inference machine.
That number is not a fantasy. It's the same hardware configuration that took the LocalMaxxing DeepSeek V4 Flash record:
Engine / Quant
Decode (tok/s)
DwarfStar · Q2_K
15.60
HipFire · MQ2 + MTP
18.99
This stack · ROCmFPX + DSpark
32.00
That's 2.05× faster than the previous unified-memory leader and 68.5% ahead of the runner-up — measured on the exact silicon this repo targets.
🧠 What Makes ROCmFPX So Good
The format
ROCmFPX is not one quantization format — it's a family of block formats designed for AMD ROCm/HIP silicon. Each block holds 32 weights as packed low-bit codes plus one or two tiny scales, and the GPU kernels are written for exactly that byte layout:
Variant
Block size
Bits / weight
Typical use
ROCmFP2
10 B / 32
2.50
Routed expert gate & up matrices (the biggest tensors)
ROCmFP3
—
3.50
Expert down projections
ROCmFP4
—
4.25
Dense / sensitive projections
A Strix-specific mixed-precision recipe combines them: the enormous routed-expert gate/up matrices at ROCmFP2, expert down at ROCmFP3, and dense projections at ROCmFP4+. With an importance matrix during quantization, the full DeepSeek V4 Flash target lands at ~2.88 bits per parameter in a 102.3 GB file — just under 95.3 GiB, so the whole model fits in Strix Halo's 128 GB unified pool with room to spare.
Why it's fast
The format is inseparable from the kernel that eats it:
Register-resident codebooks. Kernels expand the tiny packed codebooks in GPU registers using AMD's byte-permute instruction (v_perm_b32) instead of doing a separate gather from memory. No indirection, no extra loads.
Integer dot products. Packed blocks feed dp4a-style integer dot products directly — the fastest path on RDNA3/3.5 — instead of dequantizing to floats first.
Designed as one path. The file layout and the HIP kernel are specified together, so decode is a straight memory-traffic-bound stream of weights through fixed-purpose hardware.
At batch one, every generated token streams the active experts across all 43 layers, so decode is memory-traffic-limited — and ROCmFPX is built to maximize useful bytes per load.
The engine side
ROCmFPX only handles the weight traffic. The full 32 tok/s profile also uses:
DSpark draft + fused q=4 verification — a small 3-layer drafter proposes up to 3 tokens; the 284B target verifies 4 positions in one fused HIP graph pass (26.4% faster than autoregressive).
Weight reuse across verification columns — each packed dense weight is decoded once and applied to all 4 verify columns (+2.1–2.3%).
Indexed sparse prefill — ~250 tok/s on 8K prompts via DeepSeek's learned indexer.
🛠️ What This Fork Improves Over Upstream antirez/ds4
The upstream repo brought the initial DeepSeek V4 ROCm backend. This fork makes it actually run well on Strix Halo:
1. Fixed the SSD Expert Streaming Slab Allocator (src/ds4.c)
Mixed-precision GGUFs have layers with different per-expert sizes (e.g. 0731's Layer 26 uses IQ2_S gate/up at 82 B/256 vs IQ2_XXS at 66 B/256). Upstream pinned the streaming cache to the first layer's size, which bounced Layer 26 into pageable mapped views — an MMU fault on ROCm. Now the slab is sized to the maximum across all 43 layers, so 0/43 layers fall off the fast path.
In-place GGUF converter with bit-exact MXFP4 dequantization and correct element interleaving — converts IQ3_XXS/MXFP4 down experts to custom Q2_K in under 2 minutes.
4. ROCm 7.2.x diagnostics & TTM auto-sizing
make rocm-diag, make rocm-doctor, make rocm-smoke, make rocm-bench-quick, and DS4_ROCM_TTM_AUTORAISE=1 — detect bad configs and fix them automatically.
📦 Two Ways to Run DeepSeek-V4-Flash-0731
Route A — High-throughput ROCmFPX (~32 tok/s)
The ROCmFPX/ROCmFP2 target (Q2_0_ROCMFPX, ~98–102 GB) plus the DSpark drafter. This is the LocalMaxxing record path.
