DeepSeek-V4-Flash-0731 (304B, 43 layers, 256 routed experts top-6 + shared) in the CMF container, running on cortiq.

Requires cortiq 0.5.47 or newer. No configuration: the runtime detects the GPU and VRAM itself and picks the fast path, including speculative decode. Older versions needed a dozen environment variables for the same result — they are no longer necessary.

(parts-q2tp-v2 is superseded by -v3: same trunk byte for byte, the draft stack requantized for speculative decode.)

Files ship in slices that concatenate back byte for byte:

The .dspark.tsv sidecar tells the draft which experts to keep in VRAM. Without it everything still runs; speculative acceptance is just lower.

Get cortiq from the releases or cargo build --release --features gpu. Then:

Prompt format: BOS is required, and ordinary chat closes the reasoning block in the prompt — end with </think>. For reasoning mode end with <think> instead. Without BOS the output is noise; that is out-of-distribution input, not a broken model.

Generating code: use --temperature 0.6 --rep-penalty 1.0. Measured head to head on a ~6000-token three.js scene: that setting produced a complete page that ran on the first try with zero console errors. The default repetition penalty (1.1) is harmful for code — it penalizes the repeats code is made of (identifiers, digits): at low temperature the model argues itself into the token cap, at high it writes a tutorial with placeholders instead of a file. Greedy is close but left a handful of one-character typos.

cortiq bench --core --tokens 128, q2tp, one RTX PRO 6000 Blackwell, cortiq 0.5.46, no environment variables. The VRAM budget is auto-detected; the smaller points below were measured on the same card by capping it with CMF_GPU_VRAM_MB to what a card of that size would auto-detect.

Speculation engages automatically when the budget packs the trunk far enough for the draft's capture layers; below that it declines and the whole budget goes to expert packs. Plan on system RAM at least the file size — the model is memory-mapped.

4.578 on the repository's reference text, identical to the last digit on CPU and GPU and at every VRAM budget from 16 to 96 GB. Any change that moves it is a defect. (cortiq ppl pins the strict kernels itself; generation uses the fast ones.)

Known trade-off of q2tp: it reasons and writes like q4tp but is weaker at arithmetic (2+2 can come out wrong; q4tp answers 4). Do not restrict routing with CMF_MOE_MASK — generation degrades within a few dozen tokens.

Weights derive from DeepSeek's release and remain under its licence. The CMF container and the cortiq runtime are Apache-2.0 (see the repository's LICENSE and PATENTS.md).