This is a research-preview auxiliary draft checkpoint for speculative decoding with CohereLabs/North-Mini-Code-1.0-w4a16. It is not a standalone language model and it cannot generate text without the North verifier/target model. It will likely also work with CohereLabs/North-Mini-Code-1.0-fp8 with slightly lower acceptance rates.

Training

This DFlash draft model was trained on 10,000 Magicoder prompts with on-policy supervision from North Mini Code.

Important Notes

not a base model
not a fine-tuning target for standalone inference
not a correctness guarantee
not byte-parity with target-only greedy decoding

Architecture

speculator type: DFlashDraftModel
draft layers: 5 Qwen3 sliding-window layers
hidden size: 2048
attention heads: 32
key/value heads: 4
head dim: 128
MLP intermediate size: 768
sliding window: 2048
draft vocabulary: 32,000
target vocabulary: 262,144
block size: 8
sample_from_anchor: false
auxiliary target hidden-state taps: layers [2, 24, 46]
tensors: 62
serialized tensor elements: 733,274,624
BF16 parameters: 732,980,480
selected weight hash: 0ac5964c48003f2eab08c4691357dedc11227d88302ea7c76fc47aaddab6d5b9

Largest tensors include the target embedding matrix, lm_head.weight, and the draft fc.weight projection. The tensor inventory is recorded in tensor_info.json.

Training data and supervision

prompt dataset: ise-uiuc/Magicoder-Evol-Instruct-110K
license: Apache-2.0
training rows: 10,000
selected rows: 0-499, 600-5099, and 6100-11099
locked holdout: rows 500-599, never trained
supervision: on-policy responses and target hidden states generated once from CohereLabs/North-Mini-Code-1.0-w4a16
response cap: 2,048 tokens
target layers used for supervision: [2, 24, 46]
later acceptance-mined continuation data: not included in this release checkpoint

Training metadata is recorded in training_metadata.json.

Matched GB10 benchmarks

These numbers come from the same matched serving contract on a GB10 (DGX Spark) machine running Marlin (which is suboptimal but predictable). Compare the rows within the same host and contract only. These absolute token/s values are not representative of maximum performance, simply a demonstration that the draft model works. :)

Contract	Target-only	DFlash K3	DFlash / target	DSpark K4	DSpark / target
C1	39.9129 tok/s	75.5106 tok/s	1.8919x	79.7341 tok/s	1.9977x
C2	78.9486 tok/s	121.5232 tok/s	1.5393x	127.6174 tok/s	1.6165x

Model	C1 EAL	C1 draft acceptance	C2 EAL	C2 draft acceptance
DFlash K3	2.2947	43.16%	2.2856	42.85%
DSpark K4	2.4800	37.00%	2.4779	36.95%

Expected acceptance length (EAL) is the mean emitted tokens per speculative verification step, including the target bonus token.

Additional result: NVIDIA RTX 3090

DFlash was also tested on an NVIDIA RTX 3090 under the same family of matched conditions. That result is specific to DFlash; do not infer DSpark RTX 3090 portability from it.

target-only: 110.2714 tok/s
DFlash K3: 153.7188 tok/s
official Eagle K3: 77.9583 tok/s

Also being realistic, you wouldn't try to serve this on a single 3090 anyway the model weights leave no room for context.

Default proposal depth

Default proposal depth is K3. That is the selected deployment setting in config.json and the configuration used for the release evidence. More speculative tokens hit diminishing returns past position 3 for this DFlash drafter. You're likely to see the best speed here.

Runtime boundary

This checkpoint is served as the top-level model because config.json already embeds the verifier reference. Use it only with the companion North runtime, published at https://github.com/sdougbrown/north-mini-code-draft-runtime; this card does not describe a separate target-model --speculative_config deployment. The companion is a thin overlay on official vLLM 0.27.1 carrying PRs #49819 and #50937 (Cohere2MoE auxiliary hidden states, and an expert-bias loading fix), published as ghcr.io/sdougbrown/north-mini-code-runtime:v0.27.1-49819-50937.

bash

export DRAFT_MODEL="${DRAFT_MODEL:?set this to the downloaded DFlash directory or Hub model ID}"
export VLLM_USE_V2_MODEL_RUNNER=1
vllm serve "${DRAFT_MODEL}" \
  --tensor-parallel-size 1 \
  --max-model-len 320000 \
  --tool-call-parser cohere_command4 \
  --tokenizer-mode cohere \
  --cohere-format cmd4 \
  --reasoning-config '{"reasoning_start_str":"<|START_THINKING|>","reasoning_end_str":"<|END_THINKING|>"}' \
  --enable-auto-tool-choice

Limitations

32K draft vocabulary, not the 262,144-token target vocabulary
target-dependent: cannot run standalone
quantized greedy outputs are not guaranteed to match target-only greedy byte-for-byte
throughput and acceptance do not establish correctness or response quality
absolute throughput is host-specific

References

North base model: https://huggingface.co/CohereLabs/North-Mini-Code-1.0-w4a16
Magicoder dataset: https://huggingface.co/datasets/ise-uiuc/Magicoder-Evol-Instruct-110K
Companion North runtime: https://github.com/sdougbrown/north-mini-code-draft-runtime
Companion image: ghcr.io/sdougbrown/north-mini-code-runtime:v0.27.1-49819-50937
Official Eagle card: https://huggingface.co/CohereLabs/North-Mini-Code-1.0-eagle
Speculators: pinned commit f7ec34182826bc89934ce710283421778022b74d, version 0.6.0

Files in this repo

README.md
config.json
config.py
model.safetensors
tensor_info.json
training_metadata.json
SHA256SUMS
LICENSE
.gitattributes

sdougbrown/North-Mini-Code-1.0-dflash

North Mini Code 1.0 DFlash