Apathy Qwen3.8-27B DFlash drafter v3
A six-layer DFlash-family draft model for speculative decoding with
Qwen3.8-27B. This repository contains BF16 source weights; Atlas quantizes the
dense draft projections to NVFP4 when loading the speed profile.
This is a draft model, not a standalone language model. A target model must
verify every proposed token.
Verified Weschera result
On one DGX Spark / GB10, the answering optimized-qwen target and this drafter
measured the following on Weschera's fixed MinHeap prompt, single stream,
temperature 0, reasoning disabled, 400 output tokens:
| Profile | Draft vocabulary | Repetitions | Median server decode tok/s |
|---|
| BF16 target KV | 96,000 | 5 | 61.7526 |
| NVFP4 target KV | 96,000 | 5 | 74.0882 |
| NVFP4 target KV | 248,320 (full) | 5 | 72.1689 |
Full-vocabulary rates were 70.6411, 72.1877, 72.2574, 72.0858, and
72.1689 tok/s. All responses had stable output SHA-256
f51d8358ea2a5c63353ca00a29208ae2cccd3039b070043cad514cc4af9761c4.
The complete result is included at
benchmarks/weschera-minheap-400tok-fullvocab-nvfp4kv-5x.json.
A post-publication end-to-end repeatability gate then ran the same request ten
more times against the unchanged server. All ten responses retained the exact
stable output hash above. Decode throughput had median 72.321 tok/s, mean
72.151 tok/s, standard deviation 0.712 tok/s, and 0.987% coefficient of
variation (range 70.799-72.929 tok/s). Thus the token output is deterministic
for this probe, while throughput is repeatable within normal runtime variance,
not numerically deterministic. The result is included at
benchmarks/weschera-minheap-400tok-fullvocab-nvfp4kv-e2e-10x.json.
Important quality boundary
NVFP4 target KV changed the target output relative to BF16 and FP8 KV. The
72.1689 figure is therefore a deterministic throughput result for this fixed
probe, not evidence that NVFP4 KV is output-equivalent or broadly
quality-equivalent to BF16. Every response reached the 400-token cap, so the
probe also does not certify task completion. Use BF16 KV for reference behavior
and separately qualify NVFP4 KV on your workloads.
Shape and hashes
| Field | Value |
|---|
| Architecture | DFlashDraftModel |
| Layers | 6 |
| Hidden size | 5,120 |
| Vocabulary | 248,320 |
| Block size | 16 |
| Maximum draft width | gamma 15 |
| Target capture layers | 1, 10, 18, 27, 35, 44, 52, 61 |
| Tensors | 69 |
| Weights | BF16 safetensors, 4,257,372,464 bytes |
model.safetensors SHA-256 | c15685d680bd58939689dcb4c344bb325efb75536df16d77c38517ce3df2dd6c |
config.json SHA-256 | c24cb2ef1c9788f90ed05870e9df70524f75ea84ceb4809909e1afcaaee22882 |
dflash.py SHA-256 | 1073c6dd9c71b404920739f183cd8ff431b82748f6210abaca6cbc7bfc11112f |
block_size=16 means trained_drafts = block_size - 1; use gamma 15. Atlas
refuses widths above the checkpoint contract.
Use with Atlas on GB10
The included serve-atlas-gb10-72.sh records the measured single-stream
profile without requiring changes to the Atlas Git repository:
1MODEL_DIR=/path/to/optimized-qwen \
2DRAFT=/path/to/this-repository \
3SPARK_BIN=/path/to/apathy-atlas/target/release/spark \
4./serve-atlas-gb10-72.sh
Then run the fixed five-repeat harness from the Atlas performance branch:
1python3 bench/qwen38-gb10/weschera_minheap_repro.py \
2 --endpoint http://127.0.0.1:8896/v1/chat/completions \
3 --output /tmp/weschera.json \
4 --repetitions 5 --max-tokens 400 --store-output
The measured binary SHA-256 was
ebe7a7c8408bcd2d2aa492273aba8bc608c625e3e4c6c22831d014824c44f7d5.
Performance is workload-, engine-, binary-, and hardware-specific.
Provenance and limitations
The exact three-file runtime export was retained without its optimizer state,
training-state sidecar, corpus manifest, or immutable training manifest. This
publication therefore does not claim an exact reproducible training
lineage. Neighboring campaign notes are not substituted for missing artifact
provenance.
What is verified is the published runtime artifact itself: file hashes, tensor
count/shape contract, successful Atlas loading, deterministic target-verified
decoding, acceptance telemetry, and the benchmark JSON above. The base target
family is Apache-2.0 Qwen3.8-27B.
Do not infer broad quality, safety, long-context, or training-corpus claims from
the MinHeap throughput result.