DFlash is a speculative decoding method that uses a lightweight
block diffusion model
to draft multiple tokens in parallel, achieving up to
4.4× speedup over autoregressive
decoding. This is the drafter model, which must be paired with
lukey03/Qwen3.5-9B-abliterated-MLX-4bit.
1import json
2from pathlib import Path
3import mlx.core as mx
4from huggingface_hub import snapshot_download
5from dflash.model_mlx import load, stream_generate, DFlashConfig, DFlashDraftModel
6
7# ── target model ──────────────────────────────────────────────────────────────
8model, tokenizer = load("lukey03/Qwen3.5-9B-abliterated-MLX-4bit")
9
10# ── draft model ───────────────────────────────────────────────────────────────
11def load_draft(repo_id: str) -> DFlashDraftModel:
12 path = Path(snapshot_download(repo_id, allow_patterns=["*.safetensors", "*.json"]))
13 cfg = json.loads((path / "config.json").read_text())
14 config = DFlashConfig(
15 hidden_size=cfg["hidden_size"],
16 num_hidden_layers=cfg["num_hidden_layers"],
17 num_attention_heads=cfg["num_attention_heads"],
18 num_key_value_heads=cfg["num_key_value_heads"],
19 head_dim=cfg["head_dim"],
20 intermediate_size=cfg["intermediate_size"],
21 vocab_size=cfg["vocab_size"],
22 rms_norm_eps=cfg["rms_norm_eps"],
23 rope_theta=cfg["rope_theta"],
24 max_position_embeddings=cfg["max_position_embeddings"],
25 block_size=cfg["block_size"],
26 target_layer_ids=tuple(cfg["dflash_config"]["target_layer_ids"]),
27 num_target_layers=cfg["num_target_layers"],
28 mask_token_id=cfg["dflash_config"]["mask_token_id"],
29 )
30 weights = {k: v for f in path.glob("*.safetensors") for k, v in mx.load(str(f)).items()}
31 m = DFlashDraftModel(config)
32 m.load_weights(list(weights.items()))
33 return m
34
35draft = load_draft("guglxni/Qwen3.5-9B-abliterated-DFlash")
36
37# ── generate ──────────────────────────────────────────────────────────────────
38messages = [{"role": "user", "content": "Write a quicksort in Python."}]
39prompt = tokenizer.apply_chat_template(
40 messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
41)
42
43tps = 0.0
44for r in stream_generate(model, draft, tokenizer, prompt,
45 block_size=16, max_tokens=2048, temperature=0.6):
46 print(r.text, end="", flush=True)
47 tps = r.generation_tps
48
49print(f"\n\nThroughput: {tps:.1f} tok/s")
1vllm serve lukey03/Qwen3.5-9B-abliterated \
2 --speculative-config '{"method": "dflash", "model": "guglxni/Qwen3.5-9B-abliterated-DFlash", "num_speculative_tokens": 15}' \
3 --attention-backend flash_attn \
4 --max-num-batched-tokens 32768
1python -m sglang.launch_server \
2 --model-path lukey03/Qwen3.5-9B-abliterated \
3 --speculative-algorithm DFLASH \
4 --speculative-draft-model-path guglxni/Qwen3.5-9B-abliterated-DFlash \
5 --speculative-num-draft-tokens 16 \
6 --tp-size 1 \
7 --attention-backend fa3 \
8 --mem-fraction-static 0.75 \
9 --trust-remote-code
The draft is initialised from z-lab's pre-trained weights and fine-tuned for 1 000 steps on
hidden-state activations extracted from the abliterated target model. Only the 5 draft
decoder layers, projection, and norm are updated — embeddings and the LM head remain shared
with the target model at inference time.
This model is a community fine-tune of
z-lab/Qwen3.5-9B-DFlash.
The architecture, config, and
dflash.py model code are identical; only the weights differ.
If you are running the
original (non-abliterated)
Qwen/Qwen3.5-9B, use z-lab's
official draft instead.
All credit for the DFlash method, architecture, and training methodology goes to
Jian Chen, Yesheng Liang, and Zhijian Liu at z-lab.
This fine-tune adapts their work for the abliterated model variant on Apple Silicon.
1@article{chen2026dflash,
2 title = {{DFlash: Block Diffusion for Flash Speculative Decoding}},
3 author = {Chen, Jian and Liang, Yesheng and Liu, Zhijian},
4 journal = {arXiv preprint arXiv:2602.06036},
5 year = {2026}
6}
