We do not publish per-device numbers for this experimental variant. Uplift varies by device, target model, and workload. For measured results on the current default draft line, see TokForge-AccelerationPack-Draft.

Training acceptance (alpha) at the final logged epoch:

0.7350

Included files

llm.mnn
llm.mnn.weight
llm_config.json
config.json
config_cpu.json
tokenizer files
ONNX export artifact for reference

Usage

This bundle is meant for TokForge / MNN, not standard HF Inference.

Typical TokForge recipe:

json

{
  "backend_type": "opencl",
  "thread_num": 4,
  "precision": "low",
  "memory": "low",
  "sampler_type": "greedy",
  "speculative_type": "draftmodel",
  "draft_predict_length": 3,
  "draft_config_path": "/path/to/config_cpu.json"
}

Known-good draft-side config:

json

{
  "backend_type": "cpu",
  "thread_num": 2,
  "precision": "low",
  "memory": "low",
  "sampler_type": "greedy"
}

Notes

In our testing, this trained objective improved acceptance over the KL baseline.
On short device benchmarks, the runtime win was in the same general band as the KL model.
This makes it a good experimental alternative, but not a guaranteed universal replacement.

Limitations and Intended Use

Intended for speculative decoding with larger Qwen3 targets inside TokForge.
Training acceptance improved over the KL baseline, but device throughput gains stayed in a similar band on short runs.
Best current evidence is strongest on Qwen3-8B.
This is a specialized runtime artifact, not a general-purpose pretrained release.

Collection

TokForge Mobile Draft Models

TokForge

Website: tokforge.ai
Discord: Join the Discord

If you benchmark this on your own device, feel free to share results in Discord.

darkmaniac7/Qwen3-0.6B-lk-alpha-20k-MNN

Qwen3-0.6B-lk-alpha-20k-MNN

Why this repo exists

Best-known use

Benchmark notes