You can deploy this model and start earning money today!

selimaktas/
qwen3-0.6b-speculator

Name: qwen3-0.6b-speculator
Availability: InStock

safetensors

qwen3

Views

No views yet

Qwen3-0.6B Speculative Decoder for Qwen3-14B

This is a Qwen3-0.6B model fine-tuned to serve as a speculative decoder for Qwen3-14B, enabling faster inference through speculative decoding.

Model Details

Base Model: Qwen/Qwen3-0.6B
Target Model: Qwen/Qwen3-14B
Training: Knowledge distillation with ArcticTraining
Use Case: Speculative decoding for 2-4x inference speedup

Usage

With vLLM + Arctic Inference

python

from vllm import LLM

llm = LLM(
    model="Qwen/Qwen3-14B",
    speculative_model="your-username/qwen3-14b-speculator",
    num_speculative_tokens=3,
    trust_remote_code=True
)

outputs = llm.generate("Hello, how are you?", max_tokens=100)
print(outputs[0].outputs[0].text)

selimaktas/qwen3-0.6b-speculator

Qwen3-0.6B Speculative Decoder for Qwen3-14B

Model Details

Usage

With vLLM + Arctic Inference

selimaktas/
qwen3-0.6b-speculator