Qwen3-4B distilled from Qwen3-30B-A3B — On-policy 5k (continual, prompts 30k–35k)
Continual on-policy knowledge distillation of a Qwen3-4B student toward the Qwen3-30B-A3B (MoE) teacher, on 5,000 math prompts from openthoughts3_math_50k (positions 30k–35k, disjoint from prior training slices).
The student for this run is
RockToken/qwen3-30b-a3b-to-4b-onpolicy-10k, so the effective training exposure of this checkpoint is:
- Off-policy KD on 20k teacher responses (positions 0–20k of
openthoughts3_math_50k)
- On-policy KD on 10k prompts (positions 20k–30k) — produced the 10k model
- On-policy KD (this run) on 5k prompts (positions 30k–35k) — this checkpoint
Stages 2 and 3 use the same on-policy pipeline; only the seed weights, prompt slice, and step count differ.
Models
| Role | Model |
|---|
| Student | Qwen3-4B-Instruct-2507 (after stages 1–2 above) |
| Teacher | Qwen/Qwen3-30B-A3B-Instruct-2507 (MoE, ~3B active) |
enable_thinking=False throughout (both stages).
Training data
- Source:
openthoughts3_math_50k (math split of OpenThoughts-3)
- Slice used here: prompts 30k–35k → RockToken/openthoughts_prompt_math_5k_src30k-35k
- 5,000 single-user-turn prompts, mean length 261 chars, p99 845 chars
Training setup
Framework:
KDFlow — FSDP2 + SGLang rollout, Ray-orchestrated GPU co-location with sleep/wakeup.
Hardware: 1× node, 4× H100 (80 GB), ~34 h wall-clock (~136 GPU-hours).
Key hyperparameters (on-policy stage; same as stage 2)
| Group | Value |
|---|
| Backend | fsdp2, bf16, gradient ckpt on |
| Epochs | 1 |
| Train batch | 4 (micro 1) |
| Learning rate | 2e-6, cosine, warmup 5% |
| KD ratio | 1.0 |
| KD loss | reverse KL (rkl) |
| KD algorithm | vanilla_kd |
| Temperature (KD) | 1.0 |
| Rollout engine | SGLang, TP=2, 1 engine |
| Rollout batch | 2 prompts × 4 samples/prompt |
generate_max_len | 8000 |
prompt_max_len | 800 (total max_len 8800) |
| Sampling | temperature 1.0, top-p 1.0 |
| Teacher | TP=4, sleep/wakeup enabled |
Off-policy stage 1 hyperparameters (for context): batch 128, LR 2e-5, kd_ratio 0.5, kd_loss KL, max_len 16384, packing_samples on, ring attention.
Intended use
Research on distillation dynamics. Domain skew is math (OpenThoughts-3 math split).
Limitations
- Trained end-to-end on math prompts only; not tuned for chat, safety, or non-math domains.
enable_thinking=False — this student does not emit <think> traces.- Continual training may drift capabilities relative to the base checkpoint; see the wandb curves for loss dynamics.
Weights & Biases
legendary-music-16 (stage 2, 10k) and lemon-sound-17 / 2gvd60dw (this run, stage 3) under project KDFlow, entity zhao666.
