Stage-2a endpoint — chunked full-vocab forward-KL KD @4k, 600M tokens (LR 2e-4/2e-5 probe-picked). Val KL 0.5588 -> 0.1449, top-1 agree 0.851, wikitext-2 PPL 12.83 (teacher 12.09). Input to stage-2b long-context KD.

Part of a study converting full-attention Qwen3-4B into a GDN (gated DeltaNet) hybrid (27 of 36 layers converted, uniform 1:4 retention) and recovering capability via staged distillation, culminating in on-policy distillation (OPD): student rollouts scored per-token by the frozen teacher under reverse KL. Checkpoints are stock Qwen3NextForCausalLM — load with AutoModelForCausalLM (transformers >= 4.57). vLLM serving needs a small registry shim (all-dense MoE + tied-embeddings fixes) from the project repo.

Pipeline: surgery/init -> stage-1 hidden-state alignment (100M tok) -> stage-2a forward-KL KD @4k (600M) -> stage-2b long-context KD @32k (294M) -> stage-3 OPD (200M gen tokens, horizon 512->16k, token-keyed LR). Teacher/reference: Qwen/Qwen3-4B (thinking mode).