
Base model	Qwen/Qwen3-0.6B (Haidass, 596M params)
Teacher model	Qwen3.6-35B-A3B (MoE, 35B total / 3B active)
Architecture	Qwen3ForCausalLM
Training data	GSM8K (7,473 train samples)
Training steps	200
Hardware	8× Ascend 910B3 (64GB HBM)
Framework	verl + vLLM 0.18.0 + vllm-ascend 0.18.0

Architecture

Parameter	Value
hidden_size	1024
num_hidden_layers	28
num_attention_heads	16
num_key_value_heads	8 (GQA)
intermediate_size	3072
vocab_size	151936
max_position_embeddings	40960
tie_word_embeddings	true

Training Configuration

Distillation loss: k1 KL divergence + policy gradient
Student: DP=4, TP=1, NPU graph capture enabled, sleep mode disabled
Teacher: TP=4, graph capture enabled
Batch size: 48 (micro_batch=16 per GPU)
Learning rate: 1e-6
Max prompt length: 512 tokens
Max response length: 1024 tokens
Performance: 29.8s/step, 109 tok/s throughput

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "caoyao/haidass_qwen3.6-35b-a3b_opd_step_200"

tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

messages = [
    {"role": "user", "content": "Natalia sold clips to 48 of her friends in April, and then she sold half as many clips in May. How many clips did Natalia sell altogether in April and May? Let's think step by step and output the final answer after \"####\"."}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Repository

Training code and scripts: github.com/yuancaoyaoHW/verl-npu

Acknowledgements

Qwen team for Qwen3-0.6B and Qwen3.6-35B-A3B
verl project for the RL training framework with OPD support
Ascend for NPU hardware and vllm-ascend