Metric	Value
Base model	Qwen/Qwen3.5-397B-A17B (397B MoE, 17B active)
Final train loss	0.0848
LoRA rank / alpha	16 / 32
Target modules	all (13 types: q/k/v/o_proj, gate/up/down_proj, shared_expert_gate, in_proj)
Trainable params	64.2M / 396.9B (0.016%)
cutoff_len	32,768
Training examples	296 (3-view extraction from 102 raw trajectories)
Epochs	3
Runtime	1h 32 min
Hardware	8x AMD Instinct MI355X, DeepSpeed ZeRO-3
wandb	amdpilot/amdpilot-sft/runs/8hh4pmqx

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen3.5-397B-A17B", device_map="auto", torch_dtype="bfloat16"
)
model = PeftModel.from_pretrained(base_model, "JinnP/Qwen3.5-397B-A17B-LoRA-SFT-v2")