Views
No views yet
Qwen/Qwen3.5-397B-A17B, trained with LLaMA-Factory on the amdpilot_v5_1 SFT dataset.Qwen/Qwen3.5-397B-A17B to use it.32 / 6465536truefalsebf168x AMD MI355X100.06304522951443990.13314819335937547396.7738s (13.17h)| Step | Epoch | Eval loss |
|---|---|---|
| 10 | 1.7273 | 0.1846 |
| 20 | 3.3636 | 0.1579 |
| 30 | 5.0 | 0.1417 |
| 40 | 6.7273 | 0.1357 |
| 50 | 8.3636 | 0.1336 |
| 60 | 10.0 | 0.1331 |
amdpilot_v5_1 training split, 65536 tokens cover about 82/89 samples (92.13%). This is substantially better coverage than the earlier 32768 setting.1from transformers import AutoModelForCausalLM, AutoTokenizer
2from peft import PeftModel
3
4base_model_id = "Qwen/Qwen3.5-397B-A17B"
5adapter_id = "JinnP/Qwen3.5-lora-sft-v5-1-64k"
6
7tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
8base_model = AutoModelForCausalLM.from_pretrained(base_model_id, trust_remote_code=True)
9model = PeftModel.from_pretrained(base_model, adapter_id)adapter_model.safetensors: LoRA adapter weightsadapter_config.json: PEFT adapter configtokenizer.json / tokenizer_config.json / chat_template.jinja: tokenizer assetsall_results.json / eval_results.json / train_results.json: training metrics