Llama-3.1-8B Alpaca QLoRA (P1, MI300X bootcamp)
QLoRA fine-tune (rank=16, alpha=32) of meta-llama/Meta-Llama-3.1-8B-Instruct on yahma/alpaca-cleaned — 1 epochs, final loss 0.9306, 62.1 min on MI300X.
Details
- Base model:
meta-llama/Meta-Llama-3.1-8B-Instruct
- LoRA rank: 16
- Epochs: 1
- Effective batch size: 16
- Learning rate: 0.0002
- Final train loss: 0.9306
Trained as part of the MI300X 50-hour fine-tuning bootcamp.
