DeepSeek LoRA Adapter (1 Epoch)

Spesifikasi Singkat

• Base Model: unsloth/DeepSeek-R1-Distill-Llama-8B
• LoRA:
  • Rank (r) = 16
  • Alpha = 16
  • Dropout = 0,05 (default)
• Batch Size per Device: 2
• Gradient Accumulation Steps: 16
• Total Batch Size: 2 × 16 × 1 = 32
• Epochs: 1
• Total Steps: 297
• Framework: Unsloth (2× faster finetuning) + PEFT

Ringkasan Log Finetuning

Num GPUs used = 1 │ Num examples = 9.487 │ Num Epochs = 1 │ Total steps = 297
Batch size/device = 2 │ Grad. accumulation = 16 │ Total batch = 32
Trainable params: 9.437.184/8.000.000.000 (0,12% trained)

Grafik Loss