DeepSeek-R1-Distill-Qwen-7B SFT — AIME (16384)

Fine-tuned from deepseek-ai/DeepSeek-R1-Distill-Qwen-7B using QLoRA (4-bit NF4 quantization + LoRA adapters, merged before upload).

Training Configuration

Learning rate: 2e-5 (cosine schedule, 5% warmup)
Batch size: 1 per device, gradient accumulation 16 (effective batch size 16)
Epochs: 3
Max sequence length: 16384
Precision: bf16
Weight decay: 0.01

QLoRA

Quantization: 4-bit NF4 with double quantization
LoRA rank: 64
LoRA alpha: 128
LoRA dropout: 0.05
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

Loss

completion_only_loss: prompt tokens are masked; loss is computed only on assistant completion tokens
Dataset is converted from messages to prompt/completion format before training

Dataset

Trained on tinyllms/aime-1983-2023-trajectories. Examples exceeding max_seq_len are filtered out. A 10% holdout is used for evaluation (eval runs every 10 steps).

Infrastructure

GPU: NVIDIA H100 80GB
Framework: TRL 0.29 + Ray Train
Tracking: Weights & Biases (project: pocket-sheet-sft)
Run name: sft-deepseek-aime
Ray Job ID: raysubmit_GYFgmtmvD3Fcm9Cb
W&B: https://wandb.ai/psr-labs/pocket-sheet-sft/runs/ahw8j64z

tinyllms/deepseek-r1-distill-qwen-7b-sft-aime