voicebook-lora-v9-dpo
DPO continue-training from voicebook-lora-v8.
Reference policy = v8 (merged into base before training fresh adapter).
Trained on 189 (prompt, chosen, rejected) preference pairs derived from v8 audit failures.
Final metrics (1 epoch, lr=5e-6, beta=0.1):
- train_loss: 0.6793
- rewards/accuracies: 0.78 (final batch)
- rewards/margins: 0.055