qwen3.5-9b-lora-traffic-rag-sft-v1 (ablation / negative result)
QLoRA adapter trained on (question + evidence_card) -> answer format as an ablation.
This adapter underperforms the canonical v2 and is kept only for reproducibility of the negative result.
Motivation
Investigate whether fine-tuning on the evidence-card format (instead of raw context)
would help reduce the train/inference distribution gap.
Training
- Base:
Qwen/Qwen3.5-9B, 4-bit NF4 via unsloth.
- LoRA rank 32, alpha 64.
- 1 epoch, lr 3e-5, batch 2, grad_accum 8.
- v1: 90% evidence card / 10% no-context.
- v2: 50% evidence card / 40% raw context / 10% no-context.
Result
| D (canonical LoRA-v2) | rag-sft-v1 | rag-sft-v2 |
|---|
| ROUGE-L (30 smoke) | 0.414 | 0.379 | 0.405 |
| False refusal rate | 0.033 | 0.200 | 0.233 |
Card-specialised SFT teaches the model to refuse when card fields are empty,
raising false-refusal ~6-7x. The original LoRA-v2 generalises better.
Related repos
