Qwen 2.5 7B — Enterprise Customer Support LoRA Adapter
QLoRA fine-tuned adapter for enterprise customer support conversations.
Model Details
- Base model: Qwen/Qwen2.5-7B-Instruct
- Method: QLoRA (NF4 4-bit quantization + LoRA adapter)
- LoRA config: r=4, alpha=8, target_modules=[q_proj, v_proj]
- Dataset: 963 training + 86 validation customer support conversations
- Trainable parameters: 1,261,568 (0.017% of base)
- Adapter size: 21 MB
Performance
| Metric | Value |
|---|
| Final train loss | 2.44 |
| Final eval loss | 1.81 |
| Perplexity | 6.11 |
| Training time | 14 min on 2x T4 |
| Training cost | $0 (Kaggle free tier) |
Capabilities
- Multi-turn context retention
- Empathetic clarification (no over-asking)
- Out-of-scope refusal with polite redirect
- Escalation handling without fake actions
- 6 customer tones supported (neutral, frustrated, angry, confused, urgent, non-native English)
Training Details
- Framework: PEFT 0.13.2, TRL 0.12.1, transformers 4.46.3
- Precision: FP16 (Turing T4 compatibility)
- Attention: SDPA
- Optimizer: paged_adamw_8bit
- LR: 2e-4, cosine schedule, 5% warmup
- Batch: effective 32 (per_device=1, grad_accum=32)
- Epochs: 1
Limitations
- Baseline model trained on small dataset (963 examples, 1 epoch only)
- No RAG integration — knowledge is only what's in fine-tune data
- Cannot perform real actions; only describes/initiates them
- English only; multilingual support planned
License
Apache 2.0 — same as base Qwen 2.5
