Sarath569/gemma-2b-legal-qa
A QLoRA closed-book QA adapter on google/gemma-2-2b (2.6B) for legal / financial text,
trained on a teacher-distilled, LLM-judged, deduplicated QA dataset built from US case law +
SEC filings. Companion to the from-scratch 125M models (Sarath569/slm-125m-legal-*).
- Trainable (LoRA) params: 20,766,720 of ~2.6B
(1.28%).
- Train examples: 9,000 (3 epochs,
1,631,097 tokens processed).
- Final train loss ~1.1329. QLoRA cost ~$1.32.
Prompt
Uses Gemma's chat template: the question in the user turn; the model answers from parametric memory.
Limitations
2.6B model, QLoRA-tuned on a small synthetic set — research demo, not legal/financial advice.
