Saul-7B Indian Law LoRA
A LoRA fine-tune of Equall/Saul-7B-Instruct-v1 for Indian legal question answering.
Model
Base model: Equall/Saul-7B-Instruct-v1
Fine-tuned model: Demondiablo/saul-7b-indian-law-lora
This repository contains the LoRA adapter and tokenizer files required to load the fine-tuned model on top of the base Saul-7B model.
Training Data
The fine-tuning dataset was constructed from three Indian legal datasets:
kaushik-harsh-99/Indian-legal-data-v2GSMS-B/Indian-Legal-QA-BNS-BNSS-BSAnisaar/Articles_Constitution_3300_Instruction_Set
The combined dataset contained 21,665 examples after preprocessing.
Dataset 1 was limited to 12,000 examples.
Evaluation
The models were evaluated on a fixed 434-question held-out test set.
The evaluation set contained:
- 254 examples from
Indian-legal-data-v2
- 120 examples from
BNS-BNSS-BSA
- 60 examples from
Constitution-Articles
Both the base model and fine-tuned model were evaluated on the same questions and reference answers.
Gemini LLM-as-a-Judge Results
The final evaluation used Gemini 3.5 Flash Lite with a structured 1–5 scoring rubric.
| Metric | Base | Fine-Tuned | Improvement |
|---|
| Correctness | 1.873 | 2.164 | +15.50% |
| Relevance | 3.136 | 3.417 | +8.96% |
| Completeness | 1.781 | 2.023 | +13.58% |
| Hallucination | 1.929 | 2.276 | +18.04% |
| Overall | 1.836 | 2.085 | +13.55% |
For the hallucination metric, 5 indicates no hallucination and 1 indicates severe hallucination.
Question-Level Comparison
- Fine-tuned wins: 130 / 434 (30.0%)
- Base wins: 94 / 434 (21.7%)
- Ties: 210 / 434 (48.4%)
Evaluation Artifacts
The complete evaluation record is available under:
including:
heldout_434_legal_test_set.csvbase_predictions_checkpoint.jsonfine_tuned_predictions_checkpoint.jsonsaul_legal_eval_results.csvsaul_legal_eval_results.jsonsaul_legal_llm_judge_results.csvgemini_evaluation_summary.json
These files preserve the held-out test set, model predictions, question-level comparisons, and final Gemini evaluation results.
Intended Use
This model is intended for research, experimentation, and educational use in Indian legal NLP.
It should not be treated as a substitute for qualified legal advice, professional legal research, or authoritative legal sources. Model outputs may contain factual, legal, or interpretive errors.
Files
The main model repository contains the LoRA adapter and tokenizer artifacts:
1adapter_model.safetensors
2adapter_config.json
3tokenizer.json
4tokenizer.model
5tokenizer_config.json
6chat_template.jinja
The base model weights are not included in this repository and must be obtained from:
Equall/Saul-7B-Instruct-v1
Tools
Built using:
- Unsloth
- Hugging Face Transformers
- PEFT / LoRA
- Hugging Face Datasets
- Gemini LLM-as-a-judge evaluation
Status
Fine-tuning and evaluation completed.
Final evaluation coverage: 434 / 434 held-out questions.
