Nepal Civil Law Q&A (Qwen3-4B-Instruct Fine-Tuned)
NyayaLM is a specialized large language model fine-tuned for understanding and answering queries related to the Nepal Civil Code. It is based on the Qwen3-4B-Instruct architecture and has been optimized using Unsloth for efficient 4-bit LoRA (Low-Rank Adaptation) training.
The model is designed to assist in legal reasoning, providing information on civil rights, property laws, family laws, and legal procedures within the context of the Nepalese legal system.
Model Details
- Developed by: [Chhatramani Yadav]
- Base Model:
unsloth/Qwen3-4B-Instruct-2507
- Language(s): English (Legal Domain)
- License: Apache-2.0
- Fine-tuning Technique: LoRA (Rank-Stabilized - RS-LoRA)
- Quantization: 4-bit (via Unsloth/bitsandbytes)
Intended Use
NyayaLM is intended for legal researchers, practitioners, and students in Nepal. It can:
- Answer specific questions based on the Nepal Civil Code.
- Provide legal context for various civil scenarios.
- Assist in drafting and summarizing legal principles.
Note: This model is for experimental purposes only and should not be treated as professional legal advice.
Training Data
The model was trained on a high-quality mixed dataset comprising two distinct types of legal QA:
- Real-World Civil Code Legal QA (Type 2): 2,991 examples focusing on practical legal applications.
- Conversational Civil Code QA (Type 3): 2,386 examples curated for multi-turn dialogue.
Dataset Statistics:
- Total Raw Samples: 5,377
- Mixing Ratio: 70% Real-World QA / 30% Conversational QA
- Final Mixed Dataset Size: 4,272 examples
- Split: 90% Training (3,844 samples), 10% Evaluation (428 samples)
Training Hyperparameters
The following configuration was used for fine-tuning via Unsloth:
| Parameter | Value |
|---|
| LoRA Rank (r) | 16 |
| LoRA Alpha | 32 |
| LoRA Dropout | 0.08 |
| RS-LoRA | Enabled |
| Optimizer | AdamW (8-bit) |
| Learning Rate | 1.2e-4 |
| Max Sequence Length | 2048 |
| Weight Decay | 0.01 |
| Batch Size | 2 (with Gradient Accumulation) |
| Precision | 4-bit NormalFloat (NF4) |
Evaluation Results
The model was evaluated on 20 specific test cases comparing original context vs. zero-context performance.
Fine-tune Model Evaluation Metrics
| Metric | Score |
|---|
| ROUGE-1 | 0.3749 |
| ROUGE-2 | 0.1459 |
| ROUGE-L | 0.2436 |
| BERTScore (F1 Avg) | 0.8808 |
| BERTScore (With Context) | 0.8808 |
Insights: The high BERTScore indicates strong semantic alignment with legal ground truths, even when lexical overlap (ROUGE) varies.
How to Use
You can run this model using the Unsloth library for 2x faster inference:
1from unsloth import FastLanguageModel
2import torch
3
4model, tokenizer = FastLanguageModel.from_pretrained(
5 model_name = "your-username/NyayaLM-Nepal-Civil-Law",
6 max_seq_length = 2048,
7 load_in_4bit = True,
8)
9FastLanguageModel.for_inference(model)
10
11# Example Prompt
12messages = [
13 {"role": "system", "content": "You are a legal assistant specializing in Nepal Civil Law."},
14 {"role": "user", "content": "What is the time limit for registering a birth in Nepal?"}
15]
16
17inputs = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt").to("cuda")
18outputs = model.generate(input_ids=inputs, max_new_tokens=512)
19print(tokenizer.batch_decode(outputs))
qwen3-4B-civillaw-en-v1-gguf : GGUF
This model was finetuned and converted to GGUF format using
Unsloth.
Example usage:
- For text only LLMs:
./llama.cpp/llama-cli -hf chhatramani/qwen3-4B-civillaw-en-v1-gguf --jinja
- For multimodal models:
./llama.cpp/llama-mtmd-cli -hf chhatramani/qwen3-4B-civillaw-en-v1-gguf --jinja
Available Model files:
qwen3-4b-instruct-2507.Q4_K_M.gguf
Ollama
An Ollama Modelfile is included for easy deployment.
This was trained 2x faster with
Unsloth
