You can deploy this model and start earning money today!

abdullah693/
pakistan-legal-qa-llama70b

Name: pakistan-legal-qa-llama70b
Availability: InStock

Views

No views yet

Pakistan Legal QA — Llama 3.3 70B LoRA

A LoRA fine-tune of meta-llama/Llama-3.3-70B-Instruct specialising in Pakistani law, trained on a curated dataset of 5,116 question–answer pairs grounded in primary statutory text.

What it does

Given a legal question, the model produces:

A short reasoning trace explaining the applicable provision
A direct answer in plain English
An exact statutory citation (e.g. PPC s.302, CONST art.184(3), QSO art.17)

Coverage

Domain	Statute	QA pairs
Criminal	Pakistan Penal Code (PPC)	2,215
Constitutional	Constitution of Pakistan 1973	1,957
Evidence	Qanun-e-Shahadat Order 1984 (QSO)	742
Civil Procedure	Code of Civil Procedure (CPC)	143
Family	Muslim Family Laws Ordinance 1961 (MFLO)	59
Total		5,116

All answers are grounded in verbatim provision text extracted from official Pakistani statutes. A "hard slice" of 110 test questions covers Pakistani-distinctive doctrine (qisas/diyat, Qanun-e-Shahadat evidence rules, Muslim family law) where general-purpose models most commonly confuse Pakistani law with Indian or British law.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch

bnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_compute_dtype=torch.bfloat16)
base = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.3-70B-Instruct",
    quantization_config=bnb_config,
    device_map="auto"
)
model = PeftModel.from_pretrained(base, "abdullah693/pakistan-legal-qa-llama70b")
tok = AutoTokenizer.from_pretrained("meta-llama/Llama-3.3-70B-Instruct")

question = "Under PPC s.302, what is the punishment for qatl-i-amd?"
inputs = tok.apply_chat_template(
    [{"role": "user", "content": question}],
    return_tensors="pt", return_dict=True, add_generation_prompt=True
).to(model.device)

out = model.generate(**inputs, max_new_tokens=256, do_sample=False)
print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

Output format

Reasoning: <short chain-of-thought grounded in the statutory text>
Answer: <direct answer>
Citation: <canonical citation e.g. PPC s.302>

Training details

Parameter	Value
Base model	meta-llama/Llama-3.3-70B-Instruct
Method	LoRA (PEFT)
LoRA rank	64
LoRA alpha	128
LoRA dropout	0.0
Target modules	all-linear — `q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj`
Epochs	2 (134 steps)
Final train loss	0.675
Final eval loss	0.625 (monotonically decreasing across all checkpoints)
Trained via	Adaption AutoScientist / Together AI

Because the adapter targets all linear layers (including the MLP/FFN blocks where factual associations are stored), it can encode Pakistani legal knowledge rather than just output formatting. Eval loss was still declining at the end of epoch 2.

Dataset

The training data was built from licence-clean Pakistani legal corpora:

Constitution of Pakistan 1973 — full text, 305 provisions (LEGAL-UQA seed, MIT licence)
Pakistan Penal Code — 498 provisions (CC-BY)
Qanun-e-Shahadat Order 1984 — 165 provisions (CC-BY)
Muslim Family Laws Ordinance 1961 — 59 provisions (CC-BY)
Code of Civil Procedure — 143 provisions (CC-BY)

Each QA pair includes a verbatim provision excerpt as grounding context plus a chain-of-thought reasoning trace generated and verified by Claude.

Dataset: abdullah693/adaption-pakistan-law-qa-pairs

Limitations

Not legal advice. This model is for research and informational purposes only.
Coverage is limited to the five statutes above; it does not cover tax law, intellectual property, regulatory law, or case law.
The model may still hallucinate on questions outside its training distribution.
Urdu-language queries are not yet well-supported (English-only training data).

Licence

LoRA weights: CC-BY-4.0. Subject to Meta's Llama 3.3 Community Licence.