Model Card — LEGAL‑UQA Llama‑2 QLoRA

Short description

Intended use

• Research and prototyping for bilingual legal question answering.
• Assistive tools for legal document analysis and summarization (with human oversight).
• Educational and exploratory tasks in legal NLP for Urdu/English.

Model details

• Architecture: Llama‑2 family (causal decoder). 7B parameters base model; adapters trained with LoRA.
• Quantization: 4‑bit NF4 (QLoRA) applied to base weights to reduce GPU memory use.
• Adapter method: LoRA via PEFT — only adapter weights trained; base model frozen.
• Typical inputs: Single‑turn or short multi‑turn prompts formatted with Llama chat tokens: <s>[INST] ... [/INST] ... </s>.

Training data

• Source: LEGAL‑UQA (public), reformatted from original legal texts and Q&A pairs into Llama chat format.
• Examples: ~495 original rows → ~990 formatted examples (English + Urdu).
• Preprocessing: cleaning, optional context concatenation, conversion to <s>[INST] question [/INST] answer </s> strings, tokenization with the base tokenizer, and masking prompt tokens with -100 for loss computation.

Training procedure & hyperparameters (summary)

• Method: QLoRA (NF4 quantization) + LoRA adapters.
• LoRA hyperparams (typical): r=8, alpha=32, dropout=0.05 (replace with your actual values if different).
• Optimizer: 8‑bit AdamW (bitsandbytes) or equivalent.
• Learning rate: e.g., 2e‑4 (tune per run).
• Batching: small per‑device batch sizes + gradient accumulation to emulate larger batches.
• Mixed precision: fp16/bf16 where supported.
• Checkpointing: adapter weights saved separately to reduce storage.

Replace placeholder hyperparameters above with the exact values used in your run before publishing the model card on the Hub.

Evaluation

• Quantitative: validation loss tracked during training (report final number here). Consider automatic metrics only for short factual answers; they are imperfect for long, explanatory responses.
• Qualitative: manual review of bilingual outputs for factuality, citation quality, and hallucination. Human review by legal experts is recommended for production use.

Limitations & risks

• Hallucinations: the model may invent statutes, clause numbers, or case law. Treat outputs as tentative and verify with primary sources.
• Jurisdictional coverage: dataset likely focuses on specific legal texts; generalization across jurisdictions is limited.
• Bias & representativeness: training data may under‑represent certain topics or perspectives.
• Safety: do not use the model for high‑stakes legal decisions without expert oversight.

Usage example (conceptual)

• Prompt the model with the formatted chat string in Urdu or English and sample the assistant response. Always show a disclaimer in user‑facing apps indicating that outputs are for informational purposes only.

How to cite

Contact & maintainers

• Maintainer: haroon-ahmad on Hugging Face.
• For questions, issues, or collaboration request, contact via your Hugging Face profile.