Base Model: meta-llama/Llama-3.2-3B-Instruct
Training Method: QLoRA (4-bit quantization)
Dataset: ACORD (Expert-annotated legal clauses)
Training Time: 174 minutes on Kaggle T4 x2
Hardware: 2x NVIDIA Tesla T4 (32GB total)
LoRA Parameters:
- Rank (r): 32
- Alpha: 64
- Dropout: 0.05
- Target modules: All linear layers

Usage

python

from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel
import torch

# Load base model and tokenizer
base_model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.2-3B-Instruct",
    torch_dtype=torch.float16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B-Instruct")

# Load LoRA adapters
model = PeftModel.from_pretrained(base_model, "{REPO_ID}")

# Extract clause
prompt = \"\"\"<|begin_of_text|><|start_header_id|>user<|end_header_id|>

Copy the exact 'Indemnification' clause from this contract:

[Your contract text here]
<|eot_id|><|start_header_id|>assistant<|end_header_id|>

\"\"\"

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=200)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Metrics

Metric	Value
Training Loss	0.407
Average Similarity	73.6%
Excellent (>90%)	39%
Good (>70%)	60%
Partial (>50%)	78%

Supported Clause Types

Indemnification
Limitation of Liability
Termination
Confidentiality
Governing Law
Insurance
Change of Control
Expiration Date
Non-Compete

Evaluation

Tested on 100 samples from ACORD test set. Model shows strong performance on complex legal clauses.

Limitations

Trained primarily on US contracts (SEC filings, Fortune 500 ToS)
May not generalize well to contracts from other jurisdictions
Best performance on standard commercial contracts
Not a substitute for legal advice

Citation

bibtex

@misc{{llama32-legal-clause-extractor,
  author = {{{HF_USERNAME}}},
  title = {{Legal Clause Extraction with Llama 3.2 3B}},
  year = {{2025}},
  publisher = {{HuggingFace}},
  url = {{https://huggingface.co/{REPO_ID}}}
}}

License

Apache 2.0 (following Llama 3.2 license)

Acknowledgments

Dataset: ACORD (The Atticus Project)
Base Model: Meta Llama 3.2
Training Platform: Kaggle T4 GPU

Munna-K/llama-3.2-3b-legal-clause-extractor

Legal Clause Extraction Model - Llama 3.2 3B

Model Description

Training Details

Usage

Training Metrics

Supported Clause Types

Evaluation

Limitations

Citation

License

Acknowledgments

Munna-K/
llama-3.2-3b-legal-clause-extractor