This is a LORA adapter trained for automated legal document redlining. The model was fine-tuned on legal contracts to suggest protective edits that reduce risk for the reviewing party.

Base Model: Qwen/Qwen2.5-7B-Instruct Training Dataset: UmaiTech/legal-contract-gpt41-redlining-10k Adapter Type: LORA Training Date: 2025-11-11

Intended Use

This model is designed to assist legal professionals in reviewing and redlining contracts. It suggests edits to:

Add protective clauses
Strengthen indemnification language
Clarify termination rights
Reduce liability exposure
Improve notice requirements

Important: This model is an AI assistant tool. All redlines should be reviewed by qualified attorneys before use in actual contracts.

Training Details

Training Configuration

Adapter Type: LORA
LoRA Rank (r): 64
LoRA Alpha: 128
Target Modules: k_proj, v_proj, o_proj, q_proj

Training Data

The model was trained on UmaiTech/legal-contract-gpt41-redlining-10k containing pairs of original contract clauses and their redlined versions, generated using GPT-4.1.

Usage

Load Model with PEFT

python

from peft import PeftModel, PeftConfig
from transformers import AutoModelForCausalLM, AutoTokenizer

# Load adapter config
config = PeftConfig.from_pretrained("UmaiTech/qwen-2.5-7b-redline-llm-v1")

# Load base model
model = AutoModelForCausalLM.from_pretrained(
    config.base_model_name_or_path,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained(config.base_model_name_or_path)

# Load adapter
model = PeftModel.from_pretrained(model, "UmaiTech/qwen-2.5-7b-redline-llm-v1")

# Generate redline
prompt = """Redline this contract clause to be more protective:

Original: Either party may terminate this Agreement at will without notice.

Redlined:"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)
redline = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(redline)

Merge Adapter into Base Model

python

from peft import PeftModel
from transformers import AutoModelForCausalLM

# Load and merge
base_model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
model = PeftModel.from_pretrained(base_model, "UmaiTech/qwen-2.5-7b-redline-llm-v1")
merged_model = model.merge_and_unload()

# Save merged model
merged_model.save_pretrained("./merged_model")

Performance

Test Set: 100 samples from legal-contract-gpt41-redlining-10k test split

Model	Overall Score	BERTScore	ROUGE-L	BLEU	Edit Sim	Clause Pres	Latency
Qwen 2.5 7B (This Model)	0.5591	0.6895	0.4509	0.2784	0.4064	0.9024	4.6s
GPT-4.1-mini (Baseline)	0.5632	0.6841	0.4694	0.2918	0.3905	0.8805	2.6s

Key Findings:

✅ Achieves 99.3% of GPT-4.1-mini quality (0.5591 vs 0.5632)
✅ Better clause preservation (0.9024 vs 0.8805) - maintains legal concepts better
✅ Self-hosted - no API dependency, full control over data
⚡ 1.8x slower (4.6s vs 2.6s) but acceptable for batch processing

Metrics Explained:

Overall Score: Weighted average (BERTScore 40%, ROUGE-L 20%, Edit Sim 20%, Clause Pres 20%)
BERTScore: Semantic similarity using BERT embeddings (0-1, higher is better)
ROUGE-L: Longest common subsequence overlap (0-1, higher is better)
BLEU: N-gram precision metric (0-1, higher is better)
Edit Similarity: Character-level similarity (0-1, higher is better)
Clause Preservation: Legal concept retention (0-1, higher is better)

Limitations

Domain Specific: Trained on legal contracts, may not generalize to other domains
US-Centric: Training data is primarily US law-based
Context Length: Limited by base model context window
Not Legal Advice: Outputs should be reviewed by qualified attorneys
Hallucination Risk: May generate plausible but incorrect legal language

Ethical Considerations

This model is a tool to assist legal professionals, not replace them
Always have trained attorneys review AI-generated redlines
Be aware of jurisdiction-specific legal requirements
Consider data privacy when processing confidential contracts

Citation

bibtex

@misc{qwen_2.5_7b_redline_llm_v1_2025},
  author = {UmaiTech},
  title = {qwen-2.5-7b-redline-llm-v1: Fine-tuned Model for Legal Contract Redlining},
  year = {2025},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/UmaiTech/qwen-2.5-7b-redline-llm-v1}}
}

License

This adapter is released under the Apache 2.0 license. The base model may have different licensing terms.

Contact

For questions or issues:

Open an issue in the repository
Contact: UmaiTech

Disclaimer: This model is for research and assistant purposes. Always consult qualified legal counsel for actual contract work.