GLiNER ContractNER Multi - Fine-Grained Legal Entity Extraction
Developed at Agile Lab by Luca Sorrentino and Annalisa Belia, with Irene Donato as project lead.
Model Name:gliner-contractner-multi-v2.1 (Agile Lab Fine-tune)
Base Architecture: GLiNER Multi v2.1 (Backbone: microsoft/mdeberta-v3-base)
Dataset:lucasorrentino/ContractNER
Model Description
GLiNER ContractNER Multi is a multilingual span-based Named Entity Recognition (NER) model fine-tuned by Agile Lab on the ContractNER dataset. It is designed to extract fine-grained entities from legal contracts with high precision.
Built on the GLiNER Multi v2.1 architecture, this model achieves 72.49% micro F1 on the held-out test set, significantly outperforming general-purpose LLMs and domain-specific legal models in our benchmarks.
Key Highlights
Contract-Specialized: Fine-tuned on 3,240+ annotated contract chunks from SEC EDGAR filings (lucasorrentino/ContractNER).
Granular Extraction: Capable of identifying 18 specific entity types including parties, dates, financial terms (salaries, shares), and regulatory references.
Open-Vocabulary NER: Supports promptable entity extraction—you can provide custom label names at inference time without retraining.
Multilingual Capability: Inherits multilingual behavior from GLiNER Multi v2.1 and mDeBERTa-v3-base, though optimized primarily for English contracts (performance may degrade on low-resource languages).
Production-Ready: A recommended threshold of 0.8–0.9 balances high precision with acceptable recall, minimizing costly false positives in legal review workflows.
🚀 How to Use
To use this model, you need to install the gliner library.
Installation
pip install gliner
Inference Code
python
1from gliner import GLiNER
23model = GLiNER.from_pretrained("lucasorrentino/Contractner")45# Example contract text6text ="""
7This EMPLOYMENT AGREEMENT is made effective as of January 1, 2026,
8by and between Tech Solutions Inc. ("Company") and Jane Doe ("Executive").
9The Executive shall serve as Chief Technology Officer.
10The Company agrees to pay the Executive an annual base salary of $250,000.00.
11"""1213# Define the entities you want to extract (Open Vocabulary)14labels =[15"Parties","EffectiveDate","Role","Salary","TerminationDate"16]1718# Predict19entities = model.predict_entities(text, labels, threshold=0.5)2021# Print results22for entity in entities:23print(f"{entity['text']} => {entity['label']} (Score: {entity['score']:.2f})")
📊 Evaluation & Benchmarks
Evaluated on the held-out test set (158 samples, 20% stratified split, not seen during training).
Reproduce with: uv run eval.py from the model repo.
Methodology:overlap_cover matching — a prediction counts as TP if it covers 100% of the gold span (boundary tolerance ±1 char), greedy 1-to-1 matching, micro-averaged P/R/F1.
Overall Results (threshold = 0.9)
Metric
Score
Micro F1
72.49%
Micro Precision
83.13%
Micro Recall
64.27%
Macro F1
72.94%
Threshold sweep
Performance vs. Other Approaches
Approach
F1
GLiNER ContractNER (this model)
72.49%
Standalone DeBERTa models
46–78%
General-purpose LLMs (Qwen, Gemma)
< 35%
Legal-specific models (LegalBERT, ContractBERT)
< 10%
Per-Entity Results (test set, threshold = 0.9)
Entity
Precision
Recall
F1
Support
PII_Ref
94.12%
94.12%
94.12%
17
EffectiveDate
95.83%
88.46%
92.00%
26
Salary
89.47%
94.44%
91.89%
18
Percentage
91.30%
87.50%
89.36%
24
Role
96.67%
82.86%
89.23%
35
Parties
85.71%
87.10%
86.40%
62
Shares
100.00%
73.68%
84.85%
19
Price
92.31%
75.00%
82.76%
16
Court
83.33%
75.00%
78.95%
20
RenewalTerm
66.67%
83.33%
74.07%
12
Principal
82.61%
61.29%
70.37%
31
Title
82.46%
60.26%
69.63%
78
Address
62.96%
77.27%
69.39%
22
TerminationDate
71.43%
55.56%
62.50%
18
Rent
66.67%
50.00%
57.14%
8
Ratio
85.71%
35.29%
50.00%
17
Act
85.00%
25.00%
38.64%
68
Regulation
47.37%
23.68%
31.58%
38
Note on Act and Regulation: low recall is due to truncation — these entities often appear in long legal clauses that exceed the model's 384-token context window. Use sliding window inference for full-document coverage.
Per-entity metrics
Supported Entity Schema
The model was trained on the ContractNER schema. While you can use custom labels, performance is best with categories semantically similar to:
Document Metadata
EffectiveDate: Contract start date (e.g., "January 1, 2026").
TerminationDate: Contract end or expiration date.
RenewalTerm: Renewal periods or conditions.
Title: Official document title.
Actors & Roles
Parties: Legal entities entering the agreement (companies, individuals).
Role: Professional titles and positions (e.g., "Chief Executive Officer").
Contact Information
Address: Physical addresses.
PII_Ref: Personal identifiable information references (phone, email, fax).
Financial Values
Salary: Compensation amounts (always with currency symbol, e.g., "$225,000.00").
Price: Goods/services prices.
Principal: Loan principal amounts.
Shares: Stock or equity quantities.
Percentage: Percentage values (e.g., "50%").
Ratio: Financial ratios.
Rent: Lease or rental amounts.
Legal and Regulatory
Court: Judicial bodies and tribunals (e.g., "State of Texas").
Dataset:lucasorrentino/ContractNER — Real contracts from SEC EDGAR (U.S. Securities and Exchange Commission filings), based on Adibhatla et al. (2023).
Original Size: ~5,000+ annotated contract segments.
Consolidated Dataset: ~3,240 chunks after stratified reduction and class consolidation.
Adjustments:
Removed RevolvingCredit class (too rare and ambiguous).
Rebalanced dataset to ensure minimum representation per class.
Split: 80% training / 20% validation (random split).
Architecture Type: Span-based NER with entity-query matching.
Hardware: NVIDIA L4 GPU.
Training Time: ~30 minutes per fine-tuning run.
🏢 Real-World Use Cases
Document Due Diligence
When a law firm or investment fund needs to analyze large volumes of contracts ahead of an acquisition or investment, this model accelerates the extraction of key information—parties involved, relevant dates, renewal terms, and financial values—reducing manual pre-screening effort and narrowing the scope of documents that require deep legal review.
Relevant entities:Parties, EffectiveDate, TerminationDate, RenewalTerm, Salary, Price, Principal
Contract Deadline Monitoring
By extracting EffectiveDate, TerminationDate, and RenewalTerm across a contract portfolio, organizations can power alert systems that proactively surface upcoming expirations, auto-renewals, and notice windows—improving operational control over contractual obligations at scale.
When a legal team needs to assess the impact of new regulatory requirements or internal policy changes across active contracts, the model helps quickly identify documents containing regulatory references, specific parties, and relevant dates—narrowing the audit perimeter to contracts that actually warrant deeper review.
ONNX export (model.onnx) is included in the repository.
Latency Benchmark
Hardware: MacBook Pro — Apple M1 Pro (8-core, 6P+2E), 16 GB RAM Runs: 20 (after 3 warmup) · Threshold: 0.9
Backend
Input length
Median
p95
Throughput
PyTorch CPU
Short (~300 chars)
134 ms
148 ms
~7.4 docs/s
PyTorch CPU
Medium (~800 chars)
202 ms
216 ms
~5.0 docs/s
PyTorch CPU
Full chunk (~1,000 chars)
261 ms
275 ms
~3.8 docs/s
ONNX Runtime CPU
Full chunk (~1,000 chars)
308 ms
—
~3.2 docs/s
ONNX Runtime on Apple M1 is slightly slower than PyTorch due to Apple Silicon optimizations in PyTorch's ARM kernels. On x86 Linux (e.g., AWS c5), ONNX typically delivers 1.5–2× speedup over PyTorch CPU.
For documents longer than 384 tokens, use sliding window inference to avoid truncation.
Cost vs. LLM-Based Extraction
Approach
F1 on ContractNER
Cost / 1,000 docs
This model (CPU, self-hosted)
72.49%
≈$0.014
GPT-4o-mini
< 35%
≈$0.07
GPT-4o
< 35%
≈$0.33
Self-hosted cost assumes AWS c5.xlarge on-demand ($0.17/hr, ≈3.8 docs/s, full utilization). LLM costs estimated at list pricing (≈250 input + ≈50 output tokens per chunk).
