CUAD answers filtered to max 8-15 words per entity
Jurisdiction spans narrowed using keyword extraction
AMOUNT spans verified to contain digits/currency symbols
TERM spans extracted via legal keyword matching
Placeholder dates (____) removed
Stratified 70/15/15 split
Model Architecture
Base : nlpaueb/legal-bert-base-uncased
Parameters : 108,900,107
Head : Linear(768 → 11 labels)
Dropout : 0.2 (hidden + attention)
Labels : 11 (O + B/I for 5 entities)
Max length : 320 tokens
Training Configuration
Epochs : 8 (early stopping patience=3)
Best epoch : 6
Learning rate : 3e-5
Batch size : 8 per GPU × 2 GPUs × 2 accum = 32
Weight decay : 0.01
Warmup steps : 71 (10% of total)
Loss function : Standard CrossEntropyLoss
FP16 : True
Grad checkpoint : True
Performance on Test Set
Entity
Precision
Recall
F1
TERM
0.833
0.941
0.884
JURISDICTION
0.788
0.863
0.824
AMOUNT
0.587
0.746
0.657
DATE
0.440
0.659
0.528
PARTY
0.000
0.000
0.000
Micro F1
0.634
Train / Val / Test Summary
Split
Micro F1
Train
0.663
Validation
0.595
Test
0.634
Train-Test gap = 0.029 → no overfitting ✅
Known Limitations
PARTY F1 = 0.000
CUAD PARTY answers are role names: "Distributor",
"Licensee", "Company". These are indistinguishable
from regular nouns. Model trained on Run 1 could not
learn any signal for PARTY detection.
Fix in v2: replace CUAD PARTY with spaCy ORG/PERSON
entities from LEDGAR → proper names like "Apple Inc",
"Microsoft Corporation" provide clear visual signal.
DATE Precision = 0.440
DATE spans from CUAD include surrounding words:
"7th day of September , 1999 ." instead of
"September 1999". Model learned wide boundaries.
Fix in v2: filter DATE spans starting with lowercase
words. Keep only spans starting with digits or
month names.
What Works Well
TERM F1 0.884 → keyword-extracted spans are clean
JURISDICTION F1 0.824 → narrowing function effective
No overfitting → train/test gap only 2.9%
Runs on CPU → backend compatible
Superseded By
Devil1710/Legal-NER-v2 (in progress)
Fixes: PARTY proper names, DATE boundary cleanup
Target: Micro F1 > 0.75
