ModernBERT-base for primary legal-topic classification
This checkpoint assigns a primary legal topic to conversations in which the
user is seeking legal guidance. It is the second stage of the full-conversation
ModernBERT-base cascade from the
Legal Guidance in the Wild study. Input
contains chronological user messages only; assistant messages are excluded.
The first-stage detector is
modernbert-base-v3-seeks-guidance-user-len4096-seed42.
This is a research classifier, not a legal-advice system. Its topic prediction
must not be treated as a determination of jurisdiction, rights, or legal merit.
Labels
The 14 labels are: FAMILY_AND_ESTATES, HOUSING_AND_PROPERTY,
EMPLOYMENT_AND_LABOR, IMMIGRATION_AND_CITIZENSHIP, CRIMINAL_LAW,
TAX_LAW, CONSUMER_AND_PERSONAL_FINANCE, BUSINESS_AND_COMMERCIAL,
TORTS_AND_CIVIL_LIABILITY, GOVERNMENT_AND_ADMINISTRATIVE,
CIVIL_RIGHTS_AND_CONSTITUTIONAL, DATA_PRIVACY_AND_TECHNOLOGY,
INTELLECTUAL_PROPERTY, and OTHER. The exact ID mapping is stored in
config.json.
Data
- Dataset: AmirMohseni/WildChat-Legal-Classification-V3-Hierarchical
- Requested revision:
main (latest at run time)
- Train fingerprint:
6ba4c2696e889276
- Validation fingerprint:
403fe118d76c8360
- Topic-stage train / validation rows: 750 / 134 guidance-positive conversations
- Input mode: chronological user messages only
Dataset access follows the linked repository's sharing settings. The
fingerprints identify the exact loaded splits even if main later changes.
Training configuration
| Setting | Value |
|---|
| Base model | answerdotai/ModernBERT-base |
| Maximum length | 4096 |
| Epochs | 10 |
| Learning rate | 6e-05 |
| Effective batch size | 32 |
| Weight decay | 0.01 |
| Class weighting | Yes |
| Seed | 42 |
| Hardware | NVIDIA A100-SXM4-40GB |
The checkpoint was selected by silver-validation macro-F1.
Silver-validation results
| Evaluation | Accuracy | Macro-F1 |
|---|
| Conditional topic stage (N=134) | 0.724 | 0.603 |
| Full base cascade (15-way, N=290) | 0.762 | 0.550 |
The paired first-stage guidance detector obtains 0.879 macro-F1 and 0.875
positive-class F1. These development results are not final adjudicated-gold
estimates.
Inference
1from transformers import AutoModelForSequenceClassification, AutoTokenizer
2
3repo = "AmirMohseni/modernbert-base-v3-primary-topic-user-len4096-seed42"
4tokenizer = AutoTokenizer.from_pretrained(repo)
5model = AutoModelForSequenceClassification.from_pretrained(repo)
6
7# Run only after the paired guidance detector predicts guidance-seeking.
8inputs = tokenizer(user_only_conversation, return_tensors="pt", truncation=True,
9 max_length=4096)
10topic_id = int(model(**inputs).logits.argmax(-1)[0])
11topic = model.config.id2label[topic_id]
Limitations
The model was trained on English-language public LLM interaction logs with
silver labels, one source, and one seed. Several topics have limited validation
support, so macro-F1 is unstable and topic-specific errors can be substantial.
Inputs beyond 4,096 tokens are truncated. The taxonomy is jurisdiction-agnostic
and cannot substitute for legal triage by a qualified professional. Human
review is required for consequential use.
Citation
Please cite the accompanying Legal Guidance in the Wild: How Users Seek Legal
Help in Real-World LLM Conversations manuscript when it becomes available.