Base encoder: deepvk/RuModernBERT-base.
Continued pretraining: MLM on irlspbru/RusLawOD.
Inference heads:
- doc_type: document type classification from doc_typeIPS.
- classifier: multi-label legal classifier from classifierByIPS.
- keywords: multi-label keyword prediction from keywordsByIPS.
- ner: token classification from TryDotAtwo/russian-legal-ner.

headingIPS and textIPS are used together as model input for document-level heads.

Usage

python

from legal_modernbert import LegalDocumentPipeline

pipe = LegalDocumentPipeline.from_pretrained("TryDotAtwo/RuModernBERT-ruLaw")
result = pipe("Текст правового документа...")

The pipeline requires a FlashAttention 2 compatible runtime.

Data Sources

Source	Used for	Link
RuModernBERT-base	Initial encoder weights	`deepvk/RuModernBERT-base`
RusLawOD	MLM, document type, classifier, keywords	`irlspbru/RusLawOD`
Russian legal NER	NER head	`TryDotAtwo/russian-legal-ner`
Sud-resh benchmark	External MLM validation	`lawful-good-project/sud-resh-benchmark`

Benchmarks

Evaluation	Metric	Result
RusLawOD MLM validation	eval loss	0.1337
RusLawOD MLM validation	train loss	0.1537
Sud-resh MLM benchmark, this model	eval loss	0.4473
Sud-resh MLM benchmark, base `deepvk/RuModernBERT-base`	eval loss	0.5172
Sud-resh MLM benchmark	perplexity improvement vs base	~6.8%
NER test	precision	0.9970
NER test	recall	0.9884
NER test	F1	0.9927
NER test	loss	0.00133
Multitask document heads	final train loss	0.0193

The NER dataset mirror keeps attribution to the original authors and source folder in its dataset card.

Limitations

The benchmark table reports internal training/evaluation runs. Document-head quality should be validated on held-out downstream legal tasks before production use.