This multilingual clinical Named Entity Recognition (NER) model is designed to identify medical procedure mentions in biomedical and clinical text. It is based on xlm-roberta-base and fine-tuned on translated variants of the clinical NER datasets MedProcNER and CardioCCC, which consist of clinical case reports with manually annotated mentions of medical procedures, following a multi-task learning (MTL) approach and using the BIO tagging scheme for sequence labeling.

The model consists of a shared multilingual encoder and a set of language-specific token classification heads, each one being responsible for a different task. In this configuration, each classification head is trained on language-specific data manually annotated with medical procedure mentions.

Architecture: Multi-task learning (MTL)
Training setup: Multilingual, Monolabel (PROCEDURE)
Tasks: procedure_en_ner, procedure_nl_ner, procedure_sv_ner
Supported languages:
- English (en)
- Dutch (nl)
- Swedish (sv)
Base model: xlm-roberta-base
Task: Token classification (NER)
Label scheme: BIO

Training Data

The model is trained on multilingual clinical NER data from the MedProcNER and CardioCCC datasets across the supported languages. In this MTL set up, each classification head is trained to perform a specific task, which is the identification of medical procedure mentions in clinical and biomedical texts written in its assigned language. For that, each classification head is fine-tuned on language-specific data where medical procedures have been manually labeled. During training, the shared part of the model learns general medical and linguistic patterns across all languages while each classification head is fine-tuned on language-specific annotated data, so it becomes better at recognizing medical procedure names in that particular language.

The training data is provided as part of the MultiClinNER subtask of the MultiClinAI shared task, an initiative as part of the DataTools4Heart (DT4H) project, which provides translated and annotation-projected clinical corpora.

Training and test splits correspond to the MultiClinNER task at the 11th SMM4H-HeaRD Workshop (ACL 2026).

Model loading

This model uses a custom MTL architecture, and therefore cannot be loaded with:

AutoModelForTokenClassification.from_pretrained(...)

Instead, this repository provides a PyTorch checkpoint (.pt) which includes:

Encoder weights
All language-specific heads

How to use

To use the model:

Download the .pt file
Load it using the custom architecture

To facilitate this process, we provide an inference script in a GitHub repository that:

Loads the model from the checkpoint using the custom architecture
Processes .txt files from an input directory
Identifies medical procedure mentions in the language handled by the selected classification head (task)
Exports predictions as a TSV file in the format required for the MultiClinAI evaluation library:

text

filename                                label        start_span    end_span    text
MultiClinNER-sv-test-procedure-47685    PROCEDURE    979           999         intraokulära trycket

⚠ Note: We recommend pre-tokenizing the input text into words, as this matches the training setup. Providing raw text directly may lead to slightly degraded performance.

Limitations and bias

At the time of submission, no formal bias or fairness evaluation has been conducted. We intend to conduct research in these areas in the future, and if completed, this model card will be updated.

Evaluation

Evaluation was conducted using strict (exact match) and character-level metrics on the MultiClinNER test set.

Language	Strict P	Strict R	Strict F1	Char P	Char R	Char F1
en	0.6017	0.5845	0.5930	0.7776	0.7427	0.7597
nl	0.6338	0.6372	0.6355	0.7663	0.7612	0.7637
sv	0.6734	0.6467	0.6598	0.7992	0.7584	0.7783
Average			0.6294			0.7672

Additional information

Authors

NLP4BIA team at the Barcelona Supercomputing Center (nlp4bia@bsc.es).

Contact information

judith.rosell [at] bsc.es

Funding

This model is part of the DataTools4Heart project, funded by the European Union’s Horizon Europe Framework Under Grant Agreement No. 101057849.

BSC-NLP4BIA/DT4H_XLM-R_mtl_en-nl-sv_procedure

DT4H_XLM-R_mtl_en-nl-sv_procedure

Model Description