VTE-BERT-DeID is a Bio_ClinicalBERT-based sequence-classification model for identifying venous thromboembolism (VTE) phenotypes in English clinical narratives. It was fine-tuned on de-identified clinical text and evaluated on the corresponding original clinical text.

Important: “DeID” indicates that de-identified text was used for model fine-tuning. This model classifies VTE phenotypes; it does not perform de-identification.

This checkpoint uses the same underlying annotated dataset, label definitions, and train/test split described for VTE-BERT. The original publication describes the study cohort, clinical annotation process, label development, and validation design.

Model details

Developed by: Omid Jafari
Research affiliation and acknowledgement: Ang Li Lab
Base model: Bio_ClinicalBERT
Architecture: BertForSequenceClassification
Task: Single-label text classification
Language: English
Maximum sequence length: 512 tokens
Number of labels: 6
Release status: Public research release

Labels

ID	Label
0	LE-DVT+
1	PE+
2	UE-DVT
3	atypical
4	history
5	none

Operational definitions for these labels and the annotation procedures are provided in the VTE-BERT publication.

Data preparation and study design

The underlying clinical data and predefined train/test split are the same as those used in the original VTE-BERT study.

Fine-tuning inputs: Clinical text de-identified with Health-DeID and then prepared for modeling with NLPMed-Engine.
Test inputs: The corresponding original, non-de-identified clinical text from the held-out test set, prepared with the same NLPMed-Engine preprocessing workflow.
Distributed data: No training or evaluation data is included in this repository.

Fine-tuning

Parameter	Value
Training text field	final_deidentified_text
Maximum sequence length	512
Learning rate	2e-5
Weight decay	0.01
Maximum training epochs	30
Early-stopping patience	5
Training batch size	8
Evaluation batch size	8
Gradient accumulation steps	1
Model-selection metric	eval_pos_f1

The checkpoint was exported and tested with Transformers 5.10.1.

Evaluation

Numerical evaluation results are not included in this model card at this time.

Usage

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer, pipeline

model_name = "omid-jf/VTE-BERT-DeID"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

classifier = pipeline(
    "text-classification",
    model=model,
    tokenizer=tokenizer,
)

result = classifier(
    "Your clinical text here.",
    truncation=True,
    max_length=512,
)

print(result)

For an end-to-end preprocessing and inference workflow, use NLPMed-Engine. Its MLInference component can load Hugging Face sequence-classification checkpoints such as this model.

Intended use and limitations

This model is intended for research involving retrospective VTE phenotyping in clinical narratives.

It has not been validated for diagnosis, treatment selection, autonomous patient screening, or other clinical decision-making. Performance may vary across institutions, patient populations, note types, documentation practices, preprocessing configurations, and clinical settings. Users are responsible for obtaining appropriate institutional approvals and implementing applicable privacy and security controls.

License

This model is licensed under the GNU Affero General Public License v3.0. The research-use and clinical-safety statements in this model card describe the intended use and known limitations; they do not replace or add restrictions to the AGPLv3 license.

Related resources

Original gated model: VTE-BERT on Hugging Face
VTE-BERT publication: Development and Validation of VTE-BERT Natural Language Processing Model for Venous Thromboembolism
Preprocessing and inference: NLPMed-Engine
De-identification pipeline: Health-DeID

Citation

For the dataset, annotation process, label definitions, and original VTE-BERT study, please cite:

bibtex

@article{jafaridevelopment,
  title={Development and Validation of VTE-BERT Natural Language Processing Model for Venous Thromboembolism},
  author={Jafari, Omid and Ma, Shengling and Lam, Barbara D and Jiang, Jun Y and Zhou, Emily and Ranjan, Mrinal and Ryu, Justine and Bandyo, Raka and Maghsoudi, Arash and Peng, Bo and others},
  journal={Journal of Thrombosis and Haemostasis},
  publisher={Elsevier},
  year={2025},
  doi={10.1016/j.jtha.2025.07.021}
}