BioProCorpus: a corpus for robustness-oriented biomedical NER benchmarking and HybridNER evaluation under heterogeneous conditions
This repository accompanies our study on robustness-oriented biomedical named entity recognition (BioNER). It provides the resources needed to reproduce the benchmark construction, baseline experiments, and HybridNER results reported in the paper.
The repository includes:
dataset splits for four BioNER corpora,
training and evaluation code for five baseline models,
released checkpoints and prediction outputs,
HybridNER training code and trained models,
original data preprocessing and split-generation code for BioProCorpus.
1. Brief Paper Overview
BioNER is commonly evaluated on homogeneous benchmarks, which can mask issues with robustness and cross-domain generalization. We introduce BioProCorpus, a task-oriented corpus from iProX proteomics project descriptions, designed to emphasize lexical variability, boundary ambiguity, and long-range context. Under a unified protocol, we benchmark BigBird, BioBERT, DeBERTa, KeBioLM, and BINDER across four datasets and propose HybridNER, which achieves the best performance on BioProCorpus.
Four datasets are provided with train/development/test splits:
Revised JNLPBA
BioRED
AnatEM
BioProCorpus
These datasets are organized for unified BioNER benchmarking under the same training and evaluation protocol.
2.2 Baseline Training Code
The folder for baseline experiments contains notebook-based training, validation, and testing code for:
BigBird
BioBERT
DeBERTa
KeBioLM
It also includes the BINDER-model/ subfolder for BINDER experiments on all four datasets.
2.3 Best models and Predictions
This folder stores:
Corresponding prediction model (e.g., best test predictions)
2.4 HybridNER Training and Released Models
2.1 HybridNER_training/ contains HybridNER training code.
2.2 Trained_models/ provides two released model checkpoints:
model.pt
model-PG_emphasis.pt
2.5 Original Data Processing Code
The folder 2.3Original_data_processing_code/ contains the preprocessing notebooks used to construct and inspect BioProCorpus before model training. These notebooks cover:
whitespace and format cleaning,
entity-term statistics,
entity-type inspection,
distribution balance checking,
stratified data splitting,
cross-file data split analysis, and
corpus split generation.
This part is included to support reproducibility from the original annotated data to the final benchmark-ready train/development/test partitions.
3. Environment Setup
3.1 Recommended System Environment
The experiments were developed and tested in a Linux-based environment. A typical setup is:
OS: Ubuntu 22.04
Python: 3.10 or 3.11
CUDA: 12.1
PyTorch: 2.3.0
GPU: NVIDIA RTX 3090 / RTX 4090 or equivalent
A GPU environment is strongly recommended for both baseline training and HybridNER training.
eval_full_report.py: evaluation and report generation
Please ensure that dataset paths, pretrained model paths, output directories, and GPU settings are configured correctly in each script before execution.
4.4 Step 4: Use Released Checkpoints
If you do not want to retrain HybridNER from scratch, you can directly use the released checkpoints in:
2.2HybridNER_trained/
Available checkpoints:
model.pt
model-PG_emphasis.pt
Please load the desired checkpoint in the evaluation script and update the checkpoint path accordingly.
5. Notes on Data and Splits
BioProCorpus is organized into fixed training, development, and test sets for all downstream experiments. The benchmark split was designed to support fair comparison across all baseline models and HybridNER. The preprocessing notebooks in 2.3Original_data_processing_code/ document the main steps used to prepare the final corpus partitions and inspect benchmark composition.
6. Availability and Implementation
The BioProCorpus dataset, HybridNER source code, preprocessing notebooks, split-generation scripts, released checkpoints, and related benchmark resources used in this study are freely available for non-commercial use at:
If you use this repository or BioProCorpus in your research, please cite the corresponding paper.
@article{your_paper_here,
title = {BioProCorpus: A corpus for robustness-oriented biomedical NER benchmarking and HybridNER evaluation under heterogeneous conditions},
author = {Liu, Shengyu and others},
journal = {To be updated},
year = {2026}
}
8. Contact
For questions about the dataset, code, or reproducibility, please contact:
