This model is a fine-tuned version of allenai/specter2_base for multilabel scientific domain classification aligned with ERC panel taxonomy.
It achieves the following results on the held-out test set:
Best validation loss: 0.0361
Micro F1: 0.9386
Micro ROC-AUC: 0.9718
Subset accuracy: 0.7943
Model description
This model is a fine-tuned variant of SPECTER2 (allenai/specter2_base) adapted for multilabel classification of scientific documents into ERC research panels.
The model takes as input the title and abstract of a scientific publication and predicts one or more research panels.
Since scientific outputs may legitimately span multiple domains, the model is trained using sigmoid activation with binary cross-entropy loss, allowing independent assignment of multiple labels.
Key characteristics
Base model: allenai/specter2_base
Task: multilabel document classification
Labels: 28 ERC scientific panels
Activation: sigmoid (independent scores per label)
Loss: BCEWithLogitsLoss
Output: list of predicted panels with associated probabilities
Decision threshold: 0.5 (tunable)
This model enables automatic research-domain tagging aligned with the ERC panel structure.
Intended uses & limitations
Intended uses
This model is designed for:
Automatic assignment of ERC research panels
Metadata enrichment for:
research project databases
institutional repositories
funding and grant analysis pipelines
Large-scale analytics such as:
portfolio mapping
thematic analysis of research outputs
monitoring disciplinary coverage of funded projects
Predicting subject areas for documents lacking structured domain metadata
The model supports:
title only
abstract only
title + abstract (recommended)
Limitations
ERC panels are high-level categories and do not represent fine-grained subdisciplines
Labels are derived from curated datasets, semi-automatically annotated data
Class imbalance may affect recall for underrepresented panels
The model does not encode explicit hierarchical relationships between panels
Not suited for:
fine-grained subfield classification
journal recommendation
evaluation of research quality or impact
clinical, legal, or regulatory decision-making
Predictions should be treated as supportive metadata, not authoritative classifications.
How to use
from transformers import pipeline
# Replace with your actual model repo name on HuggingFace
MODEL_NAME = "nicolauduran45/erc_classifier_demo"
classifier = pipeline(task="text-classification", model=MODEL_NAME, tokenizer=MODEL_NAME)
text = ["Climate change impacts on Arctic ecosystems."]
classifier(text)
Training and evaluation data
Training data
Scientific documents with ERC-style panel annotations
Inputs:
title
abstract
Task type: multilabel classification
Dataset characteristics
Property
Value
Documents
~40k
Labels
28 panels
Input fields
Title, Abstract
Task type
Multilabel
License
Dataset-dependent
Training procedure
Preprocessing
Input text constructed as:
title + ". " + abstract
Tokenization using the SPECTER2 tokenizer
Maximum sequence length: 512 tokens
Model
Base model: allenai/specter2_base
Classification head: linear → sigmoid
Loss function: BCEWithLogitsLoss
Predictions: independent probability per label
Training hyperparameters
Hyperparameter
Value
Learning rate
2e-5
Train batch size
16
Eval batch size
16
Epochs
6
Weight decay
0.01
Optimizer
AdamW
Metric for best model
Micro F1
Training results
Epoch
Training Loss
Validation Loss
Micro F1
ROC-AUC
Accuracy
1
0.2089
0.0968
0.7576
0.8347
0.4043
2
0.0961
0.0713
0.8231
0.8888
0.5171
3
0.0719
0.0578
0.8614
0.9209
0.5829
4
0.0579
0.0458
0.9072
0.9546
0.7029
5
0.0479
0.0390
0.9264
0.9620
0.7614
6
0.0407
0.0361
0.9386
0.9718
0.7943
Evaluation results (multilabel test set)
Panel
Precision
Recall
F1-score
Support
Biotechnology and Biosystems Engineering
0.88
0.70
0.78
30
Cell Biology, Development, Stem Cells and Regeneration
0.98
0.94
0.96
54
Computer Science and Informatics
0.96
0.98
0.97
95
Condensed Matter Physics
0.97
0.99
0.98
68
Earth System Science
0.94
0.98
0.96
64
Environmental Biology, Ecology and Evolution
0.91
0.96
0.94
54
Fundamental Constituents of Matter
0.97
0.94
0.95
32
Human Mobility, Environment, and Space
0.81
0.81
0.81
21
Immunity, Infection and Immunotherapy
1.00
0.97
0.99
40
Individuals, Markets and Organisations
0.94
0.98
0.96
48
Institutions, Governance and Legal Systems
0.89
0.92
0.91
26
Integrative Biology: from Genes and Genomes to Systems
0.91
0.98
0.94
49
Materials Engineering
0.81
0.93
0.87
75
Mathematics
1.00
1.00
1.00
36
Molecules of Life: Biological Mechanisms, Structures and Functions
0.94
0.98
0.96
111
Neuroscience and Disorders of the Nervous System
1.00
1.00
1.00
30
Physical and Analytical Chemical Sciences
0.89
0.93
0.91
94
Physiology in Health, Disease and Ageing
0.94
1.00
0.97
34
Prevention, Diagnosis and Treatment of Human Diseases
This evaluation uses ERC-funded projects, where each project belongs to exactly one panel.
Only recall is reported.
Panel
Recall
Biotechnology and Biosystems Engineering
0.26
Cell Biology, Development, Stem Cells and Regeneration
0.81
Computer Science and Informatics
1.00
Condensed Matter Physics
0.77
Earth System Science
0.92
Environmental Biology, Ecology and Evolution
0.85
Fundamental Constituents of Matter
0.84
Human Mobility, Environment, and Space
0.61
Immunity, Infection and Immunotherapy
0.83
Individuals, Markets and Organisations
0.96
Institutions, Governance and Legal Systems
0.58
Integrative Biology: from Genes and Genomes to Systems
0.73
Materials Engineering
0.75
Mathematics
0.96
Molecules of Life: Biological Mechanisms, Structures and Functions
0.95
Neuroscience and Disorders of the Nervous System
0.92
Physical and Analytical Chemical Sciences
0.83
Physiology in Health, Disease and Ageing
0.60
Prevention, Diagnosis and Treatment of Human Diseases
0.94
Products and Processes Engineering
0.58
Studies of Cultures and Arts
0.27
Synthetic Chemistry and Materials
0.67
Systems and Communication Engineering
0.75
Texts and Concepts
0.62
The Human Mind and Its Complexity
0.85
The Social World and Its Interactions
0.73
The Study of the Human Past
0.83
Universe Sciences
1.00
Overall performanceOverall recall
Micro recall: 0.77
Macro recall: 0.76
Citation
@inproceedings{bovenzi2022mapping,
title={Mapping STI ecosystems via Open Data: Overcoming the limitations of conflicting taxonomies. A case study for Climate Change Research in Denmark},
author={Bovenzi, Nicandro and Duran-Silva, Nicolau and Massucci, Francesco Alessandro and Multari, Francesco and Parra-Rojas, C{\'e}sar and Pujol-Llatse, Josep},
booktitle={International Conference on Theory and Practice of Digital Libraries (TPDL)},
pages={495--499},
year={2022},
publisher={Springer International Publishing}
}
