allenai/scibert_scivocab_uncased fine-tuned as a binary classifier of citation intent: does a citation sentence cite prior work as background (BACKGROUND) or for any other reason (NOT_BACKGROUND)?

Labels

id	label	meaning
0	NOT_BACKGROUND	citation used for method, comparison, extension, motivation…
1	BACKGROUND	citation provides background/context for the citing paper

How to use

python

from transformers import pipeline

clf = pipeline("text-classification", model="hongccccccc/scibert-citation-background-classifier")
clf("Deep learning has achieved remarkable success in many NLP tasks [1, 2].")
# [{'label': 'BACKGROUND', 'score': ...}]

Training

Base model: SciBERT (uncased, scivocab; BertForSequenceClassification, single-label, 2 classes)
Data: the combined annotated citation contexts of SciCite (Cohan et al., 2019) and MultiCite (Lauscher et al., 2022) — 27,052 instances, 11,635 (43%) labeled BACKGROUND
Fine-tuned: August 2023, transformers 4.32.0

Evaluation

F1 score of 0.81 on a held-out test set (80-20 train-test split of the combined dataset).