You can deploy this model and start earning money today!

scienceverse/
bibr-segmenter-v1

Name: bibr-segmenter-v1
Availability: InStock

pytorch

bibr

scientific-references

sequence-labeling

crf

token-classification

answerdotai/ModernBERT-base

finetune

apache-2.0

Views

No views yet

bibr-segmenter-v1

ModernBERT-base + CRF reference segmenter for the bibr scientific-paper extraction pipeline. Takes the references section of a paper and emits per-character BIO labels (O, B-REF, I-REF) to split it into individual references.

Architecture: answerdotai/ModernBERT-base encoder + linear head + torchcrf.CRF decode with BIO transition constraints.
Input: the raw concatenated references-section text, one logical reference per line (newlines are the segmenter's training-time boundary marker — internal column-wrap whitespace should be collapsed per row first; see bibr/extract/extractor.py:_extract_references_ner).
Output: a list of reference strings.
Inference: sliding-window with window=2048, stride=1536, overlap-merge via per-position trust score.
Training corpus: gold-anchored, judge-corrected reference lines from psych250 + Directorate-General Economics + MDPI Social Sciences (~440 papers, 2026-05-09/10).

Held-out validation (2026-05-10)

87% of papers within ±10% of gold reference count.
99% within ±20%.
Exact ref-count match on the 3-paper psych APA holdout.

Loading

python

from bibr.ner.segmenter import RefSegmenter
seg = RefSegmenter("scienceverse/bibr-segmenter-v1")
refs = seg.segment(references_section_text)

The RefSegmenter resolver downloads the checkpoint via huggingface_hub.hf_hub_download (filename: seg_v3_gold.pt).

Caveats

In-distribution: APA-style social-science references.
Out-of-distribution: arXiv-style bracketed-numbered refs ([1] Author...) and IEEE conference proceedings — paired with bibr-parser-v1 these recover ~94% of LLM ref count but container-field accuracy drops.
Acknowledgement lines that leak into the references-section input occasionally pass through as junk "refs"; bibr's downstream completeness filter (title or authors) catches most.