Paper Pair	Base SPECTER2	Cadence Fine-tuned
Attention is All You Need ↔ BERT	0.833	0.871
Attention is All You Need ↔ Cancer Research Paper	0.852	-0.044
Similarity gap	-0.019 ❌	+0.914 ✅

The base model was confused — it rated a cancer paper as more similar to a transformer paper than BERT was. After fine-tuning, the model has crystal clear domain separation with a gap of +0.914.

Usage

python

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("rohan5manza/cadence-specter2")

embeddings = model.encode(
    ["Attention Is All You Need [SEP] The dominant sequence transduction models..."],
    normalize_embeddings=True
)

Important: Use [SEP] to separate title from abstract. Always set normalize_embeddings=True.

Training Details

Base model: allenai/specter2_base
Training pairs: 267,841 triplets (anchor, positive, negative)
Signals: bibliographic coupling + category co-occurrence
Epochs: 3
Final loss: 0.042
Hardware: NVIDIA RTX 4060 Ti 16GB
Training time: ~3.6 hours

About Cadence

Cadence is a personalized research discovery app. Try it at cadence.rohanmarar.com. Source code at github.com/Rohan5manza/cadence-backend.

Rohan5manza/cadence-specter2

cadence-specter2

Datasets used (metadata: paper, abstract, etc)

Performance

Usage

Training Details

About Cadence

Rohan5manza/
cadence-specter2