Signal A+B on a 7M-pair subsample (3 epochs). Best ablation; the FULL model is this recipe scaled to the full pool.
A 149M-parameter ModernBERT-base scientific document embedder trained with citation-context sentences as the primary contrastive signal. Part of the SciEmbed release (paper under double-blind review; author info omitted).
1from sentence_transformers import SentenceTransformer
2
3model = SentenceTransformer("anon-nlp/sciembed-ctx")
4emb = model.encode(["citation-context supervision for scientific embeddings"],
5 normalize_embeddings=True)
See the repository README. Paper: SciEmbed: Citation-Context Supervision for Scientific Document Embeddings (under review).
