You can deploy this model and start earning money today!

jimnoneill/
abstract-archon

Name: abstract-archon
Availability: InStock

Views

No views yet

Abstract Archon

Binary classifier that answers: "Is this text a real research abstract?"

Uses Potion-base-32M (512-dim static embeddings) + LogisticRegression. Designed as a quality gate for large-scale scientific publication databases where abstract fields often contain non-abstract content (figure captions, supplementary material refs, author bylines, HTML artifacts, taxonomy stubs).

Performance

Metric	Value
ROC-AUC	0.970
Accuracy	92%
Recall (real abstracts, t=0.01)	99.75%
Garbage precision (t=0.01)	99.0%

Evaluated on a 20% held-out stratified test split. Separate validation on 500 random publications from a 198M-paper database confirmed the model correctly identifies real abstracts with very low false-negative rate.

PMID 39869795 sanity check: P(abstract) = 0.794 (PASS)

Usage

python

import numpy as np
from model2vec import StaticModel
from scipy.special import expit

# Load model
data = np.load("abstract_archon_head.npz", allow_pickle=True)
coef = data['coef']
intercept = data['intercept']
scaler_mean = data['scaler_mean']
scaler_scale = data['scaler_scale']
threshold = float(data['threshold'][0])  # 0.01

# Load embedding model
embed = StaticModel.from_pretrained("minishlab/potion-base-32M")

# Predict
text = "Your abstract text here..."[:500]
emb = embed.encode([text])
x_scaled = (emb - scaler_mean) / scaler_scale
logit = x_scaled @ coef.T + intercept
prob = expit(logit).flatten()[0]

is_abstract = prob >= threshold
print(f"P(abstract) = {prob:.4f}, is_abstract = {is_abstract}")

Architecture

Embeddings: minishlab/potion-base-32M (512-dim, static, deterministic, ~20s for 200K docs)
Preprocessing: StandardScaler on embeddings
Head: LogisticRegression (C=0.01, balanced class weights)
Input: First 500 characters of text
Threshold: 0.01 (calibrated for 99.5%+ recall on real abstracts)

NPZ Keys

Key	Shape	Description
`coef`	(1, 512)	LR coefficients
`intercept`	(1,)	LR intercept
`classes`	(2,)	Class labels [0, 1]
`labels`	(2,)	['garbage', 'abstract']
`scaler_mean`	(512,)	StandardScaler mean
`scaler_scale`	(512,)	StandardScaler scale
`embed_model`	str	'minishlab/potion-base-32M'
`version`	str	'v1'
`threshold`	(1,)	Calibrated decision threshold

Training Data

Trained on ~4,000 examples (2,000 real abstracts + ~2,000 curated non-abstract texts) from a 198M-publication database. See abstract-archon-data for the full training set.

Negative examples were manually curated to remove misclassifications. Categories include figure/table captions, supplementary material references, author bylines, journal metadata scrapes, HTML-heavy content, MOESM titles, and taxonomy stubs.

Part of PubVerse

This model is part of the PubVerse scientific literature analysis pipeline, where it serves as a quality gate before clustering and impact analysis.

jimnoneill/abstract-archon