Previous version (claim only) achieved F1 0.655 but misfired on real-world claims. v2 feeds claim + retrieved evidence to the model, fixing the core limitation.

Results

Metric	v1 (claim only)	v2 (claim + evidence)
F1 macro	0.655	0.813
Accuracy	0.63	0.79

Real World Performance

Claim	v1	v2
Earth is 3rd planet from Sun	FALSE ❌	TRUE ✅
Eiffel Tower in Berlin	TRUE ❌	FALSE ✅
WW2 ended in 1945	FALSE ❌	TRUE ✅
Obama was 44th President	TRUE ✅	TRUE ✅

Architecture

Base model: xlm-roberta-base
Retriever: BAAI/bge-base-en-v1.5
Index: FAISS (56,946 FEVER evidence passages)
Input format: [claim] </s></s> [retrieved evidence]
Training: 1 fine-tuning epoch on claim+evidence pairs, lr=5e-6

Labels

ID	Label	Meaning
0	TRUE	Claim supported by evidence
1	FALSE	Claim refuted by evidence
2	UNCERTAIN	Insufficient evidence

Known Limitations

This model requires evidence at inference time. Use BGE retrieval (BAAI/bge-base-en-v1.5) to retrieve relevant passages before classification. Standalone claim input without evidence will underperform.

Part of Sift

This model is part of Sift — an open-source 5-agent fact-checking pipeline.

🤗 Live Demo

Ashg2099/xlm-roberta-factchecker

XLM-RoBERTa Fact Checker (v2 — Retrieval Augmented)

What's New in v2