Fake News Detector — BERT (Fine-tuned)
Deprecation Notice (July 2026)
This model was found to rely on a shortcut during training: it learned to associate the word "Reuters" with the "real" class (present in 61.8% of real articles vs. 1.8% of fake ones in WELFake), rather than learning generalizable signals of fake news. This produced misleadingly high benchmark accuracy (99.53%) while failing badly on real-world out-of-distribution text — especially satire and hoaxes (53.8% OOD accuracy).
Use fake-news-bert-debiased instead, which fixes this issue and documents the full diagnosis, fix, and remaining known limitations.
This model is kept public as part of the project's documented history — see the debiased repo's README for the full before/after story.
Model Description
This model is a fine-tuned version of bert-base-uncased for binary fake news classification (real vs. fake), trained on the WELFake dataset. It is part of the Fake News Detector V2 project.
- Developed by: Lax (LakshmiNarayanan Sugumar)
- Model type: BERT-base, fine-tuned for sequence classification
- Language: English
- License: Apache 2.0 (inherited from base model)
- Finetuned from model:
bert-base-uncased
- Repository: github.com/LakshmiNarayanan-Sugumar/fake_news_detector_v2
- Demo: HuggingFace Space — fake-news-detector
Uses
Direct Use
Given a news article's title and body text, the model predicts whether the article is real or fake. Intended for educational and portfolio demonstration purposes.
Out-of-Scope Use
Not intended for production moderation or fact-checking decisions without human review. Performance on news domains, time periods, or writing styles not represented in WELFake (e.g. non-English text, social media posts, satire) is unverified.
Bias, Risks, and Limitations
The model is trained on WELFake, a dataset merged from several existing fake/real news sources. Exact-duplicate articles were removed before training and evaluation, but near-duplicate or reworded articles covering the same story were not separately filtered, so reported metrics may carry minor residual optimism. The model reflects whatever stylistic or topical patterns exist in WELFake and may not generalize to other domains, languages, or adversarially written content.
How to Get Started
1from transformers import AutoTokenizer, AutoModelForSequenceClassification
2import torch
3
4tokenizer = AutoTokenizer.from_pretrained("LakshmiNarayanan-sugumar/fake-news-bert")
5model = AutoModelForSequenceClassification.from_pretrained("LakshmiNarayanan-sugumar/fake-news-bert")
6
7text = "Your article title here. Your article body text here."
8inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)
9with torch.no_grad():
10 logits = model(**inputs).logits
11prediction = torch.argmax(logits, dim=-1).item()
12print("Fake" if prediction == 1 else "Real") # adjust label mapping to match your training
Training Details
Training Data
WELFake dataset (~72k articles before cleaning), a merge of four public fake/real news datasets. After removing rows with missing title/text and exact-duplicate articles, the cleaned dataset contained 62,200 rows, split 80/10/10 into train/validation/test (stratified by label).
Training Procedure
Title and body text were concatenated into a single input field, tokenized with the BERT tokenizer (max length 512, truncation enabled). Fine-tuned for 3 epochs with a per-device batch size of 16. The best checkpoint was selected based on validation loss (epoch 2 of 3), and the final reported metrics come from a single evaluation on a held-out test set never used during training or checkpoint selection.
Training Hyperparameters
- Epochs: 3
- Batch size: 16 (train and eval)
- Base model: bert-base-uncased
- Hardware: Kaggle T4 GPU(s)
Evaluation
Testing Data
10% held-out test split (6,220 articles), evaluated once after training completed.
Results
| Metric | Score |
|---|
| Accuracy | 99.53% |
| F1 | 99.47% |
| Precision | 99.64% |
| Recall | 99.31% |
These metrics reflect a corrected training pipeline: the original version had a feature-engineering bug (duplicated text concatenation) and lacked a proper 3-way split, which risked inflating reported accuracy. This version fixes both issues; see repository README for details.
Technical Specifications
Model Architecture and Objective
BertForSequenceClassification (2 labels), built on bert-base-uncased.
Compute Infrastructure
Trained on Kaggle Notebooks using a T4 GPU.
Model Card Contact
LakshmiNarayanan Sugumar —
GitHub
