You can deploy this model and start earning money today!

maksays-003/
bangla-fake-news-xlmr

Name: bangla-fake-news-xlmr
Availability: InStock

Views

No views yet

Bangla Fake News Detector — XLM-RoBERTa

Fine-tuned xlm-roberta-base for binary fake news classification in Bangla and English. Trained on BanFakeNews-2.0 with calibrated confidence scores and optimized decision threshold.

⚠️ Critical Usage Warning — Read Before Using

Minimum Input Length: 500+ characters

This model requires full article text to work reliably. Short texts (headlines, snippets, social media posts under 200 characters) will almost always return "Credible" regardless of content — this is expected behavior, not a bug.

Input length	Reliability
< 200 chars	❌ Unreliable — do not use
200–500 chars	⚠️ Low reliability
500–1,000 chars	✓ Acceptable
1,000+ chars	✓ Best results

Why? The model was trained on full news articles (average 1,796 chars, median 553 tokens). It learned patterns from article structure, writing style, source signals, and content coherence — none of which are present in short snippets.

Correct usage:

python

# ✓ CORRECT — full article text
text = """নির্বাচনের দিন যানবাহন চলাচল বন্ধ থাকার পরেও যেভাবে ভোট দিতে যাবেন।
আসন্ন একাদশ জাতীয় সংসদ নির্বাচন ঘিরে সব ধরনের যান চলাচলে বিধিনিষেধ জারি করেছে
সড়ক পরিবহন ও সেতু মন্ত্রণালয়..."""  # 500+ chars

# ✗ WRONG — headline only (will return Credible regardless)
text = "সরকার ঘোষণা করেছে সমস্ত বিদ্যালয় বন্ধ হবে"

Model Description

Property	Value
Base model	xlm-roberta-base
Task	Binary text classification (Fake / Credible)
Languages	Bangla (primary), English (secondary), Banglish
Parameters	278M
Training GPU	Kaggle T4 (~2.7 hours)
Calibration	Temperature scaling (T=0.5308)
Decision threshold	0.05 (on calibrated Fake probability)
W&B run	g9qn9beq

Test Set Performance

Evaluated on a stratified temporal test set of 7,784 samples (195 fake, 7,589 credible) — articles from 2024+, never seen during training.

Class	Precision	Recall	F1	Support
Fake	0.9066	0.8462	0.8753	195
Credible	0.9961	0.9978	0.9969	7,589
Macro avg	0.9513	0.9220	0.9361	7,784

Key metric: Macro F1 = 0.9361 — primary metric for imbalanced evaluation. Accuracy (0.9940) is reported for reference only — it is misleading at 39:1 imbalance.

Training Details

Dataset

Primary: BanFakeNews-2.0 — 60,000 Bangla news articles (2025)
After deduplication: 51,889 unique articles
Split: Stratified 70/15/15 (train/val/test)
Class imbalance: 39:1 authentic-to-fake ratio
Fake samples in train: 906 out of 36,322

Class Imbalance Handling

The severe 39:1 imbalance required a custom training strategy:

python

# Class weights applied to CrossEntropyLoss
CLASS_WEIGHTS = {
    0: 20.0386,  # Fake — upweighted 20x
    1:  0.5126,  # Credible — downweighted
}

# Custom Trainer subclass to apply weights
class WeightedTrainer(Trainer):
    def compute_loss(self, model, inputs, return_outputs=False, **kwargs):
        labels  = inputs.pop("labels")
        outputs = model(**inputs)
        weight  = torch.tensor([20.0386, 0.5126], device=outputs.logits.device)
        loss    = F.cross_entropy(outputs.logits, labels, weight=weight)
        return (loss, outputs) if return_outputs else loss

Hyperparameters

optimizer: AdamW learning_rate: 2e-5 weight_decay: 0.01 warmup_ratio: 0.10 epochs: 5 (early stopping, patience=2) batch_size: 16 (T4 GPU) max_length: 512 tokens fp16: True metric: Macro F1 (for best model selection)

Confidence Calibration

Raw softmax outputs from fine-tuned transformers are overconfident. Temperature scaling was applied on the validation set:

python

# Optimal temperature found via NLL minimization on validation set
T_optimal = 0.5308  # T < 1.0 = model was underconfident (due to class weights)

# Calibrated probability
calibrated_probs = softmax(logits / T_optimal)

# Optimal threshold found via Precision-Recall curve
# Target: maximize recall at precision >= 0.70
threshold_fake = 0.05

Usage

Direct inference

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
from scipy.special import softmax
import torch
import json

model_name = "maksays-003/bangla-fake-news-xlmr"
tokenizer  = AutoTokenizer.from_pretrained(model_name)
model      = AutoModelForSequenceClassification.from_pretrained(model_name)
model.eval()

# Calibration config is included in the model repo
import requests
config = requests.get(
    "https://huggingface.co/maksays-003/bangla-fake-news-xlmr/resolve/main/calibration_config.json"
).json()
T          = config["temperature"]       # 0.5308
threshold  = config["threshold_fake"]    # 0.05

def predict(text: str) -> dict:
    # ⚠️ Minimum 500 characters recommended
    if len(text) < 200:
        return {"warning": "Text too short for reliable prediction"}

    inputs = tokenizer(
        text,
        return_tensors = "pt",
        max_length     = 512,
        truncation     = True,
        padding        = True,
    )
    with torch.no_grad():
        logits = model(**inputs).logits[0].numpy()

    # Apply temperature scaling
    probs     = softmax(logits / T)
    fake_prob = float(probs[0])

    return {
        "label":       "Fake" if fake_prob >= threshold else "Credible",
        "fake_prob":   round(fake_prob, 4),
        "credible_prob": round(float(probs[1]), 4),
        "threshold":   threshold,
    }

# Example — use full article text
text = """নির্বাচনের দিন যানবাহন চলাচল বন্ধ থাকার পরেও যেভাবে ভোট দিতে যাবেন।
আসন্ন একাদশ জাতীয় সংসদ নির্বাচন ঘিরে সব ধরনের যান চলাচলে বিধিনিষেধ জারি করেছে
সড়ক পরিবহন ও সেতু মন্ত্রণালয়। ২৯ ডিসেম্বর মধ্যরাত থেকে ৩০ ডিসেম্বর মধ্যরাত
পর্যন্ত সড়কপথে সব ধরনের যান চলাচল বন্ধ থাকবে।"""

result = predict(text)
print(result)
# {'label': 'Fake', 'fake_prob': 0.9998, 'credible_prob': 0.0002, 'threshold': 0.05}

Via Live API (easiest)

bash

curl -X POST https://redis-production-b1ef.up.railway.app/v1/predict \
  -H "X-API-Key: YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{"text": "your full article text here (500+ chars recommended)"}'

Known Limitations

1. Short text unreliability ⚠️

The most important limitation. Articles under 500 characters will produce unreliable results — almost always "Credible" regardless of actual content. This is a fundamental constraint of the training data distribution (average article: 1,796 chars).

2. Binary classification only

This version classifies as Fake or Credible only. The Unverified/Satire class was not included due to insufficient labeled data. Satire articles may be incorrectly labeled as either Fake or Credible.

3. Temporal drift

Trained on data up to 2024. Misinformation patterns, writing styles, and portal credibility change over time. Model should be re-evaluated and potentially retrained annually.

4. False negatives (~15%)

At threshold=0.05, approximately 15% of fake articles are missed (classified as Credible). This is a known trade-off — lowering the threshold increases recall but reduces precision.

5. Banglish handling

Code-switching between Bangla and English (Banglish) is handled by XLM-R but with lower confidence than pure Bangla or pure English text.

6. Source bias

BanFakeNews-2.0 was collected from specific Bangladeshi news portals. Political and topical biases in the training data may cause systematic errors on certain news categories.

Ethical Considerations

This model labels news articles as "Fake" — a high-stakes classification with real-world consequences:

False positives can damage the reputation of legitimate journalists
False negatives allow misinformation to spread unchallenged
Do not use this as the sole basis for editorial or legal decisions
Always show confidence scores alongside verdicts, never just labels
Always provide a feedback mechanism for users to report errors

Required disclaimer when deploying:

"This is an AI-assisted tool for first-pass screening only. It is not a substitute for professional fact-checking. Results may be incorrect. Short texts (<500 chars) are unreliable."

License note

BanFakeNews-2.0 is licensed CC BY-NC-SA 4.0 (non-commercial). This model inherits that restriction — do not use for commercial purposes without building your own dataset.

Citation

bibtex

@software{bangla_fake_news_2026,
  author    = {Md. Masum Khan},
  title     = {Bangla Fake News Detector},
  year      = {2026},
  url       = {https://github.com/Masum-Khan003/bangla-fake-news-detector},
  note      = {Fine-tuned XLM-RoBERTa for Bangla fake news detection,
               Macro-F1: 0.9361, trained on BanFakeNews-2.0}
}

@dataset{BanFakeNews2,
  title  = {BanFakeNews-2.0},
  year   = {2025},
  note   = {60,000 labeled Bangla news articles for fake news detection}
}

maksays-003/bangla-fake-news-xlmr

Bangla Fake News Detector — XLM-RoBERTa

⚠️ Critical Usage Warning — Read Before Using

Minimum Input Length: 500+ characters

Model Description

Test Set Performance

Training Details

Dataset

Class Imbalance Handling

Hyperparameters

Confidence Calibration

Usage

Direct inference

Via Live API (easiest)

Known Limitations

1. Short text unreliability ⚠️

2. Binary classification only

3. Temporal drift

4. False negatives (~15%)

5. Banglish handling

6. Source bias

Ethical Considerations

License note

Citation

Links

maksays-003/
bangla-fake-news-xlmr