You can deploy this model and start earning money today!

MartaAguilarMorcillo/
gemma-fakenews-classifier

Name: gemma-fakenews-classifier
Availability: InStock

safetensors

gemma3_text

Views

No views yet

gemma-fakenews-classifier

Model Details

Model Description

Binary fake news classifier fine-tuned from Google's Gemma 3 1B pretrained model (google/gemma-3-1b-pt). The model classifies news articles as REAL (0) or FAKE (1) based on their title and body text. It was developed as part of the BuloCheck project, a Chrome extension for automatic fake news detection.

Developed by: Marta Aguilar Morcillo
Model type: Text Classification (Binary)
Language(s): English
License: Apache 2.0 (inherited from Gemma 3 base model)
Finetuned from: google/gemma-3-1b-pt

Model Sources

Repository: https://github.com/MartaAguilarMorcillo/BuloCheck
Demo / API: https://huggingface.co/spaces/MartaAguilarMorcillo/fakenews-api

Uses

Direct Use

This model is intended to classify English-language news articles as real or fake based on their textual content (title + body). It is used as the inference backend of the BuloCheck Chrome extension via a Gradio Space API.

Out-of-Scope Use

Non-English news articles (the model was trained exclusively on English text; predictions for other languages may be significantly less accurate).
Very short texts (fewer than 10 words) or texts composed mainly of non-alphabetic characters.
Image-based, audio-based, or video-based misinformation detection.
Real-time or high-concurrency production environments without upgrading the serving infrastructure.

Bias, Risks, and Limitations

The model was trained on a combination of three English-language datasets (ISOT, FakeNewsNet, WELFake). Its performance may degrade on news topics, writing styles, or domains not well represented in those datasets.
The training datasets were built using fact-checking platforms (PolitiFact, GossipCop) and news agencies (Reuters). The model may reflect labeling biases present in those sources.
The model does not verify facts against external knowledge bases; it relies solely on linguistic patterns learned during fine-tuning. A high confidence score does not guarantee factual correctness.
Political or culturally specific content may be harder to classify correctly due to domain shift.

Recommendations

Users should treat model predictions as a probabilistic signal, not a definitive verdict. The confidence score should be taken into account alongside other sources of verification. The model should not be the sole basis for editorial or legal decisions.

How to Get Started with the Model

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

repo_id = "MartaAguilarMorcillo/gemma-fakenews-classifier"

tokenizer = AutoTokenizer.from_pretrained(repo_id)
tokenizer.pad_token = tokenizer.eos_token

model = AutoModelForSequenceClassification.from_pretrained(repo_id)
model.eval()

def predict(title: str, body: str, max_length: int = 256):
    text = title + " " + body
    inputs = tokenizer(
        text,
        truncation=True,
        max_length=max_length,
        return_tensors="pt"
    )
    with torch.no_grad():
        outputs = model(**inputs)
        probas = torch.softmax(outputs.logits, dim=-1)
        predicted = torch.argmax(probas, dim=-1).item()

    label = model.config.id2label[predicted]
    confidence = probas[0][predicted].item()

    return {
        "label": label,
        "confidence": round(confidence, 4),
        "probas": {
            "REAL": round(probas[0][0].item(), 4),
            "FAKE": round(probas[0][1].item(), 4)
        }
    }

result = predict(
    title="Scientists confirm the Earth is flat",
    body="A new NASA study reveals the Earth has been flat all along."
)
print(result)
# {'label': 'FAKE', 'confidence': 0.8367, 'probas': {'REAL': 0.1633, 'FAKE': 0.8367}}

Training Details

Training Data

The model was fine-tuned on a custom dataset combining three publicly available fake news datasets:

Dataset	Source	Instances used
ISOT Fake News Dataset	University of Victoria (Ahmed et al., 2017)	3,000 real + 3,000 fake
FakeNewsNet	Arizona State University (Shu et al., 2018)	Full dataset
WELFake	Verma et al., IEEE TCSS (2021)	3,000 real + 3,000 fake

Each instance consists of the concatenation of the article title and body text, labelled as 0 (REAL) or 1 (FAKE). The final dataset contains 9,422 instances (4,797 real, 4,625 fake), split as follows:

Train: 70% (≈ 6,595 instances)
Validation: 10% (≈ 942 instances)
Test: 20% (≈ 1,885 instances)

Split was performed with random_state=123 for reproducibility.

Training Procedure

Preprocessing

Title and body text concatenated with a single space as separator.
Tokenized with google/gemma-3-1b-pt tokenizer; pad_token set to eos_token.
Dynamic padding applied per batch via DataCollatorWithPadding.
Maximum sequence length: 256 tokens (longer sequences truncated).

Partial Fine-Tuning (Freeze / Unfreeze)

All backbone parameters were frozen except the last 8 transformer layers and the classification head (model.score: Linear(1152, 2)). This corresponds to 13.42% of total parameters being trainable (134,212,864 / 999,888,256).

Training Hyperparameters

Parameter	Value
Base model	`google/gemma-3-1b-pt`
Number of unfrozen layers	5 (+ classification head)
Optimizer	AdamW
Learning rate	2e-5
Weight decay	0.01
Batch size	4
Number of epochs	3
Scheduler	Cosine with warmup (10% warmup)
Total training steps	4,947
Warmup steps	494
Gradient clipping	max_norm = 1.0
Evaluation frequency	Every 50 steps
Evaluation batches (val)	100 batches per evaluation
Best model checkpoint	Saved at minimum validation loss
Random seed	123
Training precision	BF16

Speeds, Sizes, Times

Hardware: NVIDIA Tesla T4 (Google Colab free tier)
Training time: ~60 minutes
Model size: ~1B parameters (BF16, ~2 GB)
Best checkpoint: Step 1,950 (val_loss = 0.558)

Evaluation

Testing Data

Held-out test split of the combined dataset described above (≈ 1,885 instances, never seen during training or hyperparameter tuning).

Metrics

Accuracy (percentage of correctly classified instances).

Results

Results obtained after restoring the best model checkpoint (step 1,950, val_loss = 0.558):

Split	Accuracy
Train	88.40%
Validation	85.14%
Test	84.19%

The gap of less than 2 percentage points between validation and test accuracy indicates good generalisation to unseen data with no significant overfitting.

Environmental Impact

Hardware: NVIDIA Tesla T4 (Google Colab free tier)
Hours used: ~1 hour
Cloud Provider: Google (Colab)
Compute Region: Unknown (Google Colab does not disclose region)
Carbon emitted: Estimated using the ML Impact Calculator

Technical Specifications

Model Architecture

Backbone: Gemma 3 1B (google/gemma-3-1b-pt) — decoder-only transformer, hidden size 1152, 26 transformer layers.
Classification head: Linear(1152, 2) added by AutoModelForSequenceClassification.
Label mapping: 0 → REAL, 1 → FAKE.

Software

Python 3.11
PyTorch 2.x
Transformers (HuggingFace)
PEFT
scikit-learn
Google Colab (training environment)

Citation

Gemma Team & Google DeepMind (2024). Gemma: Open Models Based on Gemini Research and Technology. Available at: https://huggingface.co/google/gemma-3-1b-pt
Ahmed et al. (2017). Detection of Online Fake News Using N-Gram Analysis and Machine Learning Techniques. Dataset available at: https://www.kaggle.com/datasets/csmalarkodi/isot-fake-news-dataset
Shu et al. (2020). FakeNewsNet: A Data Repository with News Content, Social Context, and Spatiotemporal Information for Studying Fake News on Social Media. Dataset available at: https://www.kaggle.com/datasets/mdepak/fakenewsnet
Verma et al. (2021). WELFake: Word Embedding over Linguistic Features for Fake News Detection. Dataset available at: https://www.kaggle.com/datasets/studymart/welfake-dataset-for-fake-news

Model Card Authors

Marta Aguilar Morcillo

Model Card Contact

GitHub: https://github.com/MartaAguilarMorcillo/BuloCheck