A fine-tuned DistilBERT model that classifies text as scam or safe — built to run fully on-device (browser/edge), as the detection engine behind the ScamShield Chrome extension.

Model Description

This model detects scam, phishing, and fraudulent job/internship messages from plain text — SMS messages, emails, job offer letters, and similar written content. It was fine-tuned from distilbert-base-uncased for binary sequence classification (0 = safe, 1 = scam).

It's designed to be small and fast enough to run entirely client-side (in a browser via ONNX + Transformers.js, or on edge devices), so no user text needs to be sent to a server for scam detection.

Base model: distilbert-base-uncased
Task: Binary text classification (safe vs. scam)
Language: English
License: MIT

Intended Use

Detecting phishing/scam SMS and emails
Detecting fraudulent job/internship offers (fake recruiters, upfront-fee scams)
Powering privacy-preserving, on-device scam-detection tools (browser extensions, edge apps)

Not intended for: legal/compliance decisions, moderating content at scale without human review, or as a sole determinant of fraud — see Limitations below.

Training Data

Combined from three sources:

SMS Spam Collection Dataset — real SMS messages, ham/spam labeled
EMSCAD — Employment Scam Aegean Dataset — real vs. fraudulent job postings
Custom synthetic contrastive dataset (~255 examples, generated via LLM) — pairs of professionally-worded job offers that are identical in structure and language, differing only in whether an upfront payment is requested. This was added after testing revealed the model initially missed softly-worded scams that avoided obvious keywords (see "Model Iteration History" below).

Training Procedure

Fine-tuned for 2 epochs, learning rate 2e-5, batch size 16, weighted cross-entropy loss (to address class imbalance — scam examples are a minority class)
Trained on a single RTX 4050 (6GB VRAM) with fp16 mixed precision
Max sequence length: 512 tokens

Evaluation Results (held-out test set)

Metric	Score
Accuracy	98.5%
Precision	0.944
Recall	0.833
F1	0.885

Model Iteration History

This model went through 4 iterations, which is worth documenting honestly since it shapes how the model should be used:

v1: Baseline on the two public datasets. Confidently correct on obvious scams, but confidently wrong (near 0%) on professionally-worded scams avoiding obvious red-flag keywords.
v2: Added broad synthetic augmentation to fix this — overcorrected, started flagging legitimate offer letters as scams due to generic HR language ("verification," "documentation").
v3: Rebalanced with more safe examples — overcorrected the other way, lost recall on real scams.
v4 (this model): Used a contrastive dataset — scam/safe pairs with identical structure and language, differing only in whether payment was requested. This isolated the actual decisive signal and fixed both prior failure modes.

Limitations

Trained and evaluated primarily on job/internship offers and SMS-style messages — may not generalize well to other scam formats (e.g. crypto scams, romance scams) without further fine-tuning.
English only.
Can still be uncertain on genuinely ambiguous, real-world borderline messages — in the ScamShield extension, this model's output is combined with a rule-based red-flag layer as a safety net, rather than used alone. Using this model standalone without such a safety net is not recommended for high-stakes use.
Like any small classifier, it can be sensitive to phrasing changes near its decision boundary.

How to Use

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("rehan-ml/scamshield-scam-detector")
model = AutoModelForSequenceClassification.from_pretrained("rehan-ml/scamshield-scam-detector")

text = "Congratulations! You've been selected. Pay a refundable registration fee of $50 to confirm your position."
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    outputs = model(**inputs)

scam_probability = torch.softmax(outputs.logits, dim=1)[0][1].item()
print(f"Scam probability: {scam_probability:.4f}")

🔗 ScamShield Chrome Extension (GitHub) — the full project this model powers, including the browser extension, rule-based safety net, and training notebook.

Author

Built by Rehan Raza for OSDHack 2026.

rehan-ml/scamshield-scam-detector

ScamShield — Scam & Phishing Detection Model