You can deploy this model and start earning money today!

Void10/
distilbert-toxic-en

Name: distilbert-toxic-en
Availability: InStock

SetFit/toxic_conversations

mit

Views

No views yet

Content Moderation Model

Fine-tuned DistilBERT for real-time toxic content detection. Built as a research prototype exploring automated content moderation with a planned extension to Hindi and Hinglish language moderation.

Model Details

Property	Value
Base Model	distilbert-base-uncased
Task	Binary Text Classification
Dataset	SetFit/toxic_conversations
Training Samples	20,000
Epochs	3
Batch Size	32
Max Sequence Length	128
Mixed Precision	fp16

Performance

Metric	Score
Accuracy	95.19%
Precision	74.88%
Recall	60.16%
F1 Score	66.72%
Avg Inference Latency	237ms (CPU)

Labels

Label	Meaning
LABEL_0	Clean — no policy violation detected
LABEL_1	Toxic — abusive, threatening, or harmful content

Usage

python

from transformers import pipeline

classifier = pipeline(
    "text-classification",
    model="Void10/distilbert-toxic-en",
    return_all_scores=True
)

result = classifier("You are such a wonderful person!")
# [{'label': 'LABEL_0', 'score': 0.97}, {'label': 'LABEL_1', 'score': 0.03}]

Training Details

Fine-tuned on a 20K subsample of the SetFit toxic conversations dataset using HuggingFace Trainer API on a T4 GPU via Google Colab. Labels were binarized from multi-label annotations into clean (0) vs toxic (1).

Training Code

python

from transformers import (DistilBertTokenizerFast,
                          DistilBertForSequenceClassification,
                          TrainingArguments, Trainer)

args = TrainingArguments(
    output_dir="./results",
    num_train_epochs=3,
    per_device_train_batch_size=32,
    per_device_eval_batch_size=64,
    warmup_steps=200,
    evaluation_strategy="epoch",
    fp16=True,
)

Limitations

Class Imbalance: ~10% of training samples are toxic, causing conservative flagging and lower recall (60%). Addressable via weighted loss or oversampling.
English Only: This model was trained exclusively on English text. Performance degrades significantly on Hindi, Hinglish, or other Indic languages.
Domain: Trained on social media style text. May not generalize well to formal documents or domain-specific content.
Bias: Like all models trained on human-annotated data, this model may reflect annotator biases around identity groups and cultural context.

Research Extension (Planned)

The primary limitation of existing content moderation research — including recent work like SLM-Mod (NAACL 2025) — is the exclusive focus on English data. This model serves as the English baseline for a planned multilingual extension targeting:

Hindi toxic comment detection
Hinglish (code-mixed Hindi-English) moderation
Base model: MuRIL (Google, 2021) or IndicBERT (AI4Bharat)
Target dataset: HASOC Hindi Abusive Comment Dataset

Live Demo

Try the model live at:
huggingface.co/spaces/Void10/content-moderation-demo

Citation

If you use this model in your research, please cite:

Author

B.Tech Computer Science (AI/ML) · Shivalik College of Engineering · Dehradun, India
Built as a college research minor project · 2026