from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model and tokenizer
tokenizer = AutoTokenizer.from_pretrained("archich/hate-speech-detector")
model = AutoModelForSequenceClassification.from_pretrained("archich/hate-speech-detector")

# Example text
text = "Your text here"

# Tokenize
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True, max_length=256)

# Predict
with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=1)
    prediction = torch.argmax(probs, dim=1).item()

labels = ["NOT_HATE_SPEECH", "HATE_SPEECH"]
print(f"Prediction: {labels[prediction]} ({probs[0][prediction].item():.2%} confidence)")

Training Data

Trained on HASOC 2019 (Hate Speech and Offensive Content Identification) dataset containing:

Hindi posts from social media
English posts from social media

Label Mapping

0: NOT_HATE_SPEECH - Normal, non-offensive content
1: HATE_SPEECH - Hateful or offensive content (HOF)

Limitations & Ethical Considerations

⚠️ Important Notice:

This model is intended to assist human moderators, not replace them
May contain biases from training data
Context and cultural nuances are important - manual review recommended
False positives are possible
Should not be the sole decision-maker for content removal

Performance

Training details and metrics available in model files.

Citation

If you use this model, please cite:

@misc{hate-speech-detector,
  author = {archich},
  title = {Multilingual Hate Speech Detector},
  year = {2024},
  publisher = {HuggingFace},
  howpublished = {\url{https://huggingface.co/archich/hate-speech-detector}}
}