The opaque nature of deep learning models presents significant challenges for the ethical deployment of hate speech detection systems. To address this limitation, we introduce Supervised Rational Attention (SRA), a framework that explicitly aligns model attention with human rationales, improving both interpretability and fairness in hate speech classification. SRA integrates a supervised attention mechanism into transformer-based classifiers, optimizing a joint objective that combines standard classification loss with an alignment loss term that minimizes the discrepancy between attention weights and human-annotated rationales. Empirically, SRA achieves 2.4× better explainability compared to current baselines, and produces token-level explanations that are more faithful and human-aligned.

📄 Read the paper on arXiv

📄 Paper: "Aligning Attention with Human Rationales for Self-Explaining Hate Speech Detection"
🎯 Accepted: AAAI-26 AI Alignment Track
🔗 Demo: Live Demo on HuggingFace Spaces

Model Description

This model is a bert-base-uncased classifier fine-tuned on the HateXplain dataset with Supervised Rational Attention (SRA) — a training method that aligns the model's attention weights with human-annotated rationales.

Key Innovation

Standard BERT attention weights don't reliably explain predictions. SRA supervises a specific attention head (Layer 8, Head 7) to attend to the same tokens that human annotators identified as evidence for their labeling decisions.

Result: Attention weights that actually explain the model's decisions, achieving 2.4× better alignment with human rationales while maintaining classification performance.

Labels

Label	ID	Description
Normal	0	Non-hateful, non-offensive content
Offensive	1	Offensive but not hate speech
Hate Speech	2	Hate speech targeting protected groups

Usage

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model_name = "bragee/sra-hate-speech-bert"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

text = "I love spending time with my friends"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=1)
    pred = torch.argmax(probs, dim=1).item()

labels = ["Normal", "Offensive", "Hate Speech"]
print(f"Prediction: {labels[pred]} ({probs[0][pred]:.1%})")

Extracting Attention-Based Explanations

The key feature of this model is interpretable attention. Extract attention from the supervised head:

python

with torch.no_grad():
    outputs = model(**inputs, output_attentions=True)

# Extract attention from Layer 8, Head 7 (the supervised head)
attention = outputs.attentions[8][:, 7, :, :]  # (batch, seq, seq)
attention_weights = attention.mean(dim=1)  # Average over query positions

tokens = tokenizer.convert_ids_to_tokens(inputs["input_ids"][0])
for token, weight in zip(tokens, attention_weights[0]):
    if token not in ["[CLS]", "[SEP]", "[PAD]"]:
        print(f"{token}: {weight:.3f}")

Training Details

Training Data

Dataset: HateXplain
Size: ~20,000 posts from Twitter and Gab
Annotations: 3-class labels + token-level rationales from multiple annotators

Training Procedure

Base model: bert-base-uncased
Epochs: 5
Batch size: 16
Learning rate: 2e-5
Max sequence length: 128

SRA Configuration

Supervised attention head: Layer 8, Head 7
Attention loss weight (α): 10.0
Loss function: Cross-entropy + α × MSE(attention, rationale)

The MSE loss is only computed for offensive/hate speech examples where human rationales exist.

Evaluation

Evaluated on the HateXplain test set:

Metric	Score
Macro F1	0.68
Accuracy	0.70
Attention-Rationale Alignment	2.4× baseline

Intended Use

Primary use: Research on explainable AI and hate speech detection
Demo/educational: Understanding how attention-based explanations work
Content moderation research: Studying interpretable classifiers

Limitations

Trained on English Twitter/Gab data — may not generalize to other platforms or languages
Attention explanations are post-hoc interpretations, not guaranteed causal explanations
Should not be used as sole arbiter for content moderation decisions

Ethical Considerations

This model is intended for research purposes. Hate speech detection systems can have false positives that disproportionately affect marginalized groups. Human review should always be part of any content moderation pipeline.

Citation

bibtex

@article{eilertsen2025sra,
  title={Aligning Attention with Human Rationales for Self-Explaining Hate Speech Detection},
  author={Eilertsen, Brage and Bjørgfinsdóttir, Røskva and Vargas, Francielle and Ramezani-Kebrya, Ali},
  journal={arXiv preprint arXiv:2511.07065},
  year={2025},
  note={Accepted at AAAI-26}
}

Model Card Contact

For questions about this model, please open an issue on the model repository.