4x better F1 Macro (0.55 vs 0.14 in v2) through synthetic data augmentation
15/18 techniques with good coverage (F1 ≥ 0.3), up from 6/18
Better detection of rare techniques: Repetition, Flag-Waving, Slogans, Black-and-White Fallacy

Performance

Metric	Score
F1 Micro	0.796
F1 Macro	0.551
Techniques with F1 ≥ 0.3	15/18

Per-Technique Performance

Technique	F1	Support
Name_Calling,Labeling	0.960	724
Appeal_to_Authority	0.897	16
Red_Herring	0.880	14
Obfuscation,Intentional_Vagueness,Confusion	0.800	6
Loaded_Language	0.796	480
Whataboutism,Straw_Men,Red_Herring	0.700	26
Slogans	0.667	4
Appeal_to_fear-prejudice	0.662	246
Reductio_ad_hitlerum	0.612	68
Doubt	0.564	23
Exaggeration,Minimisation	0.524	146
Flag-Waving	0.467	23
Repetition	0.444	7
Bandwagon	0.400	8
Black-and-White_Fallacy	0.300	17

Intended Use

Analyzing social media posts for propaganda techniques
Content moderation assistance
Media literacy tools
Research on information manipulation

Usage

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

# Load model
tokenizer = AutoTokenizer.from_pretrained("vinai/bertweet-base", use_fast=False)
model = AutoModelForSequenceClassification.from_pretrained("synapti/nci-social-classifier-v3")

# Technique labels
TECHNIQUES = [
    "Loaded_Language", "Appeal_to_fear-prejudice", "Exaggeration,Minimisation",
    "Repetition", "Flag-Waving", "Name_Calling,Labeling", "Reductio_ad_hitlerum",
    "Black-and-White_Fallacy", "Causal_Oversimplification", "Whataboutism,Straw_Men,Red_Herring",
    "Straw_Man", "Red_Herring", "Doubt", "Appeal_to_Authority", "Thought-terminating_Cliches",
    "Bandwagon", "Slogans", "Obfuscation,Intentional_Vagueness,Confusion",
]

# Predict
text = "Build the wall! Build the wall! Build the wall! #MAGA"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=128)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.sigmoid(outputs.logits)[0]

# Get detected techniques
detected = [(TECHNIQUES[i], float(probs[i])) for i in range(18) if probs[i] > 0.3]
print(detected)
# [('Slogans', 0.51), ('Repetition', 0.39), ...]

Training Data

Trained on synapti/nci-social-v3:

9,069 training samples
1,134 validation samples
1,134 test samples

Data sources:

LLM-labeled HateXplain samples
SemEval span_propaganda short samples
Synthetic examples for underrepresented techniques
Balanced negative samples

Training Procedure

Base model: vinai/bertweet-base
Loss: Focal Loss (γ=2.0) for class imbalance
Optimizer: AdamW
Learning rate: 2e-5
Batch size: 32
Epochs: 8
Hardware: NVIDIA L4 GPU

Limitations

Optimized for English social media content
May not generalize well to formal news articles (use nci-technique-classifier for those)
Some rare techniques (Straw_Man, Thought-terminating_Cliches) still have limited coverage

Citation

bibtex

@misc{nci-social-classifier-v3,
  author = {Synapti},
  title = {NCI Social Media Classifier v3},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/synapti/nci-social-classifier-v3}
}

License

Apache 2.0