This model is Stage 1 of the NCI (Narrative Control Index) two-stage propaganda detection pipeline. It performs binary classification to detect whether text contains ANY propaganda techniques.

Model Description

Model Type: Binary text classifier
Base Model: answerdotai/ModernBERT-base
Training Data: synapti/nci-binary-classification (24,517 train, 1,727 validation, 1,729 test)
Language: English
License: Apache 2.0

Performance

Metric	Value
Accuracy	99.4%
Precision	98.9%
Recall	100.0%
F1 Score	99.4%
False Positive Rate	1.47%
False Negative Rate	0.00%

Confusion Matrix (Test Set, n=1,729)

                  Predicted
                  No Prop | Has Prop
Actual No Prop:      736  |     11
Actual Has Prop:       0  |    982

Threshold Analysis

Threshold	Accuracy	Precision	Recall	F1
0.3	99.2%	98.6%	100%	99.3%
0.4	99.2%	98.7%	100%	99.3%
0.5	99.4%	98.9%	100%	99.4%
0.6	99.7%	99.4%	100%	99.7%
0.7	99.7%	99.5%	100%	99.7%

Recommended threshold: 0.5 (default) or 0.6 for reduced false positives

Training Details

Loss Function: Focal Loss (gamma=2.0, alpha=0.25) for class imbalance
Optimizer: AdamW with weight decay 0.01
Learning Rate: 2e-5 with warmup ratio 0.1
Batch Size: 16 (effective 32 with gradient accumulation)
Epochs: 5 with early stopping (patience=3)
Best Model Selection: Based on F1 score on validation set

Usage

With Transformers Pipeline

python

from transformers import pipeline

detector = pipeline(
    "text-classification",
    model="synapti/nci-binary-detector-v2"
)

result = detector("The radical left is DESTROYING our country!")
# [{"label": "has_propaganda", "score": 0.99}]

result = detector("The Federal Reserve announced a 0.25% rate increase.")
# [{"label": "no_propaganda", "score": 0.98}]

With AutoModel

python

from transformers import AutoModelForSequenceClassification, AutoTokenizer
import torch

model = AutoModelForSequenceClassification.from_pretrained("synapti/nci-binary-detector-v2")
tokenizer = AutoTokenizer.from_pretrained("synapti/nci-binary-detector-v2")

text = "Wake up, people! They are hiding the truth from you!"
inputs = tokenizer(text, return_tensors="pt", truncation=True, max_length=512)

with torch.no_grad():
    outputs = model(**inputs)
    probs = torch.softmax(outputs.logits, dim=1)
    propaganda_prob = probs[0, 1].item()

print(f"Propaganda probability: {propaganda_prob:.2%}")

Two-Stage Pipeline (Recommended)

For full propaganda analysis with technique identification:

python

from transformers import pipeline

# Stage 1: Binary detection
binary_detector = pipeline(
    "text-classification",
    model="synapti/nci-binary-detector-v2"
)

# Stage 2: Technique classification
technique_classifier = pipeline(
    "text-classification",
    model="synapti/nci-technique-classifier-v2",
    top_k=None
)

text = "Some text to analyze..."

# Run Stage 1
binary_result = binary_detector(text)[0]
if binary_result["label"] == "has_propaganda" and binary_result["score"] >= 0.5:
    # Run Stage 2 only if propaganda detected
    techniques = technique_classifier(text)[0]
    detected = [t for t in techniques if t["score"] >= 0.3]
    print(f"Detected techniques: {[t['label'] for t in detected]}")
else:
    print("No propaganda detected")

Labels

Label ID	Label Name	Description
0	no_propaganda	Text does not contain propaganda techniques
1	has_propaganda	Text contains one or more propaganda techniques

Intended Use

Primary Use Cases

Media literacy tools and browser extensions
Content moderation assistance
Research on information manipulation
Educational platforms for critical thinking

Out of Scope

Censorship or automated content removal
Political targeting or surveillance
Single-source truth determination

Limitations

Optimized for English text
May have reduced performance on very short texts (<10 words)
Trained primarily on political/news content; domain shift may affect performance
Should be used as one signal among many, not as sole arbiter

Related Models

Stage 2: synapti/nci-technique-classifier-v2 - Multi-label technique classification
Dataset: synapti/nci-binary-classification

Citation

If you use this model, please cite:

bibtex

@misc{nci-binary-detector-v2,
  author = {Synapti},
  title = {NCI Binary Propaganda Detector v2},
  year = {2024},
  publisher = {HuggingFace},
  url = {https://huggingface.co/synapti/nci-binary-detector-v2}
}