This model is a fine-tuned version of Qwen/Qwen3-4B trained to identify fine-grained forms of populism in political discourse. The model performs 4-way classification to detect:

(a) No populism - Sentences without populist rhetoric
(b) Anti-elitism - Negative invocations of "elites"
(c) People-centrism - Positive invocations of the "people"
(d) Both - Sentences combining anti-elitism and people-centrism

The model was fine-tuned using LoRA (Low-Rank Adaptation) for parameter-efficient training on political speeches from Donald Trump's 2016 presidential campaign.

Model Details

Model Type

Base Model: Qwen/Qwen3-4B (4B parameters)
Fine-tuning Method: LoRA (Low-Rank Adaptation)
Task: Multi-class Text Classification (4 categories)
Language: English
Domain: Political discourse

Training Details

LoRA Configuration:

Rank (r): 16
Alpha: 32
Dropout: 0.05
Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Bias: None
Task Type: CAUSAL_LM

Training Hyperparameters:

Epochs: 3
Batch Size: 8 (per device)
Gradient Accumulation Steps: 4
Learning Rate: 2e-4
Learning Rate Scheduler: Cosine with 0.1 warmup ratio
Optimizer: paged_adamw_32bit
Weight Decay: 0.001
Max Gradient Norm: 1.0
Max Sequence Length: 300 tokens
Quantization: 4-bit (NF4) with double quantization
Mixed Precision: FP16/BF16
Total Training Steps: 1,476

Training Loss Progression:

Step 1: 4.9333
Step 100: 0.4583
Step 500: 0.0098
Step 1000: 0.003
Step 1476: ~0.001

Training Data

Dataset: coastalcph/populism-trump-2016

Data Source: Sentence-level annotations from Donald Trump's 2016 presidential campaign speeches.

Label Distribution (Original):

(a) No populism: 92%
(b) Anti-elitism: 4%
(c) People-centrism: 2%
(d) Both: 2%

Data Preprocessing:

Sentences truncated to maximum 48 words
5x upsampling applied to minority classes (b, c, d) to address class imbalance
Tokenized with max_length=300
Shuffled with seed=42

Populism Definition: Following political science literature, populism is defined as an anti-elite discourse in the name of "the people", characterized by:

Anti-elitism: Negative references to elite groups
People-centrism: Positive references to ordinary citizens or "the people"

How to Use

Option 1: Load Merged Model (Recommended)

python

from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

# Load model and tokenizer
model_name = "armaniii/Qwen3-4B-populism"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    device_map="auto",
    torch_dtype="auto"
)

# Create pipeline
pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer
)

Inference Example

python

# System prompt
SYSTEM_PROMPT = """You are a helpful AI assistant with expertise in identifying populism in public discourse.

Populism can be defined as an anti-elite discourse in the name of the "people". In other words, populism emphasizes the idea of the common "people" and often positions this group in opposition to a perceived elite group.

There are two core elements in identifying populism: (i) anti-elitism, i.e., negative invocations of "elites", and (ii) people-centrism, i.e., positive invocations of the "people".
"""

# Instruction template
INSTRUCTION = """You must classify each sentence in one of the following categories:

(a) No populism.
(b) Anti-elitism, i.e., negative invocations of "elites".
(c) People-centrism, i.e., positive invocations of the "People".
(d) Both people-centrism and anti-elitism populism.
"""

# Example sentence
sentence = "We need to stand up against the corrupt establishment that has betrayed the American people."

# Build prompt
conversation = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": INSTRUCTION + f'\n\nWhich is the most relevant category for the sentence: "{sentence}"?'}
]

# Apply chat template
prompt = tokenizer.apply_chat_template(
    conversation=conversation,
    tokenize=False,
    add_generation_prompt=True
)

# Generate response
response = pipe(
    prompt,
    do_sample=False,
    max_new_tokens=256,
    eos_token_id=tokenizer.eos_token_id,
    pad_token_id=tokenizer.eos_token_id,
)

print(response[0]['generated_text'])
# Expected output: "I would categorize this sentence as (d)..."

Expected Output Format

The model responds in the following format:

I would categorize this sentence as (X)

[Explanation of the classification decision]

Where X is one of: a, b, c, or d.

Performance

Detailed performance metrics and comparisons are available in the original paper (see Citation section below).

Key Findings:

Fine-tuned models significantly outperform zero-shot instruction-tuned LLMs
The model shows strong performance on in-domain political discourse
Cross-context evaluation demonstrates reasonable generalization to European political speeches
LoRA fine-tuning provides efficient adaptation with minimal trainable parameters

Limitations and Bias

Training Data Limitations:

Trained primarily on Donald Trump's 2016 campaign speeches
May not generalize equally well to other political contexts, time periods, or speakers
Performance may vary on non-US political discourse

Class Imbalance:

Original data is highly imbalanced (92% "No populism")
Upsampling (5x) applied to minority classes during training
Model may still show bias toward the majority class

Domain Specificity:

Optimized for sentence-level classification
Performance on longer texts may require sentence segmentation
Best suited for political discourse and campaign rhetoric

Ethical Considerations:

This model provides automated analysis and should not be the sole basis for political judgments
Results should be interpreted by domain experts
May reflect biases present in the training data

Intended Use

Primary Use Cases:

Research in political science and computational social science
Analysis of populist rhetoric in political campaigns
Educational purposes in understanding populist discourse
Automated annotation of political speech datasets

Out-of-Scope Use:

Making definitive political judgments about individuals or parties
Real-time moderation or censorship of political speech
Use without human oversight in sensitive political contexts

Citation

If you use this model, please cite the original paper:

bibtex

@misc{chalkidis2025populism,
    title={Identifying Fine-grained Forms of Populism in Political Discourse: A Case Study on Donald Trump's Presidential Campaigns},
    author={Chalkidis, Ilias and Brandl, Stephanie and Aslanidis, Paris},
    year={2025},
    archivePrefix={arXiv},
    primaryClass={cs.CL},
    url={https://arxiv.org/abs/2507.19303},
    doi={10.48550/arXiv.2507.19303}
}

Paper: Identifying Fine-grained Forms of Populism in Political Discourse

Dataset: coastalcph/populism-trump-2016

Acknowledgments

This model was developed based on the research by:

Ilias Chalkidis (University of Copenhagen)
Stephanie Brandl (University of Copenhagen)
Paris Aslanidis (Aristotle University of Thessaloniki)

The fine-tuning was performed using:

Base Model: Qwen/Qwen3-4B by Alibaba Cloud
Dataset: coastalcph/populism-trump-2016
PEFT Library: Hugging Face PEFT for LoRA implementation
Training Framework: Hugging Face Transformers and TRL

Model Card Authors

[Your Name/Organization]

Model Card Contact

[Your Contact Information]

License

This model is released under the Apache 2.0 License, consistent with the Qwen3-4B base model.

Responsible Use: This model is intended for research and educational purposes. Users should be aware of the limitations and biases described above and use the model responsibly with appropriate human oversight.

Related Resources:

📄 Paper
🤗 Dataset
💻 GitHub Repository
🎯 Demo Space