You can deploy this model and start earning money today!

MatteoFasulo/
mdeberta-v3-xstance

Name: mdeberta-v3-xstance
Availability: InStock

Views

No views yet

mdeberta-v3-xstance

A multilingual stance detection model fine-tuned from microsoft/mdeberta-v3-base on the ZurichNLP/x_stance dataset.

The model predicts whether a political comment expresses a FAVOR or AGAINST stance toward a given political question. It supports multilingual inference and demonstrates strong cross-lingual transfer across Swiss national languages.

Highlights

🌍 Multilingual stance detection (🇩🇪 German (75%), 🇫🇷 French (25%), and 🇮🇹 Italian (only a few to test zero-shot cross-lingual transfer))
⚡ Built on mdeberta-v3
🎯 Binary stance classification (FAVOR / AGAINST)
🔄 Cross-lingual transfer capabilities

Performance

Validation Split

Evaluation on the validation split:

Metric	Score
Loss	0.5643
Accuracy	79.15%
Macro F1	79.12%

X-Stance Test Set

Detailed evaluation on the X-Stance test set as provided in the original repository:

Evaluation setting	Language	Macro F1
New comments	German (DE)	81.775
New comments	French (FR)	82.569
New questions	German (DE)	77.933
New questions	French (FR)	79.935
New topics	German (DE)	76.458
New topics	French (FR)	79.691
New comments	Italian (IT)	79.161

The evaluation script used to obtain these results is:

bash

python evaluate.py \
  --gold data/test.jsonl \
  --pred predictions/mdeberta-v3_pred.jsonl

Note: mdeberta-v3_pred.jsonl is provided in this repository for reproducibility. The evaluation script is available in the original repository at https://github.com/ZurichNLP/xstance .

Quick Start

Installation

pip install transformers torch

Run inference

Using the `pipeline` API (Recommended)

python

from transformers import pipeline

classifier = pipeline(
    task="text-classification",
    model="MatteoFasulo/mdeberta-v3-xstance"
)

question = "Soll der Bundesrat ein Freihandelsabkommen mit den USA anstreben?"

comment = "Nicht unter einem Präsidenten, welcher die Rechte anderer mit Füssen tritt und Respektlos gegenüber ändern ist."

result = classifier(
    {
        "text": question,
        "text_pair": comment,
    }
)

print(result)

Example output:

[{'label': 'AGAINST', 'score': 0.9823}]

For sequence-pair classification tasks such as stance detection, the text-classification pipeline accepts a dictionary with "text" and "text_pair" keys.

Using `AutoModelForSequenceClassification`

python

import torch
from transformers import (
    AutoTokenizer,
    AutoModelForSequenceClassification,
)

model_name = "MatteoFasulo/mdeberta-v3-xstance"

tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)

question = "Soll der Bundesrat ein Freihandelsabkommen mit den USA anstreben?"

comment = "Nicht unter einem Präsidenten, welcher die Rechte anderer mit Füssen tritt und Respektlos gegenüber ändern ist."

inputs = tokenizer(
    question,
    comment,
    return_tensors="pt",
    truncation=True,
)

with torch.no_grad():
    outputs = model(**inputs)

probabilities = torch.softmax(outputs.logits, dim=-1)

prediction = probabilities.argmax(dim=-1).item()

id2label = model.config.id2label

print("Prediction:", id2label[prediction])
print("Confidence:", probabilities[0, prediction].item())

Example output:

text

Prediction: AGAINST
Confidence: 0.9823

Input Format

The model expects two text sequences:

Target political question
Candidate comment

Example:

Question:
Should Switzerland increase renewable energy subsidies?

Comment:
Investing in renewable energy will reduce emissions and improve energy independence.

The tokenizer automatically formats these as sentence pairs for mdeberta-v3.

Output Labels

The classifier predicts one of two classes.

Label	Description
FAVOR	The comment supports the target question.
AGAINST	The comment opposes the target question.

The model outputs logits for both classes.

Model Description

This model is a fine-tuned version of microsoft/mdeberta-v3-base trained for multilingual, multi-target stance detection.

Unlike sentiment analysis, stance detection predicts whether a text supports or opposes a specific target question.

Because the underlying encoder is multilingual, the model can transfer knowledge across languages and perform inference on languages that were only partially represented during training.

Intended Uses

The model is suitable for:

Political stance detection
Cross-lingual stance classification
Research on multilingual NLP
Opinion mining
Benchmarking stance detection methods

Out-of-Scope Uses

This model is not intended for:

Fact checking
Political affiliation prediction
Hate speech detection
Toxicity classification
General sentiment analysis
Automated political decision-making

Training Dataset

Training was performed using the ZurichNLP/x_stance dataset.

Dataset characteristics:

150+ political questions
67,000 candidate comments
Swiss political debates
Multilingual annotations

Languages:

German (majority)
French
Italian

Each sample consists of:

(Target Question, Candidate Comment)
→ FAVOR / AGAINST

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 2e-05
train_batch_size: 16
eval_batch_size: 32
seed: 42
optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 850
num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Macro F1
0.4466	1.0	2853	0.4920	0.7886	0.7885
0.3395	2.0	5706	0.4551	0.8077	0.8077
0.2522	3.0	8559	0.5083	0.8153	0.8153

Limitations

Although the model performs well on multilingual political stance detection, several limitations should be considered.

Trained primarily on Swiss political debates.
Binary labels only (no Neutral class).
Performance outside politics has not been evaluated.
Implicit or sarcastic opinions remain challenging.
Domain shift may reduce performance on social media or informal discussions.

Ethical Considerations

This model predicts stance, not factual correctness.

Predictions should not be interpreted as:

political affiliation
truthfulness
misinformation detection
ideological profiling

Human oversight is recommended for any downstream application.

Framework versions

Transformers 5.14.1
Pytorch 2.8.0a0+5228986c39.nv25.06
Datasets 5.0.0
Tokenizers 0.22.2

Citation

If you use this model, please cite the original X-Stance dataset.

bibtex

@inproceedings{vamvas2020xstance,
    author    = "Vamvas, Jannis and Sennrich, Rico",
    title     = "{X-Stance}: A Multilingual Multi-Target Dataset for Stance Detection",
    booktitle = "Proceedings of the 5th Swiss Text Analytics Conference (SwissText) \& 16th Conference on Natural Language Processing (KONVENS)",
    address   = "Zurich, Switzerland",
    year      = "2020",
    month     = "jun",
    url       = "http://ceur-ws.org/Vol-2624/paper9.pdf"
}

License

This model is released under the MIT License.

Please also respect the licenses of:

microsoft/mdeberta-v3-base
ZurichNLP/x_stance

Acknowledgements

This model builds upon:

Microsoft for mdeberta-v3-base
Zurich NLP Group for the X-Stance dataset
Hugging Face Transformers

MatteoFasulo/mdeberta-v3-xstance