You can deploy this model and start earning money today!

hannah-khallaf/
e2r-strategy-xlmr-large-focal

Name: e2r-strategy-xlmr-large-focal
Availability: InStock

Views

No views yet

Multilingual Easy-to-Read Strategy Classifier: XLM-R Large

This model identifies the simplification strategies realised between a standard-language sentence and its Easy-to-Read rewrite.

It is a multilingual sentence-pair multi-label classifier covering Arabic, Catalan, English, French, Italian and Spanish.

Predicted strategies

The model predicts any combination of the following six strategies:

Synonymy
Modulation
Compression
Explanation
Syntactic Change
Omission

The label order is fixed and must not be changed.

Model architecture

The model uses:

FacebookAI/xlm-roberta-large as the multilingual encoder;
paired input consisting of the standard sentence and its Easy-to-Read rewrite;
first-token/CLS pooling;
dropout of 0.1;
one linear classification layer with six independent outputs;
sigmoid probabilities for multi-label prediction;
a global decision threshold of 0.46;
a maximum input length of 512 tokens.

The classification architecture is:

text

XLM-R Large
→ first-token/CLS representation
→ dropout
→ linear layer with six outputs
→ sigmoid probabilities
→ threshold at 0.46

The model was trained with focal loss using positive-label weighting.

Released checkpoint

This repository contains seed 13, selected using development Macro-F1.

Result	Value
Selected seed	13
Best epoch	5
Development Macro-F1	69.53
Three-seed test Macro-F1, mean ± SD	59.20 ± 0.43
Decision threshold	0.46

The development score refers to the released seed. The reported test result is the mean and standard deviation across seeds 13, 42 and 87.

Installation

pip install torch transformers

Basic usage

Because this repository includes the exact custom classification architecture, load it with trust_remote_code=True.

python

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer

model_id = "hannah-khallaf/e2r-strategy-xlmr-large-focal"

tokenizer = AutoTokenizer.from_pretrained(
    model_id,
    trust_remote_code=True,
)

model = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    trust_remote_code=True,
)

standard_sentence = (
    "The committee postponed the implementation of the measure."
)

easy_to_read_rewrite = (
    "The committee decided to use the measure later."
)

inputs = tokenizer(
    standard_sentence,
    easy_to_read_rewrite,
    return_tensors="pt",
    truncation=True,
    max_length=512,
)

model.eval()

with torch.inference_mode():
    logits = model(**inputs).logits
    probabilities = torch.sigmoid(logits)[0]

labels = [
    model.config.id2label[index]
    for index in range(model.config.num_labels)
]

thresholds = model.config.e2r_classifier["thresholds"]

scores = {
    label: float(probability)
    for label, probability in zip(labels, probabilities)
}

predicted_labels = [
    label
    for label in labels
    if scores[label] >= float(thresholds[label])
]

print("Predicted strategies:", predicted_labels)
print("Scores:", scores)

Example output:

python

{
    "predicted_labels": [
        "Explanation",
        "Syntactic Change"
    ],
    "scores": {
        "Synonymy": 0.2955,
        "Modulation": 0.3413,
        "Compression": 0.2076,
        "Explanation": 0.5029,
        "Syntactic Change": 0.4802,
        "Omission": 0.1881
    }
}

Reusable prediction function

python

from __future__ import annotations

import torch
from transformers import AutoModelForSequenceClassification, AutoTokenizer


MODEL_ID = "hannah-khallaf/e2r-strategy-xlmr-large-focal"


tokenizer = AutoTokenizer.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
)

model = AutoModelForSequenceClassification.from_pretrained(
    MODEL_ID,
    trust_remote_code=True,
)

model.eval()


def classify_rewrite(
    standard_sentence: str,
    easy_to_read_rewrite: str,
) -> dict:
    inputs = tokenizer(
        standard_sentence,
        easy_to_read_rewrite,
        return_tensors="pt",
        truncation=True,
        max_length=512,
    )

    with torch.inference_mode():
        logits = model(**inputs).logits
        probabilities = torch.sigmoid(logits)[0]

    labels = [
        model.config.id2label[index]
        for index in range(model.config.num_labels)
    ]

    thresholds = model.config.e2r_classifier["thresholds"]

    scores = {
        label: float(probability)
        for label, probability in zip(labels, probabilities)
    }

    predicted_labels = [
        label
        for label in labels
        if scores[label] >= float(thresholds[label])
    ]

    return {
        "predicted_labels": predicted_labels,
        "scores": scores,
        "thresholds": thresholds,
    }

Input requirements

The standard sentence must be supplied first and the Easy-to-Read rewrite second:

python

inputs = tokenizer(
    standard_sentence,
    easy_to_read_rewrite,
    return_tensors="pt",
    truncation=True,
    max_length=512,
)

Reversing the sentence order does not represent the task on which the model was trained.

Output format

The model returns six logits in the following exact order:

text

Synonymy
Modulation
Compression
Explanation
Syntactic Change
Omission

Apply torch.sigmoid() to convert the logits to probabilities. A strategy is predicted when its probability is at least 0.46.

The thresholds are also stored in:

model.config.e2r_classifier["thresholds"]

Intended use

The model is intended for:

multilingual Easy-to-Read research;
analysis of rewriting and simplification strategies;
automatic annotation support;
corpus exploration;
comparison of standard and simplified sentence pairs.

It may support annotation, but its output should not be treated as a substitute for expert review.

Out-of-scope use

The model should not be used as evidence that a text is accessible to a particular person or reader group.

It should not be used to make medical, legal, educational or administrative decisions about individuals.

Limitations

The model assigns labels at sentence level. It does not:

identify the exact words or spans responsible for a strategy;
evaluate whether the rewrite preserves the original meaning;
measure the overall quality of the rewrite;
certify compliance with Easy-to-Read guidelines;
establish accessibility for a particular reader.

Performance differences between languages may reflect differences in corpus, domain, simplification practice, annotation procedure and label harmonisation. They should not be interpreted as inherent differences between languages.

Training languages

Arabic
Catalan
English
French
Italian
Spanish

Reproducibility files

The repository includes:

the selected model weights in SafeTensors format;
the exact custom Transformers model class;
the fixed label order;
the selected thresholds;
the resolved training configuration;
the selected seed's development summary;
the selected seed's test summary.

The exported Transformers model was verified against the original PairClassifier. The original and exported models produced identical logits, probabilities and thresholded predictions in the conversion checks.

Citation

Please cite the accompanying paper when using this model. The complete bibliographic entry will be added after publication.