This model is a fine-tuned version of davlan/afro-xlmr-base for a regression task on multilingual text data. The model outputs continuous values and has been optimized using the 🤗 Trainer API and Optuna for hyperparameter tuning.

Model Details

Model Description

This model builds upon the Afro-XLM-RoBERTa (afro-xlmr-base), a multilingual language model trained on multiple African languages. We fine-tuned it for a regression task, using a custom dataset where the goal is to predict a numerical score based on input text.

Developed by: Michel Roland
Model type: Transformer (XLM-RoBERTa variant)
Language(s): Multilingual (including many African languages)
License: Same as davlan/afro-xlmr-base
Fine-tuned from: davlan/afro-xlmr-base

Model Sources

Base model: https://huggingface.co/davlan/afro-xlmr-base
Fine-tuned repo: None

Uses

Direct Use

You can use this model to predict a numerical score from a single text input.

Example:

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained("MichelRoland/Haussa-Afro-xlmr-base")
tokenizer = AutoTokenizer.from_pretrained("MichelRoland/Haussa-Afro-xlmr-base")

inputs = tokenizer("Your input text here", return_tensors="pt")
outputs = model(**inputs)
prediction = outputs.logits.squeeze().item()

Downstream Use

Further fine-tuning on similar multilingual regression tasks
Plug into a pipeline for text-based scoring

Out-of-Scope Use

Not suitable for classification, text generation, or language translation tasks
Not intended for high-stakes decision-making without further validation

Bias, Risks, and Limitations

Inherits limitations from the Afro-XLMR base model
Performance may vary across different languages
Biases in the training data may propagate in predictions

Recommendations

Perform evaluation on your specific dataset
Consider additional fine-tuning for domain-specific tasks
Monitor fairness and bias in multilingual use-cases

How to Get Started with the Model

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification

tokenizer = AutoTokenizer.from_pretrained("MichelRoland/Haussa-Afro-xlmr-base")
model = AutoModelForSequenceClassification.from_pretrained("MichelRoland/Haussa-Afro-xlmr-base")

Training Details

Training Data

[Describe your dataset here or link to a dataset card.]

Training Procedure

Preprocessing: Tokenization with truncation and padding
Loss function: MSELoss (regression)
Trainer API: Hugging Face Trainer with compute_metrics returning RMSE and Spearman

Training Hyperparameters

Learning Rate: Log-scale search between 1e-5 and 5e-4
Epochs: 3–10
Batch Size: 8, 16, or 32
Weight Decay: 0.0–0.3
Trials: 10 (using Optuna)
Objective: Minimize eval_rmse

python

best_run = trainer.hyperparameter_search(
    direction="minimize",
    hp_space=hp_space,
    n_trials=10,
    compute_objective=lambda metrics: metrics["eval_rmse"],
    backend="optuna"
)

Evaluation

Testing Data

https://github.com/semantic-textual-relatedness/Semantic_Relatedness_SemEval2024#dataset

Metrics

RMSE (Root Mean Square Error)
Spearman’s Rank Correlation

Example result:

json

{
  "eval_spearman": 0.64
}

MichelRoland/Haussa-Afro-xlmr-base

Model Card for Afro-XLMR Fine-Tuned for Regression

Model Details

Model Description

Model Sources

Uses

Direct Use

Downstream Use

Out-of-Scope Use

Bias, Risks, and Limitations

Recommendations

How to Get Started with the Model

Training Details

Training Data

Training Procedure

Training Hyperparameters

Evaluation

Testing Data

Metrics

License

MichelRoland/
Haussa-Afro-xlmr-base