metaphor-cat-roberta-base-weights
This model is a fine-tuned version of
projecte-aina/roberta-base-ca-v2 for
metaphor detection in Catalan text using the dataset
metaphor-catalan.
It performs token classification, identifying which tokens belong to metaphorical expressions using a BIO tagging scheme.
Evaluation results on the validation set:
- Loss: 0.7232
- Precision: 0.7188
- Recall: 0.5476
- F1: 0.6216
- Accuracy: 0.9665
Model description
This model is a RoBERTa-based transformer for Catalan NLP fine-tuned for token-level metaphor detection.
The model predicts whether each token in a sentence belongs to a metaphorical expression using the following BIO labeling scheme:
- O – token is not part of a metaphor
- B-METAPHOR – beginning of a metaphorical expression
- I-METAPHOR – continuation of a metaphorical expression
The task is framed as token classification, where each token receives a label indicating whether it is part of a metaphor span.
Because metaphor tokens are significantly less frequent than literal tokens, class-weighted loss was applied during training to mitigate class imbalance.
This model can support research on figurative language detection in Catalan, computational linguistics experiments, and NLP pipelines requiring semantic analysis of figurative language.
Intended uses & limitations
Intended uses
This model can be used for:
- Detecting metaphorical expressions in Catalan text
- Linguistic analysis of figurative language
- Computational linguistics research
- Digital humanities and literary analysis
- Supporting annotation pipelines for metaphor datasets
- Preprocessing for downstream NLP tasks involving figurative language
Typical usage scenarios include:
- Annotating metaphorical language in Catalan corpora
- Supporting research on metaphor detection
- Assisting literary or stylistic analysis tools
Limitations
- The training dataset is relatively small, which limits generalization.
- The model may not perform well on unseen domains, such as social media or informal text.
- Performance may degrade on:
- highly creative language
- poetry
- domain-specific corpora
- Predictions are produced at the token level, meaning additional processing may be required to reconstruct full metaphor spans.
- Although class weighting helps mitigate imbalance, metaphor detection remains a challenging task, and some metaphors may still be missed.
The model should therefore be used as an assistive tool rather than a definitive annotation system.
Training and evaluation data
Training dataset:
metaphor-catalan
This dataset contains Catalan sentences annotated for metaphorical expressions using token-level BIO labels.
Each example contains:
tokens — tokenized sentencetags — BIO labels indicating metaphor spans
Label scheme used during training:
| Label | Description |
|---|
| O | Non-metaphorical token |
| B-METAPHOR | Beginning of metaphor |
| I-METAPHOR | Continuation of metaphor |
The dataset exhibits strong class imbalance, with metaphor tokens occurring much less frequently than literal tokens. To address this, class weights were applied during training.
Example label distribution used during training:
O: 6089 tokensB-METAPHOR: 325 tokensI-METAPHOR: extremely rare / absent in some splits
Training procedure
Preprocessing
- The dataset was tokenized using the tokenizer from
projecte-aina/roberta-base-ca-v2.
- Labels were aligned with subword tokens.
- When a word was split into multiple subword tokens:
- the first token retained the label
- subsequent tokens were ignored during loss computation
Training setup
Training was performed using the Hugging Face Transformers Trainer API.
Key elements:
- RoBERTa base model with a token classification head
- Class-weighted loss to mitigate class imbalance
- Evaluation using the seqeval metric library
- Evaluation and checkpointing performed at the end of each epoch
Training hyperparameters
- Learning rate:
2e-5
- Train batch size:
4
- Evaluation batch size:
4
- Gradient accumulation steps:
2
- Effective batch size:
8
- Epochs:
10
- Weight decay:
0.1
- LR scheduler:
linear
- Warmup steps:
50
- Logging steps:
20
- Optimizer:
AdamW
- Mixed precision:
AMP
Training results
| Training Loss | Epoch | Step | Validation Loss | Precision | Recall | F1 | Accuracy |
|---|
| — | 1 | 66 | 0.7492 | 0.2000 | 0.0476 | 0.0769 | 0.9426 |
| 0.8617 | 2 | 132 | 0.5231 | 0.2238 | 0.7619 | 0.3459 | 0.8553 |
| 0.8617 | 3 | 198 | 0.5649 | 0.3571 | 0.5952 | 0.4464 | 0.9258 |
| 0.4759 | 4 | 264 | 0.6743 | 0.6286 | 0.5238 | 0.5714 | 0.9605 |
| 0.3202 | 5 | 330 | 0.6171 | 0.6970 | 0.5476 | 0.6133 | 0.9653 |
| 0.3202 | 6 | 396 | 0.6861 | 0.6875 | 0.5238 | 0.5946 | 0.9641 |
| 0.2365 | 7 | 462 | 0.6396 | 0.6857 | 0.5714 | 0.6234 | 0.9653 |
| 0.1962 | 8 | 528 | 0.6864 | 0.7273 | 0.5714 | 0.6400 | 0.9677 |
| 0.1962 | 9 | 594 | 0.7467 | 0.6875 | 0.5238 | 0.5946 | 0.9641 |
| 0.1589 | 10 | 660 | 0.7232 | 0.7188 | 0.5476 | 0.6216 | 0.9665 |
Framework versions
- Transformers: 4.57.3
- PyTorch: 2.9.0
- Datasets: 4.0.0
- Tokenizers: 0.22.1
