You can deploy this model and start earning money today!

tommyleo2077/
deberta-v3-large-metaphor

Name: deberta-v3-large-metaphor
Availability: InStock

microsoft/deberta-v3-large

finetune

apache-2.0

Views

No views yet

DeBERTa-v3-large-metaphor

This model is a fine-tuned version of Microsoft DeBERTa-v3-large for binary indirect metaphor detection in English at token level, trained on the VU Amsterdam Metaphor Corpus (VUAMC). Training uses sentence-level segmentation and a two-class scheme (non-metaphor / metaphor), following the MIPVU annotation guidelines (Steen et al. 2010). Only indirect metaphors (metaphor_type == "met") are labeled positive.

Model description

Base model: microsoft/deberta-v3-large
Task: Token classification (binary metaphor detection)
Training unit: Full sentences, word-level labels
Label scheme: binary — 0 = non_metaphor, 1 = metaphor

Training & evaluation data

Dataset: VUAMC (VU Amsterdam Metaphor Corpus; Steen et al. 2010), covering four registers: News, Academic, Fiction, Conversation.
Split: Official NAACL FLP partition (Leong et al. 2018): 90 train documents / 27 test documents. A random 10% of training sentences is held out as a development set (seed=42).
Final splits: Train = 10,893 sentences · Dev = 1,210 sentences · Test = 4,080 sentences.

Test set genre distribution

Genre is determined by BNC-Baby document ID prefix, following the NAACL FLP 2018 shared-task mapping:

Genre	Docs	Test Fragments	Test Sentences
News	14	a1j, a1u, a31, a36, a3e, a3m, a5e, a7t, a7w, aa3, ahc, ahd, ahe, al2	579
Academic	4	b17, clw, cty, ecv	539
Fiction	3	bmw, ccw, faj	960
Conversation	6	kb7 (×3), kbd, kbh, kbw	2,002

Contamination verification

Test document filenames are explicitly excluded from the training file list at load time. No test document appears in the training or development split.

Training hyperparameters

Parameter	Value
Loss	Cross-Entropy
Epochs	4
Learning rate	2e-05
Per-device batch size	2
Gradient accumulation	8 (effective batch = 16)
Max sequence length	192
Weight decay	0.01
Warmup ratio	0.1
LR scheduler	Linear decay
Optimizer	AdamW
Best checkpoint	Selected by dev-set F1

Results

Evaluated on the 27-document NAACL FLP test split (4,080 sentences, 57,811 tokens, 6,697 gold metaphor tokens):

Metric	Value
Precision	84.00%
Recall	76.96%
F1	80.32%
Accuracy	95.63%

Results by genre

Genre	Docs	Tokens	Metaphors	Precision	Recall	F1	Accuracy
Academic	4	14,208	2,425	89.44	80.00	84.46	94.97
News	14	13,405	1,953	86.04	75.12	80.21	94.60
Fiction	3	12,764	1,079	78.87	81.28	80.05	96.58
Conversation	6	17,434	1,240	75.65	70.16	72.80	96.27

Results by part of speech (Universal POS)

POS tags with zero gold metaphors in the test set are omitted.

POS	Full name	Tokens	Metaphors	Precision	Recall	F1
DET	Determiner	4,163	213	94.06	96.71	95.37
ADP	Preposition	4,959	1,922	92.61	89.33	90.94
PRON	Pronoun	6,545	275	89.55	87.27	88.40
SCONJ	Subord. Conj.	1,299	99	79.25	84.85	81.95
VERB	Verb	6,237	1,908	82.55	74.90	78.54
ADV	Adverb	2,527	245	84.26	67.76	75.11
NOUN	Noun	8,630	1,383	80.62	66.16	72.68
ADJ	Adjective	3,511	595	80.68	63.87	71.29
PUNCT	Punctuation	7,627	2	50.00	50.00	50.00
AUX	Auxiliary verb	4,219	10	45.45	50.00	47.62
PART	Particle	1,881	8	37.50	37.50	37.50
PROPN	Proper noun	2,197	37	66.67	21.62	32.65

Label dictionary

json

{
  "0": "non_metaphor",
  "1": "metaphor"
}

Subwords are aligned to words via the tokenizer's word_ids; the first subword of each word is used for prediction. Special tokens ([CLS], [SEP], [PAD]) and continuation subwords receive IGNORE_INDEX = -100 and are excluded from all metric calculations.

Usage example

python

from transformers import AutoTokenizer, AutoModelForTokenClassification
import torch

model_path = "tommyleo2077/deberta-v3-large-metaphor"  # or local path
tokenizer = AutoTokenizer.from_pretrained(model_path)
model = AutoModelForTokenClassification.from_pretrained(model_path)
model.eval()

words = ["The", "conference", "threw", "a", "spanner", "in", "the", "works", "."]
inputs = tokenizer(
    words,
    is_split_into_words=True,
    return_tensors="pt",
    truncation=True,
    max_length=192,
)
word_ids = inputs.word_ids(batch_index=0)

with torch.no_grad():
    logits = model(**inputs).logits
preds = logits.argmax(dim=-1)[0].tolist()

word_preds = {}
for i, wid in enumerate(word_ids):
    if wid is not None and wid not in word_preds:
        word_preds[wid] = preds[i]

for i, w in enumerate(words):
    label = "metaphor" if word_preds.get(i, 0) == 1 else "non_metaphor"
    print(f"{w}\t{label}")

Citation

Model author: Tommy Leo — 1683619168tl@gmail.com

Dataset (MIPVU / VUAMC):

bibtex

@book{steen2010method,
  title     = {A Method for Linguistic Metaphor Identification: From {MIP} to {MIPVU}},
  author    = {Steen, Gerard and Dorst, Aletta G. and Herrmann, J. Berenike and Kaal, Anna and Krennmayr, Tina and Pasma, Thea},
  year      = {2010},
  publisher = {John Benjamins}
}

Train/test split:

bibtex

@inproceedings{leong2018vua,
  title     = {A Report on the 2018 {VUA} Metaphor Detection Shared Task},
  author    = {Leong, Chee Wee and Beigman Klebanov, Beata and Shutova, Ekaterina},
  booktitle = {Proceedings of the Workshop on Figurative Language Processing at NAACL-HLT 2018},
  year      = {2018}
}

Base model: Microsoft DeBERTa

This model:

bibtex

@misc{leo2025debertav3largemetaphor,
  title        = {deberta-v3-large-metaphor: Sentence-Level Fine-Tuning for Metaphor Detection},
  author       = {Leo, Tommy},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/tommyleo2077/deberta-v3-large-metaphor}},
  note         = {Contact: 1683619168tl@gmail.com}
}