You can deploy this model and start earning money today!

Deehan1866/
finetuned-deberta-wic-actual-version2

Name: finetuned-deberta-wic-actual-version2
Availability: InStock

safetensors

deberta-v2

word-sense-disambiguation

wic

deberta

cross-encoder

Deehan1866/WiC_actual

Views

No views yet

DeBERTa-v3-large Fine-tuned on WiC

Cross-encoder model for the Word-in-Context (WiC) binary sense disambiguation task. Both sentences are fed together so the model can attend across them simultaneously.

Target Word Marking

The target word is wrapped with <tgt>word</tgt> using the exact token position from the dataset (start1/start2 columns), so marking is always precise regardless of lemma or morphological variation.

Training Details

Hyperparameter	Value
Learning rate	8e-6
Effective batch	32
Epochs	10
Warmup ratio	0.15
Dropout	0.2
Label smoothing	0.05
LLRD decay	0.9

Performance

Split	Accuracy
Validation	0.7179
Test	0.7171

Usage

python

from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

tokenizer = AutoTokenizer.from_pretrained("Deehan1866/finetuned-deberta-wic-actual-version2")
model = AutoModelForSequenceClassification.from_pretrained("Deehan1866/finetuned-deberta-wic-actual-version2")

s1 = "The <tgt>bank</tgt> raised its interest rates."
s2 = "She visited her local <tgt>bank</tgt> to deposit a cheque."

enc = tokenizer(s1, s2, return_tensors="pt", truncation=True, max_length=256)
with torch.no_grad():
    logits = model(**enc).logits
pred = torch.argmax(logits).item()
print("Same sense" if pred == 1 else "Different sense")