You can deploy this model and start earning money today!

annamos/
BERT-Wikt-base-noun

Name: BERT-Wikt-base-noun
Availability: InStock

Views

No views yet

Model Card for BERT-Wikt-base-noun

Model Description

This model is an English language model based on BERT-base, fine-tuned using noun examples from English Wiktionary via supervised contrastive learning. The fine-tuning improves token-level semantic representations, particularly for tasks like Word-in-Context (WiC) and Word Sense Disambiguation (WSD).

Although trained on nouns, the model shows enhanced representation quality across the lexicon.

Developed by: Anna Mosolova, Marie Candito, Carlos Ramisch
Funded by: ANR Selexini
Model type: BERT-based transformer (BERT-base)
Language: English
License: MIT
Finetuned from model: google-bert/bert-base-uncased

Model Sources

Uses

The model is intended for extracting token-level embeddings for English, with improved sense separation.

How to Get Started with the Model

from transformers import AutoTokenizer, AutoModel

tokenizer = AutoTokenizer.from_pretrained("google-bert/bert-base-uncased")
model = AutoModel.from_pretrained("annamos/BERT-Wikt-base-noun")
sentence = 'You should knock before you enter'
tokenized = tokenizer(sentence, return_tensors='pt')
embeddings = model(**tokenized)[0]