DeBERTa-v3 CEFR Vocabulary Classifier
A fine-tuned DeBERTa-v3-base model for predicting the CEFR proficiency level of English vocabulary items.
The model classifies words into six Common European Framework of Reference (CEFR) levels:
This model is intended for:
Vocabulary difficulty estimation
Language learning applications
CEFR-aware educational tools
Vocabulary profiling
Adaptive learning systems
Linguistic research
Model Details
Item Value Base Model microsoft/deberta-v3-base Task CEFR Classification Labels A1, A2, B1, B2, C1, C2 Architecture DeBERTa-v3 Framework Hugging Face Transformers Language English
Dataset
This model was trained using CEFR annotations derived from:
Dataset:
star092304/CEFR-Annotated-WordNet
Dataset card:
The dataset provides CEFR proficiency annotations for WordNet lexical entries and was created based on the following work:
Reference Paper
CEFR-Annotated WordNet: LLM-Based Proficiency-Guided Semantic Database for Language Learning
Authors:
Masato Kikuchi
Masatsugu Ono
Toshioki Soga
Tetsu Tanabe
Tadachika Ozono
Paper:
Citation
1 @article{kikuchi2025cefrannotatedwordnet,
2 title={CEFR-Annotated WordNet: LLM-Based Proficiency-Guided Semantic Database for Language Learning},
3 author={Kikuchi, Masato and Ono, Masatsugu and Soga, Toshioki and Tanabe, Tetsu and Ozono, Tadachika},
4 year={2025}
5 }
Training Data Construction
Training examples were generated by aligning:
SemCor sense annotations
WordNet lexical entries
CEFR labels from CEFR-Annotated WordNet
Additional preprocessing included:
Lemmatization
Sense matching
Multi-word expression removal
Proper noun filtering
Only single-word lexical items were retained for training.
Training Performance
Training Curves
Training Curves Test Confusion Matrix
Confusion Matrix Evaluation Preview
Classification Report Usage
1 from transformers import AutoTokenizer
2 from transformers import AutoModelForSequenceClassification
3 import torch
4
5 model_name = "star092304/cefr-level-deberta-v3-base"
6
7 tokenizer = AutoTokenizer . from_pretrained ( model_name )
8 model = AutoModelForSequenceClassification . from_pretrained ( model_name )
9
10 label_names = [ "A1" , "A2" , "B1" , "B2" , "C1" , "C2" ]
11
12 word = "investigation"
13
14 inputs = tokenizer (
15 word ,
16 return_tensors = "pt" ,
17 truncation = True
18 )
19
20 with torch . no_grad ( ) :
21 outputs = model ( ** inputs )
22
23 pred_id = outputs . logits . argmax ( dim = - 1 ) . item ( )
24
25 print ( label_names [ pred_id ] )
Example Predictions
Word Predicted CEFR book A1 happy A1 journey A2 improve B1 investigation B2 sophisticated C1 quintessential C2
Limitations
The model predicts vocabulary difficulty at the word level.
CEFR levels can vary depending on context and meaning.
Polysemous words may belong to multiple CEFR levels depending on usage.
Predictions should be interpreted as estimated proficiency levels rather than absolute ground truth.
Intended Use
This model is intended for:
Educational research
Language learning systems
Vocabulary recommendation engines
CEFR-aware NLP pipelines
The model is not intended for high-stakes educational assessment or certification decisions.
Acknowledgements
This work builds upon:
WordNet
SemCor
CEFR-Annotated WordNet
Hugging Face Transformers
Microsoft DeBERTa-v3
Special thanks to the authors of the CEFR-Annotated WordNet dataset and paper.
