This model was developed as part of the
Computational SLA working group at Språkbanken Text.
It takes essays written in Swedish by second language learners and assigns them one of the
CEFR levels.
Of note is that it only uses the first five levels of the scale (A1 to C1), ignoring level C2 due to both lack of training data and it measuring things differently than the other levels do.
Most of the information contained in this Model Card comes from
the paper that introduced the present model.
Feel free to check it out for more in-depth information.
This model is meant for building demos and tools that display an approximate CEFR level of language learner texts.
It is important to note that the predictions from this model should be taken as illustrative rather than as autorithative.
This model should not be deployed on high-stakes situations, such as actual language assessment or decision-making regarding migration, work, education, etc.
It has relatively low performance compared to what would be needed for such situations, not to speak of potential issues regarding accountability.
The model has been trained in a heterogeneous dataset of Swedish language learner essays.
While this exposed the model to a variety of contexts, it also means that there might be biases in terms of topics and format.
We are currently studying the impact that the essay authors' first language(s) has on these models.
This model card will be updated once we have more results on this regard.
@inproceedings{sanchez-etal-2024-jingle,
title = "Jingle {BERT}, Jingle {BERT}, Frozen All the Way: Freezing Layers to Identify {CEFR} Levels of Second Language Learners Using {BERT}",
author = "Mu{~n}oz S{'a}nchez, Ricardo and
Alfter, David and
Dobnik, Simon and
Szawerna, Maria Irena and
Volodina, Elena",
editor = {Gaillat, Thomas and
Mallart, Cyriel and
Moreau, Fabienne and
Li, Jen-Yu and
Drouet, Griselda and
Alfter, David and
Volodina, Elena and
J{"o}nsson, Arne},
booktitle = "Proceedings of the 13th Workshop on Natural Language Processing for Computer Assisted Language Learning",
month = oct,
year = "2024",
address = "Rennes, France",
publisher = "LiU Electronic Press",
url = "
https://aclanthology.org/2024.nlp4call-1.11/",
pages = "137--152"
}
For more information about the model or the present Model Card, you can reach out to:
