OldBERTur: Named Entity Recognition for Diplomatic Old Icelandic
This model performs Named Entity Recognition (NER) on diplomatic Old Icelandic texts sourced from medieval manuscripts, identifying Person and Location entities.
This model is fine-tuned for NER through token classification by using a domain-adapted version of IceBERT adapted through Masked Language Modelling (MLM) on a corpus of 15 Old Icelandic texts. The NER model is designed for diplomatic Old Icelandic texts as defined in the Menota diplomatic transcription level description.
This model uses the MR + I + MIM training configuration, combining:
Source
Description
Menota (M)
Diplomatic Old Icelandic texts from the Medieval Nordic Text Archive
IcePaHC (I)
Icelandic Parsed Historical Corpus (normalised Old Icelandic texts)
MIM-GOLD-NER (MIM)
Modern Icelandic NER data for data augmentation
The superscript R means that sentence-level class resampling (sCR) was applied to the diplomatic Menota data. This is done to increase the amount of diplomatic data, while the normalised I and MIM are used to augment the diplomatic target data.
Training set statistics:
Configuration
Person
Location
Total
MR + I + MIM
40,191
12,326
52,517
Entity breakdown by source:
Source
Person
Location
Total
Menota (M)
3,969
472
4,441
IcePaHC (I)
2,797
362
3,159
MR after resampling
20,485
2,622
23,107
MIM-GOLD-NER
15,587
9,002
24,589
Evaluation sets:
Dev: 1,122 entities (912 Person; 210 Location)
Test: 1,153 entities (922 Person; 231 Location)
The dev and test sets consist exclusively of diplomatic Old Icelandic texts in order to reflect our target domain.
Evaluation Results
Metric
Score
F1
0.79
Precision
0.77
Recall
0.81
Labels
The model uses BIO tagging with the following labels:
Entity types: Only Person and Location entities are supported. Other entity types (organisations, dates, etc.) are not recognised due to scarcity in the training data.
Time period: Primarily trained on texts from 1250-1400 CE. Performance may vary on texts from other periods.
Domain: Optimised for saga literature and historical texts. May perform differently on other text types.
Domain-Adaptation
To adapt the underlying model, IceBERT, which is trained on modern texts, we use task-adaptive pre-training (TAPT) to facilitate model familiarity with Old Icelandic diplomatic texts
prior to fine-tuning for NER. We use a corpus of 15 unannotated diplomatic Old Icelandic texts from Menota for the domain-adaptation.
Hyperparameters:
Base model: mideind/IceBERT
Epochs: 8
Learning rate: 3e-5
Batch size: 32
Max sequence length: 256 tokens
Warm-up ratio: 6%
MLM masking probability: 15%
NER Training Procedure
NER is framed as a token classification task, with a classification head added on top of the domain-adapted IceBERT.
Hyperparameters:
Epochs: 5
Learning rate: 2e-5
Batch size: 16
Max sequence length: 256 tokens
Warm-up ratio: 10%
Weight decay: 0.01
Class imbalance handling:
Weighted cross-entropy loss with class weights: 0.1 for non-entities (O), 30.0 for entity classes to address class imbalance in the training data.
Citation
If you use this model, please cite:
bibtex
1@inproceedings{henningsson2026,
2 title={{OldBERTur}: Named Entity Recognition for Medieval {I}celandic},
3 author={Henningsson, Pontus and Pettersson, Eva and Lenas, Erik},
4 booktitle={{Proceedings of the Fourth Workshop on Language Technologies for Historical and Ancient Languages (LT4HALA) @ LREC-2026}},
5 year={2026},
6 address = {Palma, Mallorca, Spain},
7 pages = {469–481. European Language Resources Association (ELRA)},
8 publisher = "11 May 2026. European Language Resources Association (ELRA)",
9 url = {http://lrec-conf.org/proceedings/lrec2026/workshops/lt4hala/2026.lt4hala-1.0.pdf}
10}
Resources
For more information on our code and data, see our GitHub repository. For more information about the work in general, see our paper.
We are grateful for the great work carried out by the projects below, and for making it possible for us to use their data in order to conduct our academic research and develop NER models for Medieval Icelandic. We thank developers, annotators, scholars, project managers, and anyone else who has contributed to these projects. We also express our sincerest gratitude to the students from Uppsala University who assisted in marking and annotating entities in the two Menota works Codex Wormianus (AM 242 fol) and Vǫluspá in Hauksbók (AM 544 4to).
