This is based on SloBERTa (
https://huggingface.co/EMBEDDIA/sloberta) but fine-tuned for 5 epochs
on the text of all Slovenian-language documents available on the Slovenian Digital Library (
https://dlib.si)
from the period 1850-1919. This was about 8.2 GB of text. Note that it also contained a lot of OCR errors.
