Model created for the paper "Preferences for Idiomatic Language are Acquired Slowly --- and Forgotten Quickly: A Case Study on Swedish", TACL 2026.
1@misc{kunz2026preferencesidiomaticlanguageacquired,
2 title={Preferences for Idiomatic Language are Acquired Slowly -- and Forgotten Quickly: A Case Study on Swedish},
3 author={Jenny Kunz},
4 year={2026},
5 eprint={2602.03484},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL},
8 url={https://arxiv.org/abs/2602.03484},
9}
This is a SmolLM2-135M model continually pre-trained on the Swedish portion of Fineweb-2.
This is a research model intended for studying pre-training dynamics and I do not recommend using it for any practical purposes.
It is trained on a web corpus, and no alignment whatsoever has been performed, which means that the model will likely reflect its training data's biases and produce lots of hallucinations.
