WARNING: This model is very biased towards educational content and web articles. It has only ~24M parameters and was trained on only ~550M tokens, which if 550,109,184 tokens ÷ 24,833,280 parameters, = 22.15 tokens per parameter.
We have released Alisio-25M, a 24M param SLM trained on 550M tokens of edu. web and HQ articles. It was trained on seven, or 7 FineWeb-Edu snapshots, ranging from December 2024 to June 2025.
Note: This is the first version, and the base model, so isn't very good at following instructions unless you use few-shot samples, which isn't also very good for a 24M param model.
Training data includes from December 2024 to June 2025, so knowledge cutoff = June 2025.
