LFM2.5-1.2B Italian
LiquidAI/LFM2.5-1.2B-Base model further continued-pretrained on Italian Wikipedia and then fine-tuned with Italian instructions (Alpaca format).
Training
- Continued Pretraining: ~1% of Italian Wikipedia (20231101.it dump), texts ≥ 600 characters
- Instruction Tuning: dataset
DanielSc4/alpaca-cleaned-italian
- Technique: LoRA + Unsloth (merged in full precision / fp16)
- Original base model: multilingual, with significant improvement in Italian
Supported Languages
Improved in Italian.
Maintains English and residual capability in the other languages of the base model.
This model follows the exact continued pretraining + instruction tuning recipe recently published by LiquidAI in their official cookbook:
https://github.com/Liquid4All/cookbook/blob/main/finetuning/notebooks/cpt_translation_with_unsloth.ipynbKorean was replaced with Italian by using: Italian Wikipedia (20231101.it dump) instead of Korean Wikipedia for continued pretraining and
DanielSc4/alpaca-cleaned-italian dataset instead of the Korean Alpaca for instruction tuning
All other steps (LoRA including embed_tokens & lm_head, separate embedding learning rate, Unsloth, fp16 merge) are identical to the original notebook.
