Slovenian Fine-Tuning Attempt
This repository contains a fine-tuned version of Llama-3.2-3b that was used as a first attempt to teach the model the Slovenian language.
The goal of this project was to explore how well the base model can adapt to Slovenian through supervised fine-tuning and to evaluate the quality of generated responses in a low-resource language setting.
Motivation
Slovenian is a relatively low-resource language compared to major global languages, so this experiment was designed to test whether a general-purpose language model can be adapted to produce more natural and useful Slovenian text with limited fine-tuning data.
What this model is for
This model was trained as an early experiment for:
- Testing Slovenian language generation.
- Evaluating instruction-following behavior in Slovenian.
- Exploring domain adaptation for local-language use cases.
- Building a foundation for future, improved fine-tuning runs.
Important Notes
This is not a production-ready model.
It is an early experimental checkpoint and may still produce:
- grammatical errors,
- unnatural phrasing,
- inconsistent Slovenian vocabulary,
- occasional mixed-language output,
- hallucinations or incorrect responses.
The results should be treated as a baseline for further development rather than a final language model.
Future Work
Planned improvements include:
- expanding the Slovenian dataset,
- improving grammar and fluency,
- testing different fine-tuning strategies,
- comparing outputs against the base model,
- creating a more robust instruction-tuned version.
License
This repository follows the license of the base model and the terms of the training data used.
Please review the original model license before using this checkpoint in downstream applications.
Acknowledgements
Thanks to the open-source AI community and the creators of the base model used in this experiment.
