This repo contains a qlora adapter for
Mistral-7B-Instruct-v0.1, fine-tuned on a private instructions dataset, exclusively in Polish.
The fine-tuning took 1 hour on a single RTX 4090 with the following hyperparameters:
During the training of the adapter, the instruction format was maintained. Each prompt should be surrounded by [INST] and [\INST] tokens to leverage instruction fine-tuning.
This adapter allows the model to speak Polish more accurately than vanilla
Mistral-7B-Instruct-v0.1.
