This model is a fine-tuned version of meta-llama/Meta-Llama-3-8B-Instruct. The idea behind this model is to train on a dataset derived from a smaller subset of the tagengo-gpt4, but with improved data quality. I tried to achieve higher data quality by prompting GPT-4o, the latest OpenAI's LLM with better multilingual capabilities. The training objective is primarily focused on the Russian language (80% of the training examples). After training for 1 epoch on 2 NVIDIA A100 the model shows promising results on the MT-Bench evaluation benchmark, surpassing GPT-3.5-turbo and being on par with Suzume in Russian language scores, even though the latter is trained on 8x bigger and more diverse dataset.

The easiest way to use this model on your own computer is to use the GGUF version of this model (ruslandev/llama-3-8b-gpt-4o-ru1.0-gguf) using a program such as llama.cpp. If you want to use this model directly with the Huggingface Transformers stack, I recommend using my framework gptchain.

git clone https://github.com/RuslanPeresy/gptchain.git
cd gptchain
pip install -r requirements-train.txt
python gptchain.py chat -m ruslandev/llama-3-8b-gpt-4o-ru1.0 \
	--chatml true \
	-q '[{"from": "human", "value": "Из чего состоит нейронная сеть?"}]'

I achieved the following scores on Ru/En MT-Bench:

The following hyperparameters were used during training: