You can deploy this model and start earning money today!

The-CoLab/
llama3-7b-en-translated-ru-v2

Name: llama3-7b-en-translated-ru-v2
Availability: InStock

Views

No views yet

llama3-7b-en-translated-ru-v2

LLaMA-3 7B pretrained on English and translated Russian data, 134k steps (injection fix v2).

Validation Results

Final checkpoint: step 133,600

Validation set	Cross-entropy loss (nats)	Perplexity
English (en)	2.2451	9.44
RU (`ru`)	1.8733	6.51

Note: The training config had a copy-paste bug: the Russian validation dataloader (fineweb2-hq-ru) was keyed "ar" instead of "ru", causing the log to show ar validate. The validation data and loss value are correct Russian validation. Rerun with corrected injection config (Buckwalter transliteration fix + injection count fix).

Evaluation Results

EEE-format evaluation results are stored under eval_results/eee/ in this repository. Tasks: Global MMLU (EN/RU), PIQA, ECLeKTic, Fictive Entity (2-rate mix).

Citation

Part of the The-CoLab multilingual-transfer collection.

Training Curves

Per-step training and validation loss curves are stored in training_curves/:

File	Description
`train_loss.csv`	Training loss every 10 steps (`step,train_loss`)
`val_loss.csv`	Validation loss per language (`step,lang,val_loss`)

The-CoLab/llama3-7b-en-translated-ru-v2