LLaMA-3 7B pretrained on English bilingual data with code-switching, 34k steps.
Validation Results
Final checkpoint: step 33,400
Metric
Value
Validation language
en2
Cross-entropy loss (nats)
2.3586
Perplexity
10.58
Evaluation Results
EEE-format evaluation results are stored under eval_results/eee/ in this repository.
Tasks: Global MMLU (EN/AR/RU), PIQA, ECLeKTic, Fictive Entity (2-rate mix).
