LLaMA-3 7B pretrained on English-Arabic bilingual data with anchored training, 134k steps.
Validation Results
Final checkpoint: step 133,600
Validation set
Cross-entropy loss (nats)
Perplexity
English (en)
2.2666
9.65
AR (ar)
2.8241
16.85
Evaluation Results
EEE-format evaluation results are stored under eval_results/eee/ in this repository.
Tasks: Global MMLU (EN/AR/RU), PIQA, ECLeKTic, Fictive Entity (2-rate mix).
