LLaMA-3 7B pretrained on English and translated Arabic data, 34k steps.
Validation Results
Final checkpoint: step 33,400
Validation set
Cross-entropy loss (nats)
Perplexity
English (en)
2.3798
10.80
AR (ar)
2.3891
10.90
Evaluation Results
EEE-format evaluation results are stored under eval_results/eee/ in this repository.
Tasks: Global MMLU (EN/AR/RU), PIQA, ECLeKTic, Fictive Entity (2-rate mix).
