This model's development-set score (0.9002 hierarchical F1) is not representative of real-world performance. Per the project's system-description paper, this entire lineage of fine-tuned/ensembled sentence encoders — which scored well on the 110-item, 6-genre AraGenre dev set — collapsed to 0.22–0.44 hierarchical F1 on the actual 27,972-item hidden test set (74 specific genres under 6 broad genres). The cause: these models were calibrated to a hand-augmented definition style that does not match the test set's official genre definitions or its much larger taxonomy.
The system that actually won for this team — 0.7013 hierarchical F1, 3rd of 18 teams on the official CodaBench leaderboard — was a separate, zero-shot DeepSeek-LLM pipeline with no fine-tuning at all (stage2_llm_zeroshot_pipeline/ in the project repo). This model is not that system. It is released here for transparency and reproducibility of the project's full experimental record, not as a recommended production classifier.
Approach
Runs a two-phase X-GENRE → AraGenre MNRL fine-tuning recipe three times with different random seeds (42, 123, 777), then averages the three runs' softmax-normalized dev scores as a multi-seed ensemble of a single architecture. This repo publishes only the seed-42 checkpoint (dev hier_f1 = 0.9002). Seeds 123 (0.7608) and 777 (0.8888) exist as separate checkpoints but remain private and are out of scope for this release; the 3-seed average scored 0.9179.
Base model
intfloat/multilingual-e5-large
Training data
Phase 1: X-GENRE corpus. Phase 2: AraGenre TRAIN genres (7). No dev labels were used in training.
Hyperparameters
Seed: 42
Phase 1 epochs: 12
Phase 2 epochs: 12
Batch size: 16
Learning rate: 5e-5
Usage
python e5_large_multiseed_ensemble.py
See the project repository for the full script and data-loading requirements.
Citation
If you use this work, please cite our system-description paper:
bibtex
1@inproceedings{barmandah-etal-2026-namaa,
2 title = {NAMAA at AraGenre 2026: From Encoder Baselines to Self-Consistent LLM Ensembling for Hierarchical Arabic Genre Classification},
3 author = {Barmandah, Hassan and Elhosiny, Israa and El-Ghawi, Yousra and Nacar, Omer},
4 booktitle = {Proceedings of the 4th Arabic Natural Language Processing Conference (ArabicNLP 2026)},
5 address = {Budapest, Hungary},
6 publisher = {Association for Computational Linguistics},
7 year = {2026},
8}
Please also cite the AraGenre 2026 shared task overview paper:
bibtex
1@inproceedings{elhaj-etal-2026-aragenre,
2 title = {AraGenre 2026: A Hierarchical Definition-Guided Arabic Genre Classification Shared Task},
3 author = {El-Haj, Mo and Ezzini, Saad and Abudalfa, Shadi and Lamsiyah, Salima and Jarrar, Mustafa},
4 booktitle = {Proceedings of the 4th Arabic Natural Language Processing Conference (ArabicNLP 2026)},
5 address = {Budapest, Hungary},
6 publisher = {Association for Computational Linguistics},
7 year = {2026},
8}
