A fixed-weight combination of 8 component sentence-encoder models' dev scores for hierarchical Arabic genre classification. This is not a single fine-tuned checkpoint — the repo contains a weights JSON and a combination script, not trained weights of its own.

Authors: Hassan Barmandah (NAMAA Community; Umm Al-Qura University), Israa Elhosiny (NAMAA Community), Yousra El-Ghawi (NAMAA Community), Omer Nacar (NAMAA Community)

This ensemble's development-set score (0.9569 hierarchical F1) is not representative of real-world performance. Per the project's system-description paper, this entire lineage of fine-tuned/ensembled sentence encoders — which scored well on the 110-item, 6-genre AraGenre dev set — collapsed to 0.22–0.44 hierarchical F1 on the actual 27,972-item hidden test set (74 specific genres under 6 broad genres). Its component weights were fit via a grid search directly against dev labels, so even the 0.9569 number reflects fitting to dev, not just evaluation on it.

The system that actually won for this team — 0.7013 hierarchical F1, 3rd of 18 teams on the official CodaBench leaderboard — was a separate, zero-shot DeepSeek-LLM pipeline with no fine-tuning at all (stage2_llm_zeroshot_pipeline/ in the project repo). This artifact is not that system. It is released here for transparency and reproducibility of the project's full experimental record, not as a recommended production classifier.

Combines 8 component models' softmax-normalized similarity scores using fixed, dev-grid-searched weights:

bge-m3-zeroshot: 0.053              bge-m3-augmented-defs: 0.158
e5-large-cosine-8ep: 0.0             e5-large-cosine-10ep: 0.263
e5-large-mnrl-xgenre-lite: 0.211     e5-large-mnrl-augmented-defs: 0.105
e5-large-mnrl-xgenre-phase1: 0.158   e5-large-multiseed-ensemble: 0.053

Each component's raw similarity scores are softmax-normalized first (so cross-model score scales are comparable), then combined with the weights above.

None directly — this is a weight recipe over pre-scored component models. The component weights were selected via grid search against dev_gold.json labels.

Requires cached dev score files from running all 8 component scripts first (stage1_encoder_finetuning/scores/*_dev_scores.json), then:

See the project repository for the full script and component-model requirements.

If you use this work, please cite our system-description paper:

Please also cite the AraGenre 2026 shared task overview paper:

Apache 2.0