MoritzLaurer/mDeBERTa-v3-base-mnli-xnli fine-tuned as a cross-encoder entailment scorer for hierarchical Arabic genre classification, deliberately trained on the AraGenre dev_gold.json labels.

Authors: Hassan Barmandah (NAMAA Community; Umm Al-Qura University), Israa Elhosiny (NAMAA Community), Yousra El-Ghawi (NAMAA Community), Omer Nacar (NAMAA Community)

This model's 0.9917 hierarchical F1 is a 5-fold cross-validation estimate on the AraGenre dev set, not a real-world generalization estimate — the model is trained directly on dev labels by design (see Approach below for why), unlike the rest of the project's pipeline, which trains only on the 7 AraGenre TRAIN genres. Per the project's system-description paper, this model belongs to the broader fine-tuned-encoder lineage that collapsed to 0.22–0.44 hierarchical F1 on the actual 27,972-item hidden test set (74 specific genres under 6 broad genres) — training on dev labels does not close that gap, because the test set's official definitions and much larger taxonomy diverge from what dev training exposes the model to.

The system that actually won for this team — 0.7013 hierarchical F1, 3rd of 18 teams on the official CodaBench leaderboard — was a separate, zero-shot DeepSeek-LLM pipeline with no dev-label training at all (stage2_llm_zeroshot_pipeline/ in the project repo). This model is not that system. It is released here for transparency and reproducibility of the project's full experimental record, not as a recommended production classifier.

Cross-encoder entailment fine-tuning: MoritzLaurer/mDeBERTa-v3-base-mnli-xnli fine-tuned on (Arabic text, augmented English genre definition) pairs with a binary entailment label. Deliberately trains on dev_gold.json (unlike the rest of the pipeline, which trains only on the 7 TRAIN genres) — since the dev genres differ from the train genres, the 110 extra labeled dev examples materially help, and dev genres are not the true held-out test set for this task. The 5-fold cross-validation protocol exists specifically to get an honest, unbiased performance estimate despite training on dev.

dev_gold.json (110 examples) plus augmented definitions and synthetic hard-negative examples.

Load with AutoModelForSequenceClassification.from_pretrained(...), score each (text, definition) pair, and take the definition with the highest P(entailment).

If you use this work, please cite our system-description paper:

Please also cite the AraGenre 2026 shared task overview paper:

Apache 2.0