Nile-Chat is a family of open instruction-tuned models for Egyptian dialect, developed to handle both scripts commonly used in Egypt: Arabic script and Latin-based Arabizi. As part of the Jais project for standard Arabic and its extensions to dialectal Arabic, Nile-Chat is designed to support natural language generation in a way that reflects the script-diverse nature of Egyptian communication. These models are effective for a variety of tasks including question answering, translation and transliteration. Their range of sizes ensures accessibility, from lightweight personal deployments to more powerful setups, enabling broader use of AI technologies for Egyptian Arabic speakers. The family includes two versions:
Nile-Chat-4B: A compact 4B parameter model that balances efficiency and fluency, well-suited for generating Egyptian Arabic in both Arabic and Latin scripts.
Nile-Chat-2x4B-A6B: A 2x4B MoE parameter model with 6B Activated (soft-merging/1 Expert per script) that balances efficiency and fluency, well-suited for generating Egyptian Arabic in both Arabic and Latin scripts.
Nile-Chat-3x4B-A6B: A 3x4B MoE parameter model with 6B Activated (2 experts activated) providing high-capacity generation in Egyptian Arabic in both Arabic and Latin scripts and English.
Nile-Chat-12B: A dense 12B parameter model providing high-capacity generation capabilities, ideal for complex, multi-turn interactions and nuanced understanding in both Egyptian scripts.
Below we share some code snippets on how to get quickly started with running the model. First, install the Transformers library with:
pip install -U transformers sentencepiece
Then, copy the snippet from the section below.
Running with the pipeline API
python
1import torch
2from transformers import pipeline
34pipe = pipeline(5"text-generation",6 model="MBZUAI-Paris/Nile-Chat-12B",7 model_kwargs={"torch_dtype": torch.bfloat16},8 device="cuda"# replace with "mps" to run on a Mac device9)10
اسمي نايل-شات، على اسم نهر النيل، اطول نهر في العالم، اللي من زمان كان عامل مهم في تطور مصر، وبيساعد في معيشة الناس وأثر على التراث والثقافة بتاعتنا. وعشان انا موديل لغة، الباحثين بتوع جامعة محمد بن زايد للذكاء الاصطناعي دربوني باستخدام مجموعة من المصادر المفتوحة، وعشان كده سموني نايل-شات. لو عندك اي سؤال او محتاج مساعدة في حاجة، اتفضل اسأل!
Esmi Nile-Chat, 3ala esm nahr el Nile, atwal nahr fel 3alam w mahmo kter men el hadarat 3ala tool el taree5. Ana masroor gedan bek! A2dar asa3dak ezay el naharda?
Training Data
Nile-Chat models were trained on diverse datasets focusing on Egyptian dialect consisting of approximately 3.3B tokens during continual pre-training phase, 1.9M instructions during instruction finetuning and 0.2M samples for DPO, with a maximum length of 2048 tokens, including:
Web documents: A diverse collection of Egyptian web text ensures the model is exposed to a broad range of linguistic styles, topics, and vocabulary.
Instruction samples created from publicly available Egyptian Arabic datasets including translation and transliteration.
Translated English and multi-lingual pretraining and instruction-tuning datasets using Claude 3.5 Sonnet (v2).
The dataset covers both Egyptian Arabic and Latin scripts. Our instruction tuning dataset Egyptian-SFT-Mixture is publicly available.
Implementation Information
Nile-Chat models are based on Gemma 3 models. The Nile-Chat models were trained using 8 NVIDIA A100 80 GB GPUs in parallel using FSDP on AWS Sagemaker. The model is trained using HuggingFace transformers and parameter-efficient fine-tuning with LoRA rank of 256 for both continual pre-training and instruction finetuning, while performing full finetuning for DPO. The continual pre-training is divided into two phases: (i) general pre-training on 2.8B tokens from the Egyptian web and (ii) annealing phase with 0.5B high quality Egyptian text.
Evaluation
Nile-Chat models were evaluated on a comprehensive suite of tasks using various datasets and benchmarks to assess their performance across multiple dimensions. These included tasks such as:
EgyptianMMLU: An Egyptian version of ArabicMMLU and MMLU benchmarks.
EgyptianHellaSwag: An Egyptian version of HellaSwag (In both scripts Arabic and Latin).
Belebele Arz_Arab: Belebele is a multiple-choice machine reading comprehension dataset published by Facebook spanning 122 language variants. The Evaluation is done on the Arz_Arab part of Belebele that refers to Egyptian Arabic.
Translation: Including four directions and three languages: Arabic script Egyptian, MSA and English.
Transliteration: Transforming a sentence from Egyptian (written in Arabic script) to Arabizi (Written in Latin script) and vice-versa.
EgyptianPIQA: An Egyptian version of PIQA benchmark (In both scripts Arabic and Latin).
EgyptianWinoGrande: An Egyptian version of WinoGrande benchmark (In both scripts Arabic and Latin).
EgyptianRACE: An Egyptian version of RACE benchmark (In both scripts Arabic and Latin).
EgyptianOpenBookQA: An Egyptian version of OpenBookQA benchmark.
EgyptianAlpacaEval: An Egyptian adaptation of AlpacaEval to assess LLM instruction-following and cultural alignment.
The models were compared against a collection of existing open-source Arabic models to gauge their effectiveness, with a particular focus on performance in Egyptian. All scores are based on zero-shot performance. The prompts are written mainly in Egyptian. We used Language Model Evaluation Harness to conduct these evaluations. All evaluations are done with applying chat template except for EgyptianWinoGrande.
These models have certain limitations that users should be aware of.
Intended Usage
Open Large Language Models (LLMs) have a wide range of applications across
various industries and domains. The following list of potential uses is not
comprehensive. The purpose of this list is to provide contextual information
about the possible use-cases that the model creators considered as part of model
training and development.
Content Creation and Communication
Text Generation: These models can be used to generate creative text formats
such as poems, scripts, code, marketing copy, and email drafts.
Chatbots and Conversational AI: Power conversational interfaces for customer
service, virtual assistants, or interactive applications.
Text Summarization: Generate concise summaries of a text corpus, research
papers, or reports.
Research and Education
Natural Language Processing (NLP) Research: These models can serve as a
foundation for researchers to experiment with NLP techniques, develop
algorithms, and contribute to the advancement of the field.
Language Learning Tools: Support interactive language learning experiences,
aiding in grammar correction or providing writing practice.
Knowledge Exploration: Assist researchers in exploring large bodies of text
by generating summaries or answering questions about specific topics.
Limitations
Training Data
The quality and diversity of the training data significantly influence the
model's capabilities. Biases or gaps in the training data can lead to
limitations in the model's responses.
The scope of the training dataset determines the subject areas the model can
handle effectively.
Context and Task Complexity
LLMs perform better on tasks framed with clear prompts and
instructions. Open-ended or highly complex tasks might be challenging.
A model's performance can be influenced by the amount of context provided
(longer context generally leads to better outputs, up to a certain point).
Language Ambiguity and Nuance
Natural language is inherently complex. LLMs might struggle to grasp subtle
nuances, sarcasm, or figurative language.
Factual Accuracy
LLMs generate responses based on information they learned from their
training datasets, but they are not knowledge bases. They may generate
incorrect or outdated factual statements.
Common Sense
LLMs rely on statistical patterns in language. They might lack the ability
to apply common sense reasoning in certain situations.
Ethical Considerations and Risks
The development of large language models (LLMs) raises several ethical concerns.
In creating an open model, we have carefully considered the following:
Bias and Fairness
LLMs trained on large-scale, real-world text data can reflect socio-cultural
biases embedded in the training material.
Misinformation and Misuse
LLMs can be misused to generate text that is false, misleading, or harmful.
Guidelines are provided for responsible use with the model, see the
[Responsible Generative AI Toolkit][rai-toolkit].
Transparency and Accountability:
This model card summarizes details on the models' architecture,
capabilities, limitations, and evaluation processes.
A responsibly developed open model offers the opportunity to share
innovation by making LLM technology accessible to developers and researchers
across the AI ecosystem.
Risks identified and mitigations:
Perpetuation of biases: It's encouraged to perform continuous monitoring
(using evaluation metrics, human review) and the exploration of de-biasing
techniques during model training, fine-tuning, and other use cases.
Generation of harmful content: Mechanisms and guidelines for content safety
are essential. Developers are encouraged to exercise caution and implement
appropriate content safety safeguards based on their specific product policies
and application use cases.
Privacy violations: Models were trained on data filtered for removal of PII
(Personally Identifiable Information). Developers are encouraged to adhere to
privacy regulations with privacy-preserving techniques.
