Saudi Dialect LFM2.5 — Instruction-Tuned Arabic Dialect Model
Model Description
This model is a fine-tuned version of Liquid AI’s LFM2.5‑1.2B‑Instruct, adapted for Saudi dialect conversational generation.
The base model belongs to the LFM2.5 family — hybrid state-space + attention language models designed for fast on-device inference,low memory usage, and strong performance relative to size. It contains ~1.17B parameters, 32k context length, and supports multilingual generation including Arabic.
This fine-tuned variant specializes the model for Saudi dialect conversational patterns, improving fluency, dialect authenticity, and instruction following for regional Arabic use cases.
Intended Use
Primary Use Cases
Saudi dialect chatbots
Customer support assistants
Conversational agents
Arabic NLP research
Dialect-aware RAG pipelines
Dialogue generation systems
Out-of-Scope Uses
Legal/medical advice
Safety-critical decision making
High-precision knowledge tasks without retrieval
Sensitive content generation
Training Details
Base Model
Architecture: Hybrid state-space + attention
Parameters: ~1.17B
Context length: 32,768 tokens
Training tokens: ~28T
Languages: Multilingual including Arabic
Dataset
Fine-tuned on:
Dataset:HeshamHaroon/saudi-dialect-conversations
Domain:
Conversational dialogue
Language:
Saudi dialect Arabic
Format:
Instruction → Response pairs
Purpose:
Increase dialect authenticity and conversational naturalness.
Training Configuration
(Extracted from training notebook)
Parameter
Value
Epochs
4
Learning Rate
2e-4
Batch Size
16
Gradient Accumulation
4
Optimizer
AdamW
LR Scheduler
Linear
Warmup Ratio
0.03
Sequence Length
8096
Precision
FP16
Training Type
Supervised Fine-Tuning (SFT)
Training Procedure
Training was performed using:
Transformers
TRL SFTTrainer
LoRA fine-tuning
Mixed precision
Gradient accumulation
The base model weights were adapted rather than retrained from scratch.
Evaluation
Qualitative evaluation indicates:
Improved dialect fluency
Reduced MSA leakage
Better conversational tone
Higher lexical authenticity
Dialect-specific fine-tuning is known to significantly increase dialect generation accuracy and reduce standard-Arabic drift in Arabic LLMs.
Performance Characteristics
Strengths
Very fast inference
Low memory footprint
Strong conversational coherence
Good instruction following
Limitations
Smaller model → limited factual depth
May hallucinate
Less capable for complex reasoning vs larger models
