🧠 SALAMA LLM — Swahili Instruction-Tuned Text Generation Model
👨💻 Developer: AI4NNOV
✍️ Authors: AI4NNOV
📦 Version: v1.0
📜 License: Apache 2.0
🛠️ Model Type: Instruction-Tuned Large Language Model
🧩 Base Model: Jacaranda/UlizaLlama
🌍 Overview
SALAMA LLM is the language understanding and generation engine of the SALAMA Framework — a modular Speech-to-Speech (STS) AI pipeline built for African languages.
The model is fine-tuned on Swahili instruction datasets to enable natural, culturally relevant responses in text generation, summarization, question answering, and translation.
This model represents a major step in bridging the linguistic digital divide by providing high-quality Swahili AI text generation capabilities within an open, scalable framework.
🧱️ Model Architecture
SALAMA LLM is based on Jacaranda/UlizaLlama, fine-tuned using Parameter-Efficient Fine-Tuning (PEFT) via LoRA/QLoRA.
The architecture supports mixed Swahili-English text inputs while focusing on fluent Swahili text generation for both casual and formal domains.
| Parameter | Value |
|---|
| Base Model | Jacaranda/UlizaLlama |
| Fine-Tuning | QLoRA / LoRA (PEFT) |
| Precision | 4-bit quantization |
| Optimizer | AdamW |
| Learning Rate | 2e-5 |
| Epochs | 3–5 |
| Frameworks | Transformers, TRL, PEFT, Unsloth |
| Languages | Swahili (sw), English (en) |
📚 Datasets
| Dataset | Description | Purpose |
|---|
saillab/alpaca_swahili_taco | Swahili Alpaca-style instruction-response dataset | Instruction tuning |
Jacaranda/kiswallama-pretrained | 321M Swahili tokens, custom tokenizer (20K vocab) | Base Swahili adaptation |
| Custom Swahili QA corpus | Curated Q&A and summarization samples | Conversational fine-tuning |
🧠 Model Capabilities
✅ Text generation in Swahili and English
✅ Instruction-following, summarization, and dialogue
✅ Question answering and translation (EN ↔ SW)
✅ Sentiment and named-entity recognition
✅ Contextually and culturally aligned text generation
📊 Evaluation Metrics
| Metric | Score | Description |
|---|
| BLEU | 0.49 | Measures fluency and translation accuracy |
| ROUGE-L | 0.61 | Summarization recall and overlap |
| Accuracy (QA) | 95.5% | Accuracy on Swahili QA tasks |
| CER | 0.28 | Character Error Rate |
| F1 (avg) | 0.90+ | Weighted average across tasks |
⚙️ Usage (Python Example)
Below is a quick example to load and use SALAMA LLM for Swahili text generation:
1from transformers import AutoTokenizer, AutoModelForCausalLM
2import torch
3
4# Load model and tokenizer
5model_name = "EYEDOL/salama-llm" # Change to your Hugging Face repo name
6tokenizer = AutoTokenizer.from_pretrained(model_name)
7model = AutoModelForCausalLM.from_pretrained(
8 model_name,
9 torch_dtype=torch.bfloat16,
10 device_map="auto"
11)
12
13# Swahili text prompt
14prompt = "Andika sentensi fupi kuhusu umuhimu wa elimu."
15
16inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
17outputs = model.generate(
18 **inputs,
19 max_new_tokens=120,
20 temperature=0.7,
21 top_p=0.9,
22 repetition_penalty=1.05
23)
24
25print(tokenizer.decode(outputs[0], skip_special_tokens=True))
🦩 Example Output:
“Elimu ni msingi wa maendeleo, humwezesha mtu kuelewa dunia na kuboresha maisha yake na jamii kwa ujumla.”
⚡ Key Features
- 🧩 Optimized for African low-resource NLP contexts
- 💬 Instruction-following in Swahili and English
- ⚙️ Lightweight and efficient (QLoRA fine-tuned; runs on single 24 GB GPU)
- 🌍 Culturally aligned text generation
- 🦶 Open-source and extendable to other African languages
🚫 Limitations
- ⚠️ May underperform with heavy code-switching (Swahili-English mix)
- 👤 Not yet optimized for rare dialects or poetic forms
- 📚 Limited exposure to specialized (medical/legal) corpora
- 🔊 Relies on accurate STT transcription in end-to-end speech-to-speech use
🔗 Related Models
🧾 Citation
If you use SALAMA LLM, please cite:
1@misc{salama_llm_2025,
2 title={SALAMA LLM: Swahili Instruction-Tuned Text Generation Model},
3 author={AI4NNOV},
4 year={2025},
5 publisher={Hugging Face},
6 howpublished={\url{https://huggingface.co/EYEDOL/salama-llm}}
7}
💡 “Elimu ni msingi wa maendeleo — Knowledge is the foundation of progress.”
