HornLM — Qwen3 8B Instruct v1
HornLM is the first open-source language model fine-tuned specifically for Djibouti and the Horn of Africa.
Built on Qwen3-8B and adapted using QLoRA on a curated instruction dataset generated from more than 62,000 documents collected across government, legal, economic, educational, and institutional sources from Djibouti and East Africa.
HornLM aims to become the foundational AI infrastructure layer for the Horn of Africa, providing local knowledge retrieval, multilingual language models, and developer-friendly APIs.
Model Details
| Property | Value |
|---|
| Model name | hornlm/hornlm-8b-instruct-v1 |
| Base model | Qwen/Qwen3-8B |
| Fine-tuning method | QLoRA (4-bit NF4 + LoRA adapters) |
| LoRA rank / alpha | r=16 / α=32 |
| Training epochs | 3 |
| Effective batch size | 16 |
| Learning rate | 2e-4 |
| Max sequence length | 4096 tokens |
| Training framework | Unsloth + Hugging Face TRL |
| Precision | FP16 |
| Languages | English, French |
| Future languages | Somali, Amharic, Swahili, Arabic |
| License | Apache 2.0 |
About HornLM
HornLM is part of a broader initiative to build open AI infrastructure for the Horn of Africa.
The project includes:
- Regional knowledge corpus (5K+ documents)
- Fine-tuned language models
- RAG API for enterprise and public sector applications
- Embedding models
- Search and retrieval infrastructure
- Open datasets for African AI research
Training Dataset
The SFT dataset was generated from the HornLM Knowledge Corpus, a curated collection of public documents from Djibouti and East Africa.
Current Sources
| Source Type | Coverage |
|---|
| Government portals | Laws, decrees, public services |
| National statistics | Economic and demographic data |
| Ministries | Education, health, finance, agriculture |
| International organizations | Development reports and indicators |
| Public institutions | Administrative procedures |
| Universities | Research and educational resources |
| Local news outlets | Regional news and analysis |
The corpus currently contains more than 5,000 indexed documents.
Instruction-response pairs were generated and validated before supervised fine-tuning.
Usage
Load with Unsloth (Recommended)
1from unsloth import FastLanguageModel
2
3model, tokenizer = FastLanguageModel.from_pretrained(
4 model_name="hornlm/hornlm-8b-instruct-v1",
5 max_seq_length=4096,
6 load_in_4bit=True,
7)
8
9FastLanguageModel.for_inference(model)
10
11prompt = """
12Below is an instruction about Djibouti or the Horn of Africa.
13Write a helpful and accurate response.
14
15### Instruction:
16What are the main economic sectors of Djibouti?
17
18### Response:
19"""
20
21inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
22
23outputs = model.generate(
24 **inputs,
25 max_new_tokens=256,
26 do_sample=False
27)
28
29print(tokenizer.decode(outputs[0], skip_special_tokens=True))
Load with Transformers + PEFT
1from transformers import AutoTokenizer, AutoModelForCausalLM
2from peft import PeftModel
3
4base = AutoModelForCausalLM.from_pretrained(
5 "Qwen/Qwen3-8B",
6 load_in_4bit=True,
7 device_map="auto",
8)
9
10model = PeftModel.from_pretrained(
11 base,
12 "hornlm/hornlm-8b-instruct-v1"
13)
14
15tokenizer = AutoTokenizer.from_pretrained(
16 "hornlm/hornlm-8b-instruct-v1"
17)
Prompt Format
HornLM was fine-tuned using an instruction-following format.
1Below is an instruction about Djibouti or the Horn of Africa.
2Write a helpful and accurate response.
3
4### Instruction:
5{user question}
6
7### Response:
Intended Use
- Questions about Djibouti
- Questions about the Horn of Africa
- Public administration
- Economic analysis
- Educational applications
- Research assistance
- Government knowledge systems
- Retrieval-Augmented Generation (RAG)
- AI applications for startups and institutions
Out-of-Scope Use
- Medical advice
- Legal advice without verification
- Financial decisions
- High-risk government decision-making
- General-purpose chat unrelated to the region
Users should always verify critical information using official sources.
Roadmap
HornLM v1
- English
- French
- Djibouti-focused knowledge
- RAG API
- 5K+ documents
HornLM v2
- Somali support
- Amharic support
- Swahili support
- Expanded East African corpus
HornLM v3
- Multimodal model
- OCR for government documents
- Speech support
- Regional AI agents
Project
HornLM is the first open-source AI infrastructure layer for Djibouti and the Horn of Africa.
Our mission is to make regional knowledge accessible through open, multilingual, and sovereign AI systems.
Features:
- 5K+ regional documents
- RAG API
- Fine-tuned LLMs
- Open datasets
- Developer platform
Citation
1@misc{hornlm2026,
2 author = {Margagui Team},
3 title = {HornLM: Open-Source AI Infrastructure for the Horn of Africa},
4 year = {2026},
5 publisher = {Hugging Face},
6 howpublished = {https://huggingface.co/hornlm/hornlm-8b-instruct-v1}
7}
