Generates short, helpful replies in Moroccan Darija (Arabic script) for everyday home-services questions — plumber prices in Tangier, finding a painter in Casablanca, washing-machine repair quotes, etc. Built as a commercial-safe (Track A) variant that excludes CC-BY-NC-ND data.

How to use

python

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

tok = AutoTokenizer.from_pretrained('Qwen/Qwen2.5-1.5B-Instruct')
base = AutoModelForCausalLM.from_pretrained(
    'Qwen/Qwen2.5-1.5B-Instruct',
    torch_dtype=torch.bfloat16,
    device_map='cuda',
)
model = PeftModel.from_pretrained(base, 'Samielakkad1/jakma-darija-A-adapter')

msg = [{'role':'user','content':'شحال كيسوا تركيب صنبور؟'}]
prompt = tok.apply_chat_template(msg, tokenize=False, add_generation_prompt=True)
out = model.generate(**tok(prompt, return_tensors='pt').to('cuda'), max_new_tokens=200)
print(tok.decode(out[0], skip_special_tokens=True))

Training details

Setting	Value
Base model	Qwen/Qwen2.5-1.5B-Instruct
Method	LoRA via PEFT 0.13.2 / TRL SFTTrainer
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Target modules	q_proj, k_proj, v_proj, o_proj, up_proj, gate_proj, down_proj
Epochs	2
Learning rate	2e-4
Batch size	1 (grad accumulation 16)
Max sequence length	1024
Optimizations	NEFTune α=5.0, SFT sequence packing
Hardware	1× NVIDIA RTX A5000 (24 GB) on RunPod
Total wall time	~80 minutes
Final train loss	0.5612

Training data

Source	Records	License
DODa (Darija Open Dataset)	48,035	research-personal
DODa parallel pairs	18,410	research-personal
jak.ma hand-crafted exemplars	210 (10× upsampled from 21)	internal / hand-crafted
Track A train total	53,325	commercial-safe subset

Validation: 6,665. Test: 6,665. Quality filter dropped 537. Deduplication dropped 755. Upsampling weights bias toward hand-crafted in-domain examples.

Intended use

Powering the jak.ma Moroccan home-services platform conversational layer.
Research baseline for Moroccan Darija LLM work at the 1–2 B parameter scale.

Limitations

Small base model (1.5 B). Not competitive with frontier models on open-ended reasoning.
Adapter is 1.94 GB because the trainer saved embed_tokens + lm_head alongside the LoRA weights. Load with peft.PeftModel.from_pretrained — the loader handles vocab alignment.
Trained on a curated Track A corpus; may underperform on topics outside home-services.

Author

Sami El Akkad (@Samielakkad1) — MSc, Tsinghua SIGS — building jak.ma.

Source

Training code: https://github.com/Samielakkad/jakma

samielakkad1/jakma-darija-A-adapter

jakma-darija-A-adapter

What it does