Developed by Muhammad Zubair, a Software Engineer specializing in AI Infrastructure and Cloud Platforms. This is the first AI model specifically fine-tuned for the Khattak dialect of Pashto (spoken in Karak, Kohat, and Nowshera).

👤 Developer Profile

Name: Muhammad Zubair
Profession: Software Engineer
Portfolio: mzubair.online
Project Case Study: Khatta-ka-LLM by Muhammad Zubair

🌟 Model Description

Standard Pashto AI models often fail to understand regional dialects. Khatta-ka-LLM was developed by Muhammad Zubair to bridge this gap. It understands the unique vocabulary, spelling, and grammar of the Khattak Pashto dialect (e.g., using "شوشته" and "ایس نه کاون").

Built on top of junaid008/qehwa-pashto-llm (Qwen2.5-7B), Khatta-ka AI has been fine-tuned using LoRA via Unsloth to adapt to the Khattak dialect. It understands regional Khattak vocabulary, grammar shifts (such as using "یں" instead of "یم" for the first-person copula, and "کاوں" instead of "کوم"), and phonetic variations.

⚠️ Note: This model is a LoRA adapter. It is designed to take instructions in English or standard Pashto and reply in authentic Khattak Pashto.

✨ Capabilities

✅ Answers questions in Khattak Pashto ✅ Responds to English instructions in Khattak Pashto ✅ Responds to Standard Pashto instructions in Khattak dialect ✅ Understands regional vocabulary specific to Karak, Kohat, and Nowshera ✅ Trained 2x faster using Unsloth by Muhammad Zubair

📘 The Official Khattak Dialect Grammar & Rulebook

Here is the complete summary of the rules this AI has been trained on to understand the Khattak dialect:

1. Pronouns & Possession

My / Mine: Uses مو والا (Mo wala) instead of standard Zama.
To me: Uses موته (Mota) instead of standard Mala.
To you: Uses توته (Tota) instead of standard Tata.
From me: Uses رونه (Rona) instead of standard Rana.
We: Uses موخ (Moxh) instead of standard Mung.

2. The "Noon Ghunna" (ں) Rule (Crucial!)

Verbs (First Person): When saying "I do something", the verb ends in a nasal Noon Ghunna (ں) instead of the standard "M" (م).
- I do: کاوں (Kaon)
- I wash: وینزاں (Wenzan)
- I write: لېکاں (Lekan)
- I will come: روساں (Rosan)
The "I am" Copula: Standard Pashto uses یم (Yam). Khattak uses یں (Yay + Noon Ghunna). Example: Za khwash یں (I am happy).
Nouns: Nouns do not use the Noon Ghunna. They keep the hard "N" (ن). Example: Asmon (اسمون), Baron (بارون), Pashtun (پښتون).

3. Tenses & Verbs

Future Tense: Uses بو (Bo) instead of standard به (Ba). Example: Za bo sabo char kaon (I will do work tomorrow).
"To Say/Tell": Uses یای (Yai) for "he says" and ویاوں (Wayaon) for "I will tell".
"To Read": Uses یای (Yai) instead of standard Lwali. Example: Agha kitob yai (He is reading a book).

4. Phonetics & Vowel Shifts

The "O" Shift: Many standard "A" sounds shift to a deep "O" sound.
- Asmaan ➡️ Asmon (اسمون)
- Baraan ➡️ Baron (بارون)
- Bazaar ➡️ Bazor (بازور)
- Maakham ➡️ Mashom (ماښوم)
- Yaada ➡️ Yoda (یوده)
Masculine Singular Nouns: Words that end in "i" in standard Pashto often end in "a" in Khattak.
- Gadi (Car) ➡️ Gada (ګاډه)
- Kanri (Stone) ➡️ Konrha (کاڼه)
- Halak/Kakai (Boy) ➡️ Orhka (اوړکه)

5. Unique Khattak Vocabulary

Family: Grandmother = انیو (Anyo)
People: Boy = اوړکه (Orhka), Girl = اوړکې (Orhkay)
Animals: Snake = منګور (Mangor)
Nature: Day = ورېز (Wrez)
Actions: Stand up = چګ شه (Chag sha), Flying = نغېږي (Neghezxhi), Crying = ژوړې (Zorhay)
Adjectives: Big = استر (Astr), Bad/Dirty = کسو (Kso), Tired (Singular) = ستړه (Starha)
Items: Tea leaves = پتې (Pathay)

💻 Installation

pip install transformers accelerate torch peft

For faster inference (Recommended):

pip install unsloth

🚀 How to Use

This model is an adapter. To use it, load it alongside the base model.

✅ Method 1 — Unsloth (Recommended - 2x Faster)

Best for: Speed-optimized usage, Colab, A100/H100/T4

from unsloth import FastLanguageModel
import torch

model_name = "Muhammad-Zubair796/Khatta-ka"

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name     = model_name,
    max_seq_length = 2048,
    dtype          = None,
    load_in_4bit   = True,
)
FastLanguageModel.for_inference(model)

alpaca_prompt = """Below is an instruction. Write a detailed response in Pashto.

### Instruction:
{}

### Response:
"""

inputs = tokenizer(
    [
        alpaca_prompt.format("Where did the boy go?", "")
    ], return_tensors = "pt").to("cuda")

outputs = model.generate(**inputs, max_new_tokens = 64, use_cache = True)
response = tokenizer.batch_decode(outputs, skip_special_tokens = True)[0]

khattak_output = response.split("### Response:\n")[-1].strip()
print(khattak_output)

✅ Method 2 — Standard Transformers & PEFT

Best for: Standard production environments without Unsloth

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base_model_name = "junaid008/qehwa-pashto-llm"
adapter_name = "Muhammad-Zubair796/Khatta-ka"

tokenizer = AutoTokenizer.from_pretrained(base_model_name)
base_model = AutoModelForCausalLM.from_pretrained(
    base_model_name,
    torch_dtype = torch.bfloat16,
    device_map  = "auto",
)

# Load the Khattak Adapter
model = PeftModel.from_pretrained(base_model, adapter_name)

alpaca_prompt = """Below is an instruction. Write a detailed response in Pashto.

### Instruction:
{}

### Response:
"""

def generate(prompt):
    inputs = tokenizer(
        alpaca_prompt.format(prompt, ""),
        return_tensors = "pt",
    ).to("cuda")

    outputs = model.generate(
        **inputs,
        max_new_tokens     = 64,
        temperature        = 0.7,
        do_sample          = True,
        pad_token_id       = tokenizer.eos_token_id,
    )

    response = tokenizer.decode(outputs[0], skip_special_tokens=True)
    return response.split("### Response:\n")[-1].strip()

print(generate("Where did the boy go?"))

⚙️ Hardware Requirements

Method	VRAM	Speed
bfloat16 full	16GB+	✅ Fast
4-bit quantized	8GB+	✅ Good
Unsloth (4-bit)	8GB+	🔥 2x Faster

📊 Training Details

This model was trained using Unsloth for optimized, memory-efficient fine-tuning by Muhammad Zubair.

Parameter	Value
Base model	`junaid008/qehwa-pashto-llm`
Training Framework	Unsloth + TRL (SFTTrainer)
Epochs	4
Sequence length	2,048 tokens
LoRA Rank (r)	16
LoRA Alpha	16
Target Modules	q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Learning rate	2e-4
Optimizer	adamw_8bit
Batch Size	2 (with 4 gradient accumulation steps)
Warmup Steps	10

🗣️ Example Outputs

English input:

Prompt: I am doing nothing, just sitting. Output: زه ایس نه کاوں، بس نوست یں

English input:

Prompt: Where did the boy go? Output: اوړکه چےتا لوړو؟

English input:

Prompt: The weather is very beautiful today, it is raining. Output: نن موسم ډېر شوشته دا، بارون وریږي

📜 License

Apache 2.0 — free to use, modify, and distribute with attribution.

🤝 Citation

@misc{khattaka-pashto-llm,
  author    = {Muhammad Zubair},
  title     = {Khatta-ka AI: Khattak Pashto LLM},
  year      = {2026},
  publisher = {HuggingFace},
  url       = {https://huggingface.co/Muhammad-Zubair796/Khatta-ka}
}