llama-3.2-1b-urdu-en-qlora – AI Model by kashif1085 | AlphaNeural AI

You can deploy this model and start earning money today!

kashif1085/
llama-3.2-1b-urdu-en-qlora

like

0

meta-llama/Llama-3.2-1B-Instruct

Views

No views yet

llama-3.2-1b-urdu-en-qlora

QLoRA fine-tune of meta-llama/Llama-3.2-1B-Instruct for Urdu→English translation.

Training

Dataset: kashif1085/urdu-en-translation-instruct (filtered subset of Helsinki-NLP/opus-100, en-ur)
QLoRA: r=16, alpha=32, dropout=0.05
4-bit NF4 + double-quant, fp16 compute (T4 GPU)
3 epochs, lr=0.0002, cosine schedule
Effective batch size: 2 × 4 = 8
Hardware: Colab T4 (free tier), trained with Unsloth (~2x speedup)
Total training time: ~63 minutes

Evaluation (n=200)

	BLEU	chrF
Base	3.97	25.48
LoRA	10.10	31.78
Δ	+6.13	+6.30

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

base = AutoModelForCausalLM.from_pretrained('meta-llama/Llama-3.2-1B-Instruct')
tok  = AutoTokenizer.from_pretrained('meta-llama/Llama-3.2-1B-Instruct')
model = PeftModel.from_pretrained(base, 'kashif1085/llama-3.2-1b-urdu-en-qlora')

Limitations

Trained on a small slice (8000 pairs) — not a production translator.
Biased toward the news-domain style of opus-100.
Portfolio piece. Not for real-world deployment.
BLEU 10 is well below production NMT systems (Google Translate ~30-40 BLEU on this language pair). The +154% relative improvement over base demonstrates the technique works at small scale.

Loss curve