Base model: Qwen2.5-3B-Instruct (4-bit quantized)
Method: QLoRA (r=16, alpha=32)
Training: 4 epochs on ~98K samples (final train loss 0.465)
Languages: English, Hindi, Telugu, Kannada, Tamil
Domains: General instruction following, coding, reasoning, safety alignment, Indic languages

Training Epochs

Epoch	Dataset Size	Learning Rate	Focus
1	36K	2e-4	Core instruction following
2	36K	5e-5	Continued refinement
3	98K	2e-5	Expanded: safety, Indic languages, clean instructions
4	98K	1e-5	Careful refinement (low LR, anti-forgetting)

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("laabamone/laabam-ai-3b-v1")
tokenizer = AutoTokenizer.from_pretrained("laabamone/laabam-ai-3b-v1")

messages = [{"role": "user", "content": "Hello, who are you?"}]
inputs = tokenizer.apply_chat_template(messages, return_tensors="pt")
outputs = model.generate(inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

License

Apache 2.0