from transformers import AutoModelForCausalLM, AutoProcessor
import torch

model_id = "pankajpandey-dev/gemma-4-e4b-hindi-instruct"
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.bfloat16, device_map="auto")
proc  = AutoProcessor.from_pretrained(model_id)

msgs = [{"role": "user", "content": [{"type": "text", "text": "मशीन लर्निंग को आसान शब्दों में समझाओ।"}]}]
inputs = proc.apply_chat_template(msgs, add_generation_prompt=True, tokenize=True,
                                  return_dict=True, return_tensors="pt").to(model.device)
out = model.generate(**inputs, max_new_tokens=256, use_cache=True)
print(proc.decode(out[0], skip_special_tokens=True))

Example outputs

Prompt: भारत के बारे में एक रोचक तथ्य बताओ।

भारत दुनिया में सबसे अधिक भाषाओं वाले देशों में से एक है — 22 आधिकारिक भाषाएँ और 1,000 से अधिक बोलियाँ। हिंदी एक इंडो-आर्यन भाषा है, जबकि तमिल एक द्रविड़ भाषा है।

Training details


Base model	`unsloth/gemma-4-E4B-it`
Method	LoRA (r=16, α=16), response-only loss
Framework	Unsloth
Data	~10k Hindi instruction pairs (AI4Bharat indic-instruct: anudesh + dolly, hi splits)
Epochs	2
LR / schedule	1e-4, cosine
Precision	bf16 (4-bit QLoRA base)
Hardware	Single NVIDIA L4 (24 GB)
Final train loss	~0.29

Trained text-only (vision layers frozen), single-BOS chat template to avoid double-BOS corruption.

Related repos

GGUF (Q4/Q5/Q8): pankajpandey-dev/gemma-4-e4b-hindi-instruct-GGUF
LoRA adapter: pankajpandey-dev/gemma-4-e4b-hindi-instruct-lora

Provenance & license (please read)

Mixed-license lineage — review all before redistribution or commercial use:

Weights derive from Gemma 4, under the Gemma Terms of Use.
Data from AI4Bharat indic-instruct-data-v0.1:
- Dolly split — from databricks-dolly-15k, CC-BY-SA-3.0.
- Anudesh split — responses from Llama-2-70B, so the Llama 2 Community License applies.

Raw training data is not redistributed here. You are responsible for complying with the Gemma, Llama 2, and CC-BY-SA terms.

Limitations

~8B-class model: strong Hindi fluency, but can hallucinate facts and occasionally repeat phrasing on long open-ended generation.
Tuned for single-turn Hindi instructions; long multi-turn chat is not the focus.
Not safety-aligned for production.

Acknowledgements

Base model by Google (Gemma 4). Data by AI4Bharat. Fine-tuning with Unsloth. 🙏