ai4bharat/indic-instruct-data-v0.1 — anudesh (Hindi split): native crowd-sourced Hindi instructions
ai4bharat/indic-instruct-data-v0.1 — dolly (Hindi split, filtered to chrF ≥ 60): broad instruction variety

All examples ≤ 2048 tokens, formatted with the MiniCPM5 ChatML template.

Training Configuration

Hyperparameter	Value
LoRA rank	32
LoRA alpha	64
LoRA dropout	0.0
Target modules	q, k, v, o, gate, up, down
Batch size (effective)	16
Learning rate	2e-4
LR scheduler	cosine
Warmup steps	15
Epochs	2
Total steps	500
Precision	fp16 (4-bit base)
Hardware	NVIDIA Tesla T4 (Colab)
Training time	~60 minutes
Final training loss	1.108

Usage

With Transformers

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_id = "pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct"

tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_id,
    torch_dtype=torch.float16,
    device_map="auto",
    trust_remote_code=True,
)

messages = [
    {"role": "user", "content": "नमस्ते! बारिश के दिन पर एक छोटी कविता लिखो।"}
]

inputs = tokenizer.apply_chat_template(
    messages,
    add_generation_prompt=True,
    return_tensors="pt",
).to(model.device)

outputs = model.generate(
    inputs,
    max_new_tokens=256,
    temperature=0.7,
    top_p=0.9,
    do_sample=True,
    repetition_penalty=1.1,
)
print(tokenizer.decode(outputs[0][inputs.shape[-1]:], skip_special_tokens=True))

Recommended Generation Parameters

temperature: 0.7 (lower = more focused, higher = more creative)
top_p: 0.9
repetition_penalty: 1.1
max_new_tokens: 256–512 depending on task

LoRA Adapter Only

If you prefer to load the LoRA adapter on top of the base model (~85 MB vs 2.2 GB), it's available in the lora_adapter/ folder of this repo:

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("openbmb/MiniCPM5-1B", trust_remote_code=True)
model = PeftModel.from_pretrained(base, "pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct", subfolder="lora_adapter")

Example Outputs

Prompt: बारिश के दिन पर एक छोटी कविता लिखो। Response: (creative Hindi poetry generation)

Prompt: मशीन लर्निंग क्या है? सरल हिंदी में समझाइए। Response: (simplified Hindi explanation of ML)

Prompt: नमस्ते! अपना परिचय दीजिए। Response: (conversational Hindi self-introduction)

Quantized Versions (GGUF)

For running locally with llama.cpp, Ollama, LM Studio, or other GGUF-compatible inference engines.

Acknowledgements

OpenBMB for the MiniCPM5-1B base model
AI4Bharat (IIT Madras) for the indic-instruct-data dataset
Unsloth for the training framework

Citation

If you use this model in your work, please cite:

bibtex

@misc{pandey2026minicpm5hindi,
  title  = {MiniCPM5-1B-Hindi-Instruct},
  author = {Pankaj Pandey},
  year   = {2026},
  url    = {https://huggingface.co/pankajpandey-dev/MiniCPM5-1B-Hindi-Instruct}
}

Part of an ongoing effort to bring strong open-source LLMs to Indian languages. Feedback and contributions welcome via the community tab.