Indra is a fine-tuned LLM built on Qwen2.5-Coder-1.5B-Instruct, trained on Indian language and history datasets. It can converse in 10 Indian languages, answer questions about Indian history & culture, and still write code.

✨ Highlights

🗣️ 10 Indian languages — Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Odia
📜 Indian history & culture — From Indus Valley to modern India
🔄 Bilingual conversations — Hinglish, code-switching, vernacular queries
💻 Coding preserved — Still writes Python, JavaScript, and full-stack code
📦 Lightweight — 1.5B parameters, runs on consumer GPUs

Quick Start

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("RockySinghRajput/Indra", torch_dtype="auto", device_map="auto")
tokenizer = AutoTokenizer.from_pretrained("RockySinghRajput/Indra")

messages = [
    {"role": "system", "content": "आप Indra हैं, एक बुद्धिमान AI सहायक।"},
    {"role": "user", "content": "भारत के स्वतंत्रता संग्राम के बारे में बताइए।"}
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer([text], return_tensors="pt").to(model.device)

output = model.generate(inputs.input_ids, max_new_tokens=512, temperature=0.7, top_p=0.9)
print(tokenizer.decode(output[0][len(inputs.input_ids[0]):], skip_special_tokens=True))

Hinglish Example

python

messages = [
    {"role": "system", "content": "You are Indra, a helpful AI that understands Hindi and English."},
    {"role": "user", "content": "Mujhe Python mein ek calculator banana hai, kaise karoon?"}
]

Model Details

Property	Value
Base Model	Qwen/Qwen2.5-Coder-1.5B-Instruct
Parameters	1.5B
Type	Causal Language Model (merged LoRA fine-tune)
Languages	English, Hindi, Bengali, Tamil, Telugu, Marathi, Gujarati, Kannada, Malayalam, Punjabi, Odia
License	Apache 2.0
Developed by	RockySinghRajput
Related Model	IndraCoder (coding-focused version)

Supported Languages

Language	Script	Code
Hindi	देवनागरी	hi
Bengali	বাংলা	bn
Tamil	தமிழ்	ta
Telugu	తెలుగు	te
Marathi	मराठी	mr
Gujarati	ગુજરાતી	gu
Kannada	ಕನ್ನಡ	kn
Malayalam	മലയാളം	ml
Punjabi	ਪੰਜਾਬੀ	pa
Odia	ଓଡ଼ିଆ	or
English	Latin	en

Training Details

Training Data

Fine-tuned on 6 Indian language & culture datasets:

Dataset	Purpose	Content
CohereForAI/aya_dataset	Multilingual Indian conversations	10 Indian languages
Cognitive-Lab/Hindi-Instruct	Hindi instruction following	Hindi Q&A
sarvamai/samvaad-hi-en-instruct-v2	Bilingual conversations	Hindi-English
CohereForAI/aya_collection (India-filtered)	Indian history & culture	History, heritage, knowledge
CohereForAI/aya_collection (Hindi WikiQA)	Hindi knowledge base	Wikipedia-sourced Hindi QA
ai4bharat/IndicSentiment	Hindi language understanding	Sentiment analysis

Indian History Coverage

The model has been trained on Indian history spanning:

Ancient India — Indus Valley Civilization, Vedic period, Maurya & Gupta Empires
Medieval India — Delhi Sultanate, Mughal Empire, Vijayanagara, Maratha Empire, Bhakti & Sufi movements
Modern India — British Raj, Freedom struggle, Independence, Republic
Indian Constitution — Fundamental rights, governance, democracy
Culture & Heritage — Art, architecture, literature, philosophy, classical music, Ayurveda, Yoga

Training Procedure

Method: LoRA (Low-Rank Adaptation) → merged into base model
LoRA Config: r=16, alpha=16, dropout=0.05
Target Modules: q_proj, k_proj, v_proj, o_proj
Epochs: 2
Learning Rate: 2e-5 (lower to preserve base capabilities)
Optimizer: paged_adamw_8bit
Sequence Length: 512 tokens
Precision: FP16 mixed precision
Quantization: 4-bit NF4 (QLoRA) during training

Compute Infrastructure

Hardware: NVIDIA T4 GPU
Training Time: ~2 hours

Capabilities

✅ What Indra Can Do

Converse in Indian languages — Answer questions in Hindi, Bengali, Tamil, Telugu, and more
Indian history & culture — Detailed knowledge of Indian civilization
Hinglish/bilingual — Handle mixed Hindi-English naturally
General knowledge — Science, geography, current affairs with Indian context
Coding — Write code in Python, JavaScript, and other languages
Sentiment analysis — Understand sentiment in Hindi text

⚠️ Limitations

1.5B model — Smaller than commercial LLMs; may produce shorter or less nuanced responses
Script limitations — Stronger in Hindi/Devanagari; other Indian scripts may have lower quality
Not a translator — Optimized for conversation, not professional translation
May hallucinate — Always verify historical facts and generated content
English-centric base — Indian language abilities are fine-tuned on top of an English-dominant base

❌ Out-of-Scope Use

Professional translation services
Legal or medical advice
Factual source of record for academic research
Generating harmful or culturally insensitive content

Evaluation

Test	Language	Task	Result
Hindi Chat	Hindi	Gandhi's role in freedom struggle	✅ Detailed response
Indian History	English	Gupta Empire Golden Age	✅ Accurate overview
Hinglish	Mixed	"Python mein calculator banana hai"	✅ Code + Hindi explanation
Coding	English	Binary search implementation	✅ Working code
Indian Culture	Hindi	Classical music ragas	✅ Cultural knowledge

Model Family

Model	Focus	Repo
Indra (this model)	Indian languages + history	RockySinghRajput/Indra
IndraCoder	Coding + debugging	RockySinghRajput/Indracoder
IndraCoder-7B	Advanced coding + chat (coming soon)	RockySinghRajput/IndraCoder-7B

Citation

bibtex

@misc{indra2025,
  title={Indra: An Indian Language AI Assistant},
  author={RockySinghRajput},
  year={2025},
  publisher={HuggingFace},
  url={https://huggingface.co/RockySinghRajput/Indra}
}

Contact

HuggingFace: RockySinghRajput