⚠️
Note: This 0.6B version is undertrained and does not reliably follow Hindi instructions. For a working Hindi model, please use
Qwen3-4B-Hindi-Instruct-v2 (
GGUF here ).
Qwen3-0.6B Hindi Instruct v1 — GGUF
The smallest Hindi-capable instruction model you can run locally — fits in 370MB, runs on any laptop, no GPU needed.
Fine-tuned from Qwen/Qwen3-0.6B on English to Hindi instruction pairs.
Quantized to GGUF for local inference with llama.cpp, LM Studio, and Ollama.
Quick Start
Download the model using huggingface-cli:
huggingface-cli download pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1-GGUF Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf --local-dir .Run with llama.cpp:
./llama-cli -m Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf -p "भारत की राजधानी क्या है?" -n 256Or open directly in LM Studio — New Model — Search pankajpandey-dev
Available Versions
File Quantization Size RAM Needed Best For Qwen3-0.6B-Hindi-v1-Q2_K.gguf Q2_K 0.28 GB 1 GB Minimum hardware Qwen3-0.6B-Hindi-v1-Q4_K_M.gguf Q4_K_M 0.37 GB 1.5 GB Recommended Qwen3-0.6B-Hindi-v1-Q5_K_M.gguf Q5_K_M 0.41 GB 2 GB Better quality Qwen3-0.6B-Hindi-v1-Q8_0.gguf Q8_0 0.60 GB 2.5 GB Highest quality
Not sure which to pick? Always start with Q4_K_M — best balance of speed, size, and quality.
Model Details
Property Value Base Model Qwen/Qwen3-0.6B Parameters 600M Architecture Qwen2 — fully supported by llama.cpp Fine-tune Method QLoRA with LoRA r=16 alpha=16 Training Steps 60 steps Training Data 2000 English to Hindi instruction pairs Max Context 2048 tokens Languages Hindi and English License Apache 2.0 — commercial use allowed
Example Prompts
Hindi Question Answering:
User: भारत की राजधानी क्या है?
Model: भारत की राजधानी नई दिल्ली है।Hindi Instructions:
User: मुझे चाय बनाने का तरीका बताओ।
Model: चाय बनाने के लिए पहले पानी गरम करें...Mixed Language:
User: Python में for loop कैसे लिखते हैं?
Model: Python में for loop इस तरह लिखते हैं...Compatibility
Tool Status llama.cpp Full support LM Studio Full support Ollama Full support Jan Full support Open WebUI Full support
Recommended Settings
Temperature: 0.7
Top-P: 0.9
Top-K: 40
Repeat Penalty: 1.1
Context Length: 2048Why This Model?
Tiny — 370MB, one of the smallest Hindi-capable GGUF models available
Fast — runs fully on CPU, no GPU required
Hindi-first — specifically trained for Hindi instruction following
Open — Apache 2.0, free for personal and commercial use
Actively maintained — v2 coming soon with more data
Roadmap
Done: v1 — Base Hindi fine-tune on Qwen3-0.6B
Next: v2 — 10x larger dataset, improved Hindi fluency
Next: v3 — better instruction following
Next: Qwen3-1.7B-Hindi — bigger model, same niche
Next: Live demo Space on HuggingFace
Related Repos
Repo Description pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1 Full precision model in safetensors format pankajpandey-dev/Qwen3-0.6B-Hindi-Instruct-v1-GGUF This repo — GGUF quantized versions
About the Author
Made by pankajpandey-dev
Building open-source Hindi AI models for India
Follow for weekly model updates and new Hindi LLM releases.
Found this useful? Please like this repo — it helps other Hindi speakers find it.
