llama.cpp builds of Bharat-Tiny-LLM — India's first native edge AI for Hinglish & Hindi. These run cross-platform: Android, Raspberry Pi, CPU, and GPU via llama.cpp / llama-cpp-python.

Built by eulogik

Files

File	Format	Size	Use
`bharat-tiny-llm-q4_k_m.gguf`	GGUF Q4_K_M	~1.06 GB	Recommended — best size/quality for edge
`bharat-tiny-llm-f16.gguf`	GGUF f16	~3.55 GB	Full precision, for re-quantizing

Quick start

pip install llama-cpp-python

python

from llama_cpp import Llama

llm = Llama(model_path="bharat-tiny-llm-q4_k_m.gguf", n_ctx=1024)
print(llm.create_chat_completion(
    messages=[{"role": "user", "content": "Chai peete hain?"}],
    temperature=0.3, top_p=0.85, max_tokens=256, repeat_penalty=1.25,
)["choices"][0]["message"]["content"])

⚠️ Generation config matters. The base Qwen2.5-1.5B emits garbled out-of-script tokens at high temperature. Always use temperature ≈ 0.3 + repeat_penalty ≥ 1.25.

Other builds

Build	Repo	Size
MLX 4-bit (Apple Silicon)	`eulogik/Bharat-Tiny-LLM`	~880 MB
PyTorch fp16 (server / fine-tune)	`eulogik/Bharat-Tiny-LLM-fused`	~3.3 GB

License

Apache-2.0 (base Qwen2.5-1.5B weights Apache-2.0; LoRA adapter Apache-2.0).

eulogik/Bharat-Tiny-LLM-GGUF

Bharat-Tiny-LLM (GGUF)

Files

Quick start

Other builds

Links

License

eulogik/
Bharat-Tiny-LLM-GGUF