Build	Repo	Size	Use
MLX 4-bit (edge / Apple Silicon)	`eulogik/Bharat-Tiny-LLM`	~880 MB	Recommended for Mac / on-device
GGUF Q4_K_M (llama.cpp, Android / Pi / CPU)	`eulogik/Bharat-Tiny-LLM-GGUF`	~1.06 GB	Cross-platform, llama.cpp
PyTorch fp16 (this repo)	`eulogik/Bharat-Tiny-LLM-fused`	~3.3 GB	Server / fine-tuning base

Quick start (transformers)

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")
tokenizer = AutoTokenizer.from_pretrained("eulogik/Bharat-Tiny-LLM-fused")

messages = [{"role": "user", "content": "Chai peete hain?"}]
prompt = tokenizer.apply_chat_template(messages, add_generation_prompt=True)
inputs = tokenizer(prompt, return_tensors="pt")
out = model.generate(
    **inputs,
    max_new_tokens=256,
    temperature=0.3,
    top_p=0.85,
    repetition_penalty=1.25,
    no_repeat_ngram_size=3,
    do_sample=True,
)
print(tokenizer.decode(out[0][inputs.input_ids.shape[1]:], skip_special_tokens=True))

⚠️ Generation config matters. The base Qwen2.5-1.5B emits garbled out-of-script tokens at high temperature. Always use temperature ≈ 0.3 + repetition_penalty ≥ 1.25 + no_repeat_ngram_size = 3. The bharat-tiny-llm PyPI package applies these for you.

License

Apache-2.0 (base Qwen2.5-1.5B weights Apache-2.0; LoRA adapter Apache-2.0).

eulogik/Bharat-Tiny-LLM-fused

Bharat-Tiny-LLM (fused · fp16)

For most users

Quick start (transformers)

Links

License

eulogik/
Bharat-Tiny-LLM-fused