Views
No views yet
bosmet/angaadi-base-v5 for local llama.cpp / Ollama deployment.| Quant | Size | Notes |
|---|---|---|
f16 | 3.45 GB | Full precision (no quality loss). Use as the gold reference. |
q8_0 | 1.83 GB | 8-bit, <1% quality loss. Best when VRAM allows. |
q5_k_m | 1.26 GB | 5-bit balanced. ~1-2% quality loss; a good default. |
q4_k_m | 1.11 GB | 4-bit production default. ~3-5% loss; runs anywhere. |
1ollama pull hf.co/bosmet/angaadi-base-v5-gguf:f16
2
3# Optional: bake in the Angaadi system prompt + temp 0
4cat > Modelfile <<EOF
5FROM hf.co/bosmet/angaadi-base-v5-gguf:f16
6SYSTEM "You are Angaadi's customer assistant. Answer concisely using the facts available."
7PARAMETER temperature 0
8PARAMETER num_ctx 1024
9EOF
10ollama create angaadi-qa -f Modelfile
11ollama run angaadi-qa "tea"llama-server -m model.f16.gguf -c 1024 --port 8080