Views
No views yet
bosmet/angaadi-base-v2 for local llama.cpp / Ollama deployment.| Quant | Size | Notes |
|---|---|---|
f16 | 3.45 GB | Full precision (no quality loss). Use as the gold reference. |
q8_0 | 1.83 GB | 8-bit, <1% quality loss. Best when VRAM allows. |
q5_k_m | 1.26 GB | 5-bit balanced. ~1-2% quality loss; a good default. |
q4_k_m | 1.11 GB | 4-bit production default. ~3-5% loss; runs anywhere. |
1ollama pull hf.co/bosmet/angaadi-base-v2-gguf:f16
2
3cat > Modelfile <<EOF
4FROM hf.co/bosmet/angaadi-base-v2-gguf:f16
5SYSTEM "You are Angaadi's customer assistant. Answer concisely using the facts available."
6PARAMETER temperature 0
7PARAMETER num_ctx 1024
8EOF
9ollama create angaadi-qa -f Modelfile
10ollama run angaadi-qa "tea"