Views
No views yet
llama-quantize Q4_K_M (4-bit k-quant, medium variant)convert_hf_to_gguf.py1llama-cli -m ibm-granite_granite-4.1-8b.q4_k_m.gguf -p "Hello" -n 200
2Ollama
3Create a Modelfile:
4FROM ./ibm-granite_granite-4.1-8b.q4_k_m.gguf
5Then:
6ollama create granite-4.1-8b -f Modelfile
7ollama run granite-4.1-8b
8Original Model
9For full model details, capabilities, benchmarks, and intended use, see the upstream card:
10https://huggingface.co/ibm-granite/granite-4.1-8b
11Quantization reduces precision and may slightly degrade output quality versus the bf16 original. For highest fidelity, use the f16 GGUF in this repo or the original Hugging Face weights.