Views
No views yet
llama.cpp's
convert_hf_to_gguf.py and llama-quantize.| File | Quant | Notes |
|---|---|---|
Gazal-R1-32B-GRPO-preview-BF16.gguf | BF16 | Full-precision master GGUF, source for the quantized files below |
Gazal-R1-32B-GRPO-preview-Q8_0.gguf | Q8_0 (8-bit) | Near-lossless, ~half the size of BF16 |
Gazal-R1-32B-GRPO-preview-Q4_K_M.gguf | Q4_K_M (4-bit) | Smallest, some quality loss vs full precision |
./llama-cli -m Gazal-R1-32B-GRPO-preview-Q4_K_M.gguf -p "Your prompt here"