Views
No views yet
| Filename | Quant Type | Size | Description |
|---|---|---|---|
NousCoder-14B.q2_k.gguf | Q2_K | 5.36 GB | Smallest size, lowest quality. Best for testing if model works. |
NousCoder-14B.q3_k_l.gguf | Q3_K_L (Large) | 7.36 GB | Better quality 3-bit quantization. |
NousCoder-14B.q3_k_m.gguf | Q3_K_M (Medium) | 6.82 GB | Small size with slightly better quality than Q3_K_S. |
NousCoder-14B.q3_k_s.gguf | Q3_K_S (Small) | 6.20 GB | Very small, low quality. Not recommended for most uses. |
NousCoder-14B.q4_0.gguf | Q4_0 | 7.93 GB | Basic 4-bit quantization. Good balance of size and quality. |
NousCoder-14B.q4_1.gguf | Q4_1 | 8.74 GB | 4-bit with higher accuracy than Q4_0. |
NousCoder-14B.q4_k_m.gguf | Q4_K_M (Medium) | 8.38 GB | K-quant 4-bit, best balance of size and quality. Most popular choice. |
NousCoder-14B.q4_k_s.gguf | Q4_K_S (Small) | 7.98 GB | K-quant 4-bit, optimized for smaller size. |
NousCoder-14B.q5_0.gguf | Q5_0 | 9.56 GB | Basic 5-bit quantization. Higher quality than Q4. |
NousCoder-14B.q5_1.gguf | Q5_1 | 10.37 GB | 5-bit with higher accuracy than Q5_0. |
NousCoder-14B.q5_k_m.gguf | Q5_K_M (Medium) | 9.79 GB | K-quant 5-bit, excellent quality-to-size ratio. |
NousCoder-14B.q5_k_s.gguf | Q5_K_S (Small) | 9.56 GB | K-quant 5-bit, good quality with reasonable size. |
NousCoder-14B.q6_k.gguf | Q6_K | 11.29 GB | 6-bit quantization. Very high quality, larger size. |
NousCoder-14B.q8_0.gguf | Q8_0 | 14.62 GB | 8-bit quantization. Near-original quality, largest quantized size. |
1# With llama.cpp
2./llama-cli -m NousCoder-14B.q4_k_m.gguf -p "Your prompt" -n 128
