Views
No views yet
llama.cpp, text-generation-webui, LM Studio, Ollama, and other compatible backend frameworks.| Filename | Quant Type | File Size | Description |
|---|---|---|---|
| BitCPM4-CANN-0.5B-F16.gguf | 16-bit | 870 MB | The unquantized base model weights in full precision. Maximum possible fidelity. |
| BitCPM4-CANN-0.5B-Q8_0.gguf | 8-bit | 463 MB | Near-perfect accuracy retention. Offers a massive size reduction while acting indistinguishably from the F16 version. |
| BitCPM4-CANN-0.5B-Q6_K.gguf | 6-bit | 358 MB | Excellent option for low-resource edge devices demanding strong logic retention. |
| BitCPM4-CANN-0.5B-Q5_K_M.gguf | 5-bit | 317 MB | Great middle-ground for balancing speed, size, and remaining reasoning capability. |
| BitCPM4-CANN-0.5B-Q5_K_S.gguf | 5-bit | 310 MB | Slightly more aggressive 5-bit compression format focused on minimizing footprint. |
| BitCPM4-CANN-0.5B-Q4_K_M.gguf | 4-bit | 279 MB | Recommended. The absolute sweet spot for local 4-bit execution, maintaining surprising coherence for its sub-300MB size. |
| BitCPM4-CANN-0.5B-Q4_K_S.gguf | 4-bit | 267 MB | Highly optimized for speed. Perfect for deeply embedded systems or background text processing. |
| BitCPM4-CANN-0.5B-Q3_K_M.gguf | 3-bit | 235 MB | Ultimate compression limit. Use exclusively under extremely severe hardware memory limitations. |
llama.cpp (Command Line)llama.cpp, you can run the model directly from your terminal. Replace the filename with the specific version you downloaded:1./llama-cli \
2 -m BitCPM4-CANN-0.5B-Q4_K_M.gguf \
3 -p "Explain the concept of artificial intelligence to a five-year-old." \
4 -n 256 \
5 -c 2048 \
6 --temp 0.7