Views
No views yet
| File | Quant | Size | Use Case |
|---|---|---|---|
Qwen3-4B-Q8_0.gguf | Q8_0 | ~4.5GB | Maximum quality |
Qwen3-4B-Q6_K.gguf | Q6_K | ~3.5GB | Near-lossless |
Qwen3-4B-Q5_K_M.gguf | Q5_K_M | ~3.1GB | High quality |
Qwen3-4B-Q4_K_M.gguf | Q4_K_M | ~2.6GB | Recommended default |
Qwen3-4B-Q3_K_M.gguf | Q3_K_M | ~2.1GB | Low VRAM |
Qwen3-4B-IQ4_XS.gguf | IQ4_XS | ~2.4GB | Imatrix 4-bit |
Qwen3-4B-IQ3_XXS.gguf | IQ3_XXS | ~1.8GB | Imatrix 3-bit |
Qwen3-4B-IQ2_M.gguf | IQ2_M | ~1.6GB | Imatrix 2-bit |
Qwen3-4B-IQ1_S.gguf | IQ1_S | ~1.2GB | Extreme compression |
Qwen3-4B-fp16.gguf | FP16 | ~8.0GB | Full precision |
imatrix.dat | — | — | Importance matrix |
1# llama.cpp
2./llama-cli -m Qwen3-4B-Q4_K_M.gguf \
3 --ctx-size 8192 -n 512 \
4 -p "<|im_start|>system\nYou are a helpful assistant.<|im_end|>\n<|im_start|>user\nHello!<|im_end|>\n<|im_start|>assistant\n"
5
6# Ollama
7ollama run hf.co/DuoNeural/Qwen3-4B-GGUF:Q4_K_M| Platform | Link |
|---|---|
| HuggingFace | huggingface.co/DuoNeural |
| Website | duoneural.com |
| GitHub | github.com/DuoNeural |
| X / Twitter | @DuoNeural |
| duoneural@proton.me | |
| Newsletter | duoneural.beehiiv.com |
| Support | buymeacoffee.com/duoneural |