Views
No views yet
| Format | Description |
|---|---|
| F16 | Full precision (16-bit), better quality, larger size ⚖️ |
| Q8_K_XL | Quantized (8-bit XL variant, based on the quantization table of the unsloth model Qwen3-4B-Thinking-2507), smaller size, faster inference ⚡ |
| Q4_K_XL | Quantized (4-bit XL variant, based on the quantization table of the unsloth model Qwen3-4B-Thinking-2507), smaller size, faster inference ⚡ |
llama.cpp:./main -m ./gguf-file-name.gguf -p "Hello world!"