Views
No views yet
auto-round tool. For better compatibility and smaller size, we provide unified multimodal projector (mmproj) files in F16, BF16, and F32 formats.| File Name | Quant Type | Size | Description |
|---|---|---|---|
gemma-4-E4B-it-Q2_K_S.gguf | Q2_K_S | 4.0 GB | Extremely high compression, significant quality loss. |
gemma-4-E4B-it-Q2_K_MIXED.gguf | Q2_K_MIXED | 5.6 GB | Recommended high-compression option. Uses Q4 for KV cache with good quality. |
gemma-4-E4B-it-Q3_K_S.gguf | Q3_K_S | 4.3 GB | Very high compression, notable quality loss. |
gemma-4-E4B-it-Q3_K_M.gguf | Q3_K_M | 4.3 GB | Balanced 3-bit quantization. |
gemma-4-E4B-it-Q3_K_L.gguf | Q3_K_L | 4.3 GB | High quality 3-bit quantization. |
gemma-4-E4B-it-Q4_0.gguf | Q4_0 | 4.8 GB | Standard 4-bit quantization, good balance. |
gemma-4-E4B-it-Q4_1.gguf | Q4_1 | 5.1 GB | Higher quality 4-bit quantization than Q4_0. |
gemma-4-E4B-it-Q4_K_S.gguf | Q4_K_S | 4.8 GB | Small 4-bit K-quant, good efficiency. |
gemma-4-E4B-it-Q4_K_M.gguf | Q4_K_M | 4.8 GB | Recommended 4-bit K-quant, excellent balance. |
| `gemma-4-E4B-it-Q5_0.gguf" | Q5_0 | 5.3 GB | Standard 5-bit quantization, very high quality. |
gemma-4-E4B-it-Q5_1.gguf | Q5_1 | 5.5 GB | Higher quality 5-bit quantization than Q5_0. |
gemma-4-E4B-it-Q5_K_S.gguf | Q5_K_S | 5.3 GB | Small 5-bit K-quant, very high quality. |
gemma-4-E4B-it-Q5_K_M.gguf | Q5_K_M | 5.3 GB | Recommended 5-bit K-quant, near-lossless. |
gemma-4-E4B-it-Q6_K.gguf | Q6_K | 5.8 GB | 6-bit K-quant, virtually indistinguishable from F16. |
gemma-4-E4B-it-Q8_0.gguf | Q8_0 | 7.5 GB | 8-bit quantization, near-lossless. |
mmproj-model-f16.gguf | F16 | 945 MB | Unified Projector in Float16 format. |
mmproj-model-bf16.gguf | BF16 | 946 MB | Unified Projector in BFloat16 format. |
mmproj-model-f32.gguf | F32 | 1.8 GB | Unified Projector in Float32 format. |
auto-round --model google/gemma-4-E4B-it --output_dir ./quantized/ --scheme <SCHEME> --iters 0llama.cpp. For multimodal usage, you must specify the projector file:./llama-cli -m gemma-4-E4B-it-Q4_K_M.gguf --mmproj mmproj-model-f16.gguf --image your_image.jpg -p "Describe this image."