Views
No views yet
unsloth/Qwen3.5-9B.
Full merged safetensors weights are available at Jakelolipopp/Qwen3.5-9B-AltText-v4-merged.mmproj) were converted and quantized using llama.cpp.| File Name | Size |
|---|---|
Qwen3.5-9B-AltText-v4-BF16.gguf | 16.69 GB |
Qwen3.5-9B-AltText-v4-Q2_K.gguf | 3.56 GB |
Qwen3.5-9B-AltText-v4-Q3_K_M.gguf | 4.31 GB |
Qwen3.5-9B-AltText-v4-Q4_K_M.gguf | 5.24 GB |
Qwen3.5-9B-AltText-v4-Q5_K_M.gguf | 6.02 GB |
Qwen3.5-9B-AltText-v4-Q6_K.gguf | 6.85 GB |
Qwen3.5-9B-AltText-v4-Q8_0.gguf | 8.87 GB |
mmproj-Qwen3.5-9B-AltText-v4-BF16.gguf | 0.86 GB |
mmproj-Qwen3.5-9B-AltText-v4-Q8_0.gguf | 0.58 GB |
Q8_0: Near-lossless quality, highest precision (recommended for maximum accuracy).Q6_K: High quality, virtually indistinguishable from full precision with noticeable size reduction.Q5_K_M: Great balance between quality and speed/VRAM usage (Recommended default).Q4_K_M: Fast inference, compact size, great for lower memory devices.Q3_K_M / Q2_K: Ultra-compressed for minimal resource footprints.mmproj)mmproj-...-BF16.gguf: Full bfloat16 vision projector for maximum visual fidelity.mmproj-...-Q8_0.gguf: 8-bit quantized vision projector with lower VRAM overhead.-m) and the vision projector (--mmproj):1# Run interactive CLI with an image
2./llama-cli \
3 -m Qwen3.5-9B-AltText-v4-Q5_K_M.gguf \
4 --mmproj mmproj-Qwen3.5-9B-AltText-v4-BF16.gguf \
5 --image /path/to/image.jpg \
6 -p "Generate detailed alternative text describing this image:" \
7 -c 4096 --temp 0.21./llama-server \
2 -m Qwen3.5-9B-AltText-v4-Q5_K_M.gguf \
3 --mmproj mmproj-Qwen3.5-9B-AltText-v4-BF16.gguf \
4 --port 8080 -c 4096Modelfile:1FROM ./Qwen3.5-9B-AltText-v4-Q5_K_M.gguf
2MMPROJ ./mmproj-Qwen3.5-9B-AltText-v4-BF16.gguf
3
4PARAMETER temperature 0.2
5PARAMETER top_p 0.951ollama create qwen3.5-9b-alttext-v4 -f Modelfile
2ollama run qwen3.5-9b-alttext-v4 "Describe this image: /path/to/image.png"