Note: This conversion includes the text backbone only (language model weights). Vision encoder and multimodal projector weights are excluded, as llama.cpp does not yet support the phi4-siglip vision architecture. The text model is architecturally identical to Phi-4-reasoning-plus (Phi3ForCausalLM).

Available Files

Filename	Quant Type	Size	Description
`phi-4-reasoning-vision-f16.gguf`	F16	~28 GB	Full precision (float16)
`phi-4-reasoning-vision-q8_0.gguf`	Q8_0	~15 GB	8-bit quantization (near-lossless)
`phi-4-reasoning-vision-q6_k.gguf`	Q6_K	~12 GB	6-bit K-quant
`phi-4-reasoning-vision-q5_k_m.gguf`	Q5_K_M	~9.9 GB	5-bit K-quant medium
`phi-4-reasoning-vision-q5_k_s.gguf`	Q5_K_S	~9.5 GB	5-bit K-quant small
`phi-4-reasoning-vision-q4_K_M.gguf`	Q4_K_M	~8.5 GB	4-bit K-quant medium (recommended)
`phi-4-reasoning-vision-q4_k_s.gguf`	Q4_K_S	~7.9 GB	4-bit K-quant small
`phi-4-reasoning-vision-q3_k_l.gguf`	Q3_K_L	~7.4 GB	3-bit K-quant large
`phi-4-reasoning-vision-q3_k_m.gguf`	Q3_K_M	~6.9 GB	3-bit K-quant medium
`phi-4-reasoning-vision-q3_k_s.gguf`	Q3_K_S	~6.1 GB	3-bit K-quant small
`phi-4-reasoning-vision-q2_k.gguf`	Q2_K	~5.2 GB	2-bit K-quant (smallest, lowest quality)

How to Use

With Ollama

bash

# Download the Q4_K_M GGUF and create a Modelfile:
cat > Modelfile <<'EOF'
FROM ./phi-4-reasoning-vision-q4_K_M.gguf

TEMPLATE """<|system|>
{{ if .System }}{{ .System }}{{ else }}You are a helpful AI assistant with vision capabilities. You can analyze images and reason about them step by step.{{ end }}<|end|>
<|user|>
{{ .Prompt }}<|end|>
<|assistant|>
"""

PARAMETER stop "<|end|>"
PARAMETER stop "<|endoftext|>"
PARAMETER temperature 0.7
PARAMETER top_p 0.9
PARAMETER num_ctx 4096
EOF

ollama create phi4-vision -f Modelfile
ollama run phi4-vision

With llama.cpp

./llama-cli -m phi-4-reasoning-vision-q4_K_M.gguf -p "Explain the theory of relativity in simple terms." -n 512

Model Details

Original Model: microsoft/Phi-4-reasoning-vision-15B
Architecture: Phi3ForCausalLM (text backbone of Phi-4-reasoning-vision)
Parameters: ~15B (text model)
Hidden Size: 5120
Layers: 40
Attention Heads: 40 (10 KV heads, GQA)
Vocab Size: 100,352
Tokenizer: GPT-2 (BPE)
Context Length: Up to 131,072 tokens (with RoPE scaling)
License: MIT

Conversion Details

Converted using llama.cpp convert_hf_to_gguf.py
Vision tower (model.vision_tower.*) and multimodal projector (model.mm_projector.*) weights were skipped during conversion
The model config was remapped from Phi4ForCausalLMV (phi4-siglip) to Phi3ForCausalLM (phi3) since the text backbone is architecturally identical
Quantization performed via llama_model_quantize() with CUDA acceleration
243 text tensors converted, 452 vision tensors excluded

Original Model Card

For full details on training, capabilities, safety, and intended use, please refer to the original model card.

Disclaimer

This is an unofficial GGUF conversion. The original model was created by Microsoft Research. All credit for the model architecture, training, and capabilities belongs to the Microsoft Phi team. Please refer to the original model's license for usage terms.

jamesburton/Phi-4-reasoning-vision-15B-GGUF

Phi-4-reasoning-vision-15B-GGUF