Phi-4-mini-reasoning GGUF
GGUF conversion and Q4_K_M quantization of
Microsoft Phi-4-mini-reasoning for efficient local inference.
Original Model
Official model:
This repository contains community-created GGUF conversions of the
original model.
Please refer to the official model card for complete information
about the model, training, intended use, limitations, and license.
Files
| File | Format | Approx. Size | Use |
|---|
Phi-4-mini-reasoning-F16.gguf | F16 | ~7.2 GB | Higher precision |
Phi-4-mini-reasoning-Q4_K_M.gguf | Q4_K_M | ~2.4 GB | Recommended |
Recommended Version
Q4_K_M
Phi-4-mini-reasoning-Q4_K_M.gguf is recommended for most local
inference setups.
It provides a much smaller memory footprint than the F16 version while
maintaining a good balance between quality and efficiency.
F16
The F16 version is provided for users who have sufficient memory and
want to use a higher-precision GGUF representation.
Quantization
Conversion pipeline:
Microsoft Phi-4-mini-reasoning
→ F16 GGUF
→ Q4_K_M
The Q4_K_M file was generated using the quantization tools from
llama.cpp.
Quantization:
Q4_K_M
No importance matrix was used.
llama.cpp
Basic inference:
llama-cli -m Phi-4-mini-reasoning-Q4_K_M.ggufGPU offloading:
llama-cli -m Phi-4-mini-reasoning-Q4_K_M.gguf -ngl 99Run a local server:
llama-server -m Phi-4-mini-reasoning-Q4_K_M.gguf -ngl 99Compatible Software
The GGUF files can be used with software supporting the GGUF format,
including:
- llama.cpp
- llama-cpp-python
- LM Studio
- Jan
- Other GGUF-compatible inference engines
Context
Phi-4-mini-reasoning is designed for reasoning-focused language
generation.
For model-specific capabilities and limitations, refer to the official
Microsoft model card.
License
The original model is provided under its respective Microsoft license.
Please review the official model repository before using or
redistributing the model:
Disclaimer
This is a community GGUF conversion and quantization.
This repository is not an official Microsoft repository.
Conversion Environment
Conversion and quantization were performed using llama.cpp.
Conversion:
Quantization:
Quantization type:
