Phi-4-mini-instruct GGUF
GGUF versions of Microsoft Phi-4-mini-instruct prepared
for efficient local inference.
This repository contains an F16 GGUF version and a Q4_K_M
quantized version.
Available Files
| File | Format | Approx. Size |
|---|
Phi-4-mini-instruct-F16.gguf | F16 | 7.2 GB |
Phi-4-mini-instruct-Q4_K_M.gguf | Q4_K_M | 2.4 GB |
Recommended Version
Phi-4-mini-instruct-Q4_K_M.gguf is recommended for most
consumer hardware and local inference setups.
It provides a substantially smaller memory footprint than F16
while retaining the advantages of the GGUF format.
Original Model
The original model is Microsoft's official Phi-4-mini-instruct.
Original model:
Please refer to Microsoft's original model repository for
technical details, intended use, limitations, and licensing.
Quantization
The Q4_K_M file was generated from the F16 GGUF model using
llama.cpp quantization tools.
Quantization format:
Q4_K_M
llama.cpp
Basic usage:
llama-cli -m Phi-4-mini-instruct-Q4_K_M.ggufGPU offloading:
llama-cli -m Phi-4-mini-instruct-Q4_K_M.gguf -ngl 99Local server:
llama-server -m Phi-4-mini-instruct-Q4_K_M.gguf -ngl 99Compatible Software
- llama.cpp
- llama-cpp-python
- LM Studio
- Jan
- Other GGUF-compatible runtimes
Model Variants
F16
Higher precision with substantially larger memory requirements.
Q4_K_M
Smaller model size and lower memory requirements, making it
more suitable for local deployment on consumer hardware.
Disclaimer
This is a community GGUF conversion and quantization.
It is not an official Microsoft repository.
License
Please follow the licensing terms of the original Microsoft
Phi-4-mini-instruct model.
SHA256
Checksums can be generated with:
