This is a 3-bit quantized GGUF of Microsoft's Phi-4-mini-instruct 3.8B parameters Small Language Model.
Description
The weights are quantized with llama.cpp to a 3-bit quantization (Q3_K_M). The Screen capture below shows the simple CLI interface, which runs in Linux Terminal locally on a computer with Intel CORE i5 CPU without internet.
Executing GGUF
Simply download the "Phi-4-mini-instruct-GGUF-Q3_K_M.gguf" file or clone the repository
To run this Language Model in a simple CLI interface, provide the directory path to "llama-cli" and "Phi-4-mini-instruct-GGUF-Q3_K_M.gguf" in the Terminal.
(Path to llama CLI)/llama.cpp/build/bin/llama-cli --color --conversation --model (Path to model GGUF file)/Phi-4-mini-instruct-GGUF-Q3_K_M.gguf
