IBM Granite 3.3 2B Instruct — a dense transformer, quantized to Q4_K_M with a Q6_K output head for fast CPU inference on IBM Power (POWER9 VSX, POWER10/11 MMA-accelerated) via LibrePower. No GPU. Size: 1.5G. Apache-2.0, cryptographically signed by IBM.

Run it

Ubuntu / Debian ppc64le:

curl -fsSL https://linux.librepower.org/install.sh | sudo sh
sudo apt install librepower-llama
wget https://huggingface.co/librepowerai/Granite-3.3-2B-Instruct-Power/resolve/main/Granite-3.3-2B-Instruct-Q4_K_M.gguf
lp-llama-completion -m Granite-3.3-2B-Instruct-Q4_K_M.gguf -p "Hello!" -n 64 -t $(nproc)

IBM AIX 7.3 (big-endian):

dnf install llama-aix
wget https://huggingface.co/librepowerai/Granite-3.3-2B-Instruct-Power/resolve/main/Granite-3.3-2B-Instruct-Q4_K_M-be.gguf
lp-llama-completion -m Granite-3.3-2B-Instruct-Q4_K_M-be.gguf -p "Hello!" -n 64 -t $(nproc)

Good for

IBM's own dense small model: enterprise chat, RAG, structured JSON, classification/routing — Apache-2.0, signed

Credits

Base model: IBM Granite (Apache-2.0). Quantization & Power packaging: LibrePower.