Filename	Size	Quant Type
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-f16.gguf	6.33 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-F16
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q2_k.gguf	1.20 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q2_K
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q3_k_l.gguf	1.61 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q3_K_L
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q3_k_m.gguf	1.53 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q3_K_M
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q3_k_s.gguf	1.44 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q3_K_S
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q4_0.gguf	1.84 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q4_0
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q4_1.gguf	2.02 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q4_1
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q4_k_m.gguf	1.92 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q4_K_M
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q4_k_s.gguf	1.85 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q4_K_S
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q5_0.gguf	2.21 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q5_0
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q5_1.gguf	2.39 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q5_1
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q5_k_m.gguf	2.25 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q5_K_M
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q5_k_s.gguf	2.21 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q5_K_S
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q6_k.gguf	2.60 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q6_K
granite-4.0-h-micro-base-DISTILL-4.5-opus-high-think-q8_0.gguf	3.37 GB	GRANITE-4.0-H-MICRO-BASE-DISTILL-4.5-OPUS-HIGH-THINK-Q8_0

Quick Start

Ollama

bash

# Use Q4_K_M (recommended)
ollama run hf.co/glogwa68/granite-4.0-h-micro-DISTILL-OPUS-4.5-think-GGUF:Q4_K_M

# Or other quantizations
ollama run hf.co/glogwa68/granite-4.0-h-micro-DISTILL-OPUS-4.5-think-GGUF:Q8_0
ollama run hf.co/glogwa68/granite-4.0-h-micro-DISTILL-OPUS-4.5-think-GGUF:Q2_K

llama.cpp

bash

# Download and run
llama-cli --hf-repo glogwa68/granite-4.0-h-micro-DISTILL-OPUS-4.5-think-GGUF --hf-file granite-4.0-h-micro-distill-opus-4.5-think-q4_k_m.gguf -p "Hello, how are you?"

# With server
llama-server --hf-repo glogwa68/granite-4.0-h-micro-DISTILL-OPUS-4.5-think-GGUF --hf-file granite-4.0-h-micro-distill-opus-4.5-think-q4_k_m.gguf -c 2048

LM Studio / GPT4All

Download the .gguf file of your choice and load it in your application.

Quantization Details

Type	Bits	Use Case
Q2_K	2	Extreme compression, low quality
Q3_K_M	3	Very compressed
Q4_K_M	4	Recommended - Best size/quality
Q5_K_M	5	High quality
Q6_K	6	Very high quality
Q8_0	8	Near lossless
F16	16	Original precision

Original Model

This is the quantized version of granite-4.0-h-micro-DISTILL-OPUS-4.5-think

Base Model: ibm-granite/granite-4.0-h-micro-base
Fine-tuning Dataset: TeichAI/claude-4.5-opus-high-reasoning-250x
Special Feature: Thinking/Reasoning with <think> tags