File	Quantization	Size	BPW	Description
MiniCPM-V-4.6-F16.gguf	F16	1.5GB	16.0	Full precision
MiniCPM-V-4.6-Q4_K_M.gguf	Q4_K_M	812MB	8.5	4-bit, medium
MiniCPM-V-4.6-Q4_K_S.gguf	Q4_K_S	812MB	8.5	4-bit, small
MiniCPM-V-4.6-Q3_K_M.gguf	Q3_K_M	812MB	8.5	3-bit, medium
MiniCPM-V-4.6-Q3_K_S.gguf	Q3_K_S	812MB	8.5	3-bit, small
MiniCPM-V-4.6-Q2_K.gguf	Q2_K	812MB	8.5	2-bit, smallest

Model Info

Architecture: MiniCPMV4_6ForConditionalGeneration
Vision Encoder: SigLIP2-400M (27 layers, hidden_size=1152)
Language Model: Qwen3.5-0.8B (24 layers, hidden_size=1024)
Total Parameters: 1.3B
Context Length: 262K tokens
License: Apache-2.0

Usage

llama.cpp

llama-server -m MiniCPM-V-4.6-Q2_K.gguf --port 8080

Python

python

from llama_cpp import Llama

llm = Llama(model_path="./MiniCPM-V-4.6-Q2_K.gguf", n_ctx=512, n_threads=4)
output = llm.create_chat_completion(
    messages=[{"role": "user", "content": "你好，你是谁？"}],
    max_tokens=100,
    temperature=0.0,
)
print(output['choices'][0]['message']['content'])

Ollama

FROM ./MiniCPM-V-4.6-Q2_K.gguf

bash

ollama create minicpm-v-4.6 -f Modelfile
ollama run minicpm-v-4.6

Quantization Guide

Type	Bits	Quality	Speed	Use Case
F16	16	Best	Slow	Development
Q4_K_M	4	Good	Fast	General use
Q3_K_M	3	Medium	Faster	Balanced
Q2_K	2	Lower	Fastest	Edge/mobile

Conversion

bash

python convert_hf_to_gguf.py openbmb/MiniCPM-V-4.6 --outfile MiniCPM-V-4.6-F16.gguf --outtype f16
llama-quantize MiniCPM-V-4.6-F16.gguf MiniCPM-V-4.6-Q2_K.gguf Q2_K

License

Apache-2.0

llaa33219/MiniCPM-V-4.6-GGUF

MiniCPM-V-4.6 GGUF

Available Variants

Model Info

Usage

llama.cpp

Python

Ollama

Quantization Guide

Conversion

License

llaa33219/
MiniCPM-V-4.6-GGUF