# Download the model and mmproj
wget https://huggingface.co/nawta/Heron-NVILA-Lite-2B-Q5_K_M-GGUF/resolve/main/Heron-NVILA-Lite-2B-Q5_K_M.gguf
wget https://huggingface.co/nawta/Heron-NVILA-Lite-2B-mmproj-GGUF/resolve/main/mmproj-heron-nvila-lite-2b-f16.gguf

# Run inference
./llama-mtmd-cli \
  -m Heron-NVILA-Lite-2B-Q5_K_M.gguf \
  --mmproj mmproj-heron-nvila-lite-2b-f16.gguf \
  --image your_image.jpg \
  -p "この画像について説明してください。"

Memory Requirements

Original Model: <a href="https://huggingface.co/turing-motors/Heron-NVILA-Lite-2B-hf" target="_blank" rel="noopener noreferrer" class="text-blue-400 hover:text-blue-300 underline underline-offset-2 transition-colors">turing-motors/Heron-NVILA-Lite-2B-hf
Quantization: Q5_K_M
Format: GGUF (for llama.cpp)
File Size: 1.2 GB
Languages: Japanese, English

Quantization	LLM	mmproj	Total
F16	3.3 GB	807 MB	~4.1 GB
Q8_0	1.8 GB	807 MB	~2.6 GB
Q6_K	1.4 GB	807 MB	~2.2 GB
Q5_K_M	1.2 GB	807 MB	~2.0 GB
Q4_K_M	1.1 GB	807 MB	~1.9 GB
Q3_K_M	0.9 GB	807 MB	~1.7 GB
Q2_K	0.7 GB	807 MB	~1.5 GB

License

This model inherits the license from the original Heron-NVILA-Lite-2B model. Please refer to turing-motors/Heron-NVILA-Lite-2B-hf for license details.

Acknowledgments

Original model by Turing Motors
GGUF conversion and quantization by nawta