Views
No views yet
| File | Description |
|---|---|
granite-docling-258M-bf16.gguf (317 MB) | LLM weights in BF16 |
mmproj-model-f16.gguf (182 MB) | Vision projector (mmproj) in F16 |
1# Build llama.cpp (if not already built)
2cd llama.cpp && make -j
3
4# Run with multimodal (image + text) inference
5./llama-mtmd-cli \
6 -m granite-docling-258M-bf16.gguf \
7 --mmproj mmproj-model-f16.gguf \
8 --chat-template chatml1# Describe an image
2echo "Describe this image in detail." | ./llama-mtmd-cli \
3 -m granite-docling-258M-bf16.gguf \
4 --mmproj mmproj-model-f16.gguf \
5 -p /path/to/image.png
6
7# Text-only inference (no vision)
8echo "What is the capital of France?" | ./llama-mtmd-cli \
9 -m granite-docling-258M-bf16.gguf \
10 -p - --chat-template chatmlRockMan256/granite-docling-258M (original weights, not remote)ibm-granite/granite-docling-258M (vision encoder/connector unchanged from base)