Views
No views yet
llama.cpp1# sample images can be found in images folder
2
3# fp16
4./llava-cli -m ggml-model-f16.gguf --mmproj mmproj-model-f16.gguf --image example_2.png -c 4096 -e \
5 -p "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: <image>\nWhy is the image funny? ASSISTANT:" \
6 --temp 0.0
7
8# int4
9./llava-cli -m ggml-model-Q4_K_M.gguf --mmproj mmproj-model-f16.gguf --image example_2.png -c 4096 -e \
10 -p "A chat between a curious user and an artificial intelligence assistant. The assistant gives helpful, detailed, and polite answers to the user's questions. USER: <image>\nWhy is the image funny? ASSISTANT:" \
11 --temp 0.01# sample images can be found in images folder
2
3# fp16
4ollama create Bunny-Llama-3-8B-V-fp16 -f ./ollama-f16
5ollama run Bunny-Llama-3-8B-V-fp16 'example_2.png
6Why is the image funny?'
7
8# int4
9ollama create Bunny-Llama-3-8B-V-int4 -f ./ollama-Q4_K_M
10ollama run Bunny-Llama-3-8B-V-int4 'example_2.png
11Why is the image funny?'