calcuis/

fastvlm-gguf

fastvlm-gguf

• run it with gguf-connector; simply execute the command below in console/terminal

GGUF file(s) available. Select which one to use:

1. fastvlm-0.5b-iq4_nl.gguf
2. fastvlm-0.5b-q4_0.gguf
3. fastvlm-0.5b-q8_0.gguf

Enter your choice (1 to 3): _

• opt a gguf file in your current directory to interact with; nothing else

for the latest update, you should be able to customize how many tokens for output (see picture)

connector f5 (alternative 1)

• note: f5 is different from f6 (above) and f9 (below); you don't need gguf files with f5

connector f7 (alternative 2)*

connector f9 - advanced mode (alternative 3)**

**for advanced mode, it's available to specify your text prompt with picture input (see above)

reference

• base model from apple
• gguf-connector (pypi)