You can run this model locally with full vision support using Ollama.
1. Download the files
Keep both .gguf files and the Modelfile in the same directory.
2. Create the model
Open your terminal in that folder and run:
ollama create qwopus9b-vision -f Modelfile
3. Run and Analyze
ollama run qwopus9b-vision
Note: To analyze an image, simply provide the local path to the image file in your prompt (e.g., /path/to/screenshot.png Describe the errors in this image).
💡 Technical Recommendations
Minimum VRAM: 12 GB.
Recommended VRAM: 16 GB (especially for contexts above 32k tokens).
KV Cache Optimization: To handle extremely long contexts (>64k tokens) on 16GB cards, consider setting the environment variable OLLAMA_KV_CACHE_TYPE=q8_0.
🛡️ About the "Abliterated" Version
This model has undergone an "abliteration" process to remove built-in refusal mechanisms. It will follow instructions more directly and is less likely to refuse requests due to safety filters, making it highly effective for specialized tasks like security auditing or uncensored creative writing.
🙏 Credits
Special thanks to huihui-ai for the original fine-tuning and abliteration, and the Unsloth team for their state-of-the-art quantization tools.
