You can deploy this model and start earning money today!

samgreen/
Qwen2.5-VL-72B-Instruct-GGUF

Name: Qwen2.5-VL-72B-Instruct-GGUF
Availability: InStock

quantized

Qwen/Qwen2.5-VL-72B-Instruct

Views

No views yet

Qwen2.5-VL-72B-Instruct

Converted and quantized using HimariO's fork using this procedure. No IMatrix.

The fork is currently required to run inference and there's no guarantee these checkpoints will work with future builds. Temporary builds are available here. The latest tested build as of writing is qwen25-vl-b4899-bc4163b.

Edit:

As of 1-April-2025 inference support has been added to koboldcpp.

Original model

Usage

./llama-qwen2vl-cli -m Qwen2.5-VL-72B-Instruct-Q4_K_M.gguf --mmproj qwen2.5-vl-72b-instruct-vision-f16.gguf -p "Please describe this image." --image ./image.jpg

samgreen/Qwen2.5-VL-72B-Instruct-GGUF

Qwen2.5-VL-72B-Instruct

Usage

samgreen/
Qwen2.5-VL-72B-Instruct-GGUF