NVFP4 (4-bit) weight-only quantization of mPLUG/GUI-Owl-1.5-32B-Instruct, a Qwen3-VL–based vision GUI-agent model. This build is optimized for efficient serving on NVIDIA Blackwell hardware (e.g. GB10 / DGX Spark / RTX 50-series) with vLLM.

Base model: mPLUG/GUI-Owl-1.5-32B-Instruct (Qwen3VLForConditionalGeneration)
Quantization: NVFP4, compressed-tensors format (nvfp4-pack-quantized), weight-only (W4A16)
Scope: only the language-model Linear layers are quantized; the vision tower, multimodal merger, and lm_head are kept in their original precision so the image path and output head are unaffected
Method: llm-compressor data-free RTN (QuantizationModifier, scheme NVFP4A16) — no calibration dataset
Size: ~21 GB (down from ~63 GB BF16)

Serving with vLLM

Requires a vLLM build with compressed-tensors NVFP4 support (loads via MarlinNvFp4LinearKernel on Blackwell):

bash

vllm serve Hadidiz9/GUI-Owl-1.5-32B-Instruct-NVFP4 \
  --served-model-name gui-owl-32b \
  --max-model-len 32768 \
  --trust-remote-code \
  --kv-cache-dtype fp8 \
  --limit-mm-per-prompt '{"image":5}' \
  --allowed-local-media-path /

Notes

Context length: 32768 tokens.
The model consumes screenshots (images) as input for GUI-agent tasks; the vision preprocessor config is included.
Quantization was performed on an NVIDIA GB10.

License

Inherits the Apache-2.0 license of the base model mPLUG/GUI-Owl-1.5-32B-Instruct. All credit for the original model belongs to the mPLUG / GUI-Owl authors. This repository only redistributes a quantized copy of their weights.