UI-Venus-1.5-30B-A3B-AWQ

Quantization recipe

• Tool: llm-compressor (vLLM project)
• Method: AWQModifier (duo_scaling=False) + QuantizationModifier (scheme="W4A16")
• Ignored (kept in full precision): lm_head, the vision tower (*visual*), and the MoE router/gate (*mlp.gate) -- quantizing the router breaks MoE routing; quantizing the (comparatively small) vision tower has a poor accuracy/size tradeoff for a grounding-critical model, consistent with the proven QuantTrio/Qwen3-VL-30B-A3B-Instruct-AWQ (AutoAWQ) and cyankiwi/Qwen3-VL-32B-Instruct-AWQ-4bit (llm-compressor) reference builds for the same base architecture family.
• MoE handling: Qwen3VLMoeTextExperts stores weights as fused 3D tensors (gate_up_proj/down_proj). llm-compressor's generic MoE linearization (FusedExpertsProtocol / load_context()) converts these to per-expert 2D nn.Linear modules before AWQ runs, then reconverts on save.
• Calibration dataset: HuggingFaceM4/FineVision, aguvis-stage-1 config -- real desktop/web screenshots paired with click/double-click/right-click/drag coordinate-grounding instructions (e.g. "Click on 'Events'" -> click(x=0.43, y=0.88)), a much closer domain match for a UI-grounding model than a generic image-captioning calibration set. 128 samples, streamed (not bulk-downloaded) and shuffled from the ~459k-row split.
• Images were resized (aspect-preserving) to a ~768-vision-token budget before calibration to keep sequences well under the 2048-token calibration max_seq_length without truncating instruction text.
• Loading: device_map=None (CPU-resident) + llm-compressor's sequential onloading -- the 60GB FP16 model doesn't fit in 48GB VRAM, so layers are onloaded to GPU one at a time for calibration and offloaded afterward.
• moe_calibrate_all_experts=True (llm-compressor default): all 128 experts see all calibration tokens, not just the top-8 a given token would route to at inference time -- needed for reliable per-expert AWQ scales given how sparse top-8-of-128 routing is against a modest calibration sample.

Serving

Intended use