This private artifact was converted from unsloth/Qwen3.6-35B-A3B-GGUF file Qwen3.6-35B-A3B-MXFP4_MOE.gguf into standard Hugging Face safetensors for vLLM serving.

The source GGUF mixes MXFP4 MoE expert tensors with K-quant/Q8/F32 tensors. The converter streams GGUF tensors, dequantizes them to fp32, applies the Qwen3.6/GDN tensor remaps, performs the required RMSNorm offset handling, fuses MoE gate and up expert tensors into mlp.experts.gate_up_proj, then casts once to BF16 before writing sharded safetensors.

Validation

GGUF dry-run mapping: 733 input tensors, 0 skipped, 0 unmapped.
MXFP4 probe: blk.0.ffn_gate_exps.weight dequantized successfully to fp32 with shape (256, 512, 2048).
Output index: 693 HF keys, 21 safetensor shards, 69,321,221,376 bytes.
Transformers load: passed as AutoModelForImageTextToText with expected missing visual tower weights only.
Numeric spot-checks: passed with max_abs_vs_bf16=0 for sampled direct, norm-offset, reshape/GDN, K-quant expert, and MXFP4 fused expert tensors.
Transformers generation smoke: passed.
vLLM smoke: passed on vLLM 0.23.0 with --language-model-only.

vLLM command used

VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve /path/to/model   --served-model-name qwen36-35b-a3b-gguf-mxfp4_moe-bf16   --dtype bfloat16   --max-model-len 4096   --gpu-memory-utilization 0.85   --attention-backend triton_attn   --gdn-prefill-backend triton   --moe-backend triton   --trust-remote-code   --language-model-only

The visual tower files are intentionally absent because the GGUF source contains language weights only; use vLLM text serving with --language-model-only.

exolabs/Qwen3.6-35B-A3B-MXFP4_MOE-dequant-bf16-vllm

Qwen3.6-35B-A3B MXFP4_MOE GGUF Dequantized BF16 for vLLM

Validation

vLLM command used

exolabs/
Qwen3.6-35B-A3B-MXFP4_MOE-dequant-bf16-vllm