The upstream Qwen/Qwen3.6-35B-A3B repository was used for config, tokenizer, chat template, and generation config only. Weight tensors come from the Q4_K_M GGUF and were dequantized in fp32, transformed back to HF/vLLM layout where llama.cpp stores Qwen3.5/3.6 Gated DeltaNet tensors differently, then cast once to BF16.

Validated on vLLM 0.23.0 with:

bash

VLLM_USE_FLASHINFER_SAMPLER=0 vllm serve . \
  --served-model-name qwen36-gguf-q4km-bf16-gdnfix \
  --dtype bfloat16 \
  --max-model-len 4096 \
  --moe-backend triton \
  --attention-backend triton_attn \
  --gdn-prefill-backend triton \
  --trust-remote-code \
  --language-model-only

Smoke test output for The capital of France is begins with Paris.

exolabs/Qwen3.6-35B-A3B-Q4KM-dequant-bf16-vllm

Qwen3.6-35B-A3B Q4_K_M GGUF dequantized BF16 for vLLM

exolabs/
Qwen3.6-35B-A3B-Q4KM-dequant-bf16-vllm