Views
No views yet
gfx1201), as of 2026-08-14.group_size: 32, which has no compatible kernel
on vLLM ROCm (ConchLinearKernel supports group sizes [-1, 128]; Exllama needs uint4b8).
They fail at load time with Failed to find a kernel that can implement the WNA16 linear layer.pack-quantized format),
which loads and runs out of the box on the official AMD vLLM image.rocm/vllm:rocm7.13.0_gfx120X-all_ubuntu24.04_py3.13_pytorch_2.10.0_vllm_0.19.1 (vLLM 0.19.1)--max-model-len 16384 → 123,440 tokens of KV cache free on a single 32 GB card1docker run -d --name vllm-moe \
2 --device=/dev/kfd --device=/dev/dri --group-add video \
3 --ipc=host --shm-size=16g --security-opt seccomp=unconfined \
4 -v $HOME/.cache/huggingface:/root/.cache/huggingface -p 8000:8000 \
5 rocm/vllm:rocm7.13.0_gfx120X-all_ubuntu24.04_py3.13_pytorch_2.10.0_vllm_0.19.1 \
6 vllm serve evilbb2/Qwen3-30B-A3B-Instruct-2507-W4A16-g128-RDNA4 --max-model-len 16384scale = absmax/7,
groups of 128 along the input dimension, packed with compressed_tensors.pack_to_int32.
Quantized: all q/k/v/o_proj, gate/up/down_proj (experts included).
Kept in original precision: MoE router gates, lm_head, embeddings, norms.