Qwen3.5 MoE models are natively multimodal (VLM). Their HuggingFace checkpoints use Qwen3_5MoeForConditionalGeneration with weights prefixed as model.language_model.*. This repo provides the language model backbone only, with:

architectures: ["Qwen3_5MoeForCausalLM"]
model_type: "qwen3_5_moe_text"
Weight keys at model.layers.* (standard causal LM format, no language_model. prefix)
Vision encoder and MTP weights removed

Model structure

Architecture: Hybrid GatedDeltaNet + Full Attention, Sparse Mixture-of-Experts
Total parameters: ~35B (3B active per token)
Dtype: bfloat16

How to use with vLLM

python

from vllm import LLM
llm = LLM(model="codecho/Qwen3.5-35B-A3B-text-only", trust_remote_code=True, tensor_parallel_size=2)