CUDA_VISIBLE_DEVICES=0,1 vllm serve /path/to/gemma-4-70b-a7b-coder \
  --served-model-name vllm/doobee \
  --host 0.0.0.0 \
  --port 23333 \
  --dtype bfloat16 \
  --tensor-parallel-size 2 \
  --enable-expert-parallel \
  --max-model-len 200000 \
  --gpu-memory-utilization 0.96 \
  --trust-remote-code \
  --reasoning-parser gemma4 \
  --tool-call-parser gemma4 \
  --enable-auto-tool-choice \
  --default-chat-template-kwargs '{"enable_thinking": true}' \
  --generation-config vllm \
  --language-model-only \
  --skip-mm-profiling \
  --max-num-seqs 1 \
  --max-num-batched-tokens 8192

For clients that should not receive reasoning text, send "include_reasoning": false in chat-completion requests.

files

config.json
generation_config.json
tokenizer.json
tokenizer_config.json
chat_template.jinja
model.safetensors.index.json
MXFP4/BF16 safetensor shards

license

This model is released under the Gemma license.

LLMWildling/gemma-4-70b-a7b-coder

gemma-4-70b-a7b-coder

model

serving

files

license

LLMWildling/
gemma-4-70b-a7b-coder