llama-server \
  -m Qwen3.5-9bxOpus-4.7-sonic-GGUF.Q4_K_M.gguf \
  --port 8001 --alias qwen3.5-9b-opus \
  -c 65536 -n 8192 \
  --temp 0.6 --top-p 0.95 --top-k 40 --repeat-penalty 1.05 \
  --flash-attn on --ctk q8_0 --ctv q8_0 \
  --jinja --chat-template-kwargs '{"enable_thinking": true}' \
  -ngl -1

Recomment configure

Using mmproj is more like to Multi-Token Prediction(MTP) -- reason for support text, image, video, audio and pdf, is an advanced training and inference technique for memory and cache. I recomment to use --ctk q8_0 --ctv q8_0 for better cache and load.

base

llama-server \
  -m Qwen3.5-9bxOpus-4.7-sonic-GGUF.Q4_K_M.gguf \
  --port 53281 \
  -c 131072 \
  --parallel 1 \
  --flash-attn on \
  --no-context-shift \
  -ngl -1 \
  --jinja \
  --chat-template-kwargs "{\"enable_thinking\": true}" \
  --mmproj Qwen3.5-9bxOpus-4.7-sonic-GGUF.BF16-mmproj.gguf

Preset Configs

Mode	Flags
Coding	`--temp 0.4 --top-p 0.9 --top-k 50 --repeat-penalty 1.1`
Reasoning	`--temp 0.6 --top-p 0.95 --top-k 40`
Low VRAM	`-c 32768 -n 4096 --flash-attn off -ngl 20`

Key Specs

Base: Qwen3.5 9B · Format: Q4_K_M GGUF
Context: 32K–64K practical · Output: up to 8K tokens
KV cache: 8-bit (q8_0) · GPU: full offload (-ngl -1)
Thinking mode: optional — improves reasoning, adds latency

Caveats

Quality drops beyond 64K context
9B class — sensitive to sampling parameters
Thinking mode can produce unstable output if misaligned with training