Qwen3-30B-A3B-GGUF

Original Model

Run with LlamaEdge

• LlamaEdge version:
  • Thinking: v0.17.0 and above
  • No Thinking: v0.18.2
• Prompt template
  • Prompt type: chatml (for thinking)
  • Prompt string
    text

    1<|im_start|>system 2{system_message}<|im_end|> 3<|im_start|>user 4{prompt}<|im_end|> 5<|im_start|>assistant
  • Prompt type: qwen3-no-think (for no thinking)
  • Prompt string
    text

    1<|im_start|>system 2{system_message}<|im_end|> 3<|im_start|>user 4{user_message_1}<|im_end|> 5<|im_start|>assistant 6{assistant_message_1}<|im_end|> 7<|im_start|>user 8{user_message_2}<|im_end|> 9<|im_start|>assistant 10<think> 11 12</think>
• Context size: 128000
• Run as LlamaEdge service
  bash

  1wasmedge --dir .:. --nn-preload default:GGML:AUTO:Qwen3-30B-A3B-Q5_K_M.gguf \ 2 llama-api-server.wasm \ 3 --model-name Qwen3-30B-A3B \ 4 --prompt-template chatml \ 5 --ctx-size 128000