# Clone the fork with Qwen3-Omni support
git clone https://github.com/phnxsystms/llama.cpp.git
cd llama.cpp
git checkout qwen3omni

# Build
mkdir build && cd build
cmake .. -DGGML_CUDA=ON
cmake --build . -j

# Run text inference
./bin/llama-cli -m qwen3-omni-30B-Q8_0.gguf -p "Hello!" -ngl 99

# Run multimodal inference  
./bin/llama-mtmd-cli -m qwen3-omni-30B-Q8_0.gguf --mmproj mmproj-qwen3-omni-30B-F16-fixed.gguf --image your_image.jpg -p "Describe this image"

Model Info

Base model: Qwen3-Omni-30B-A3B-Instruct
Architecture: MoE-based multimodal (48 layers, 128 experts)
Capabilities: Text + Vision
Tested: Distributed inference on 5-GPU cluster (41-44 tok/s)

llama.cpp Fork

The Qwen3-Omni architecture support is available at:

Repository: https://github.com/phnxsystms/llama.cpp
Branch: qwen3omni

License

Apache 2.0 (same as original Qwen3-Omni model)

phnxsystms/Qwen3-Omni-30B-A3B-Instruct-GGUF

Qwen3-Omni-30B-A3B-Instruct GGUF

Files

Usage

Model Info

llama.cpp Fork

License

phnxsystms/
Qwen3-Omni-30B-A3B-Instruct-GGUF