Qwen3.5-122B-A10B-ROCMFP4-MTP-GGUF
The first ROCmFP4 MTP quantization of Qwen3.5-122B optimized for AMD Strix Halo (Ryzen AI 300 series) APUs.
Key Features
- Format: GGUF with integrated MTP (Multi-Token Prediction) head
- Quantization: ROCmFP4 (custom 4-bit quantization for AMD GPUs)
- Architecture: A10B (Adaptive 10/40 layer hybrid attention for 128K context)
- Context: 256K tokens
- MTP Acceptance: 92-95%
Benchmarks (Strix Halo 128GB, Aug 25 2026)
Tested with llama.cpp ROCmFPX b257, Vulkan backend, MTP enabled.
| Prompt Size | Prompt Speed | Gen Speed |
|---|
| 36 tokens | 91.5 t/s | 43.8 t/s |
| 71 tokens | 99.2 t/s | 39.3 t/s |
| 932 tokens | 379.6 t/s | 41.0 t/s |
- Generation: ~41-44 t/s (consistent across prompt sizes)
- Prompt processing: up to 380 t/s (large prompts saturate GPU bandwidth)
- MTP accept rate: 94%
- Model size: ~65 GB (5 shards)
Usage
llama-server \ --model Qwen3.5-122B-A10B-ROCMFP4-STRIX-00001-of-00002-00001-of-00005.gguf \ --port 8080 \ -c 262144 \ --n-gpu-layers 99 \ --spec-type draft-mtp \ --spec-draft-n-max 2 \ --spec-draft-n-min 1 \ --spec-draft-p-min 0.75 \ --spec-draft-type-k q8_0 \ --spec-draft-type-v q8_0Notes
- Built with llama.cpp b10615+ (ROCmFPX b257)
- Requires AMD ROCm 6.2+ or Vulkan backend
- MTP head is bundled in the GGUF (no separate draft model needed)
- Optimized for 128GB+ VRAM systems
- Hybrid attention (10/40 full layers) enables 256K context in 128GB VRAM
License
Apache 2.0
Generated by Tom Yoder | August 2026
