This model is an unmodified GGUF quantization of unsloth/Qwen3.6-35B-A3B-NVFP4-Fast, made with the llama.cpp conversion tool. Please refer to unsloth repository for quality metrics. Your feedback, contributions, optimization proposals and questions are highly appreciated.

This GGUF is primarily intended to be run fully on the GPU without offloading to the CPU. However, you can absolutely run it with offloading. The relevant guides are provided in this repo.

I've tested this model on my setup (5070 Ti 16GB, 5060 Ti 16GB, 32GB DDR5). Token generation and prefill speeds will vary depending on your hardware. Please use the metrics below as a rough estimate of your potential performance.

This GGUF requires a recent build of llama.cpp. If the commands below raise a compatibility error, please update your llama.cpp version. All commands have been tested on llama.cpp version b9957 - ensure your version is >=b9957.

The token generation speeds listed below were achieved using varying prompt lengths, KV cache quantization, and the more powerful GPU 0. These optimizations simply let more computations to be done on 5070ti.