05/08/26: The CUDA flash attention branch has also been merged to master, please use the master branch and recompile!
05/07/26: The PR branch has been merged to master. All of the quants and imatrix have been updated with the newest conversion and are ready to re-download. These quants include MTP tensors for when that gets added upstream eventually.
05/05/26: I've updated all the quants to use the fused QKV conversion. The PR branch supports both fused + unfused so it's not necessary to download the new quants, but it may provide a small speed boost.
This is a text-only GGUF quantization of XiaomiMiMo/MiMo-V2.5. This means that image and audio input is not present in this GGUF, and will not be available until support is added upstream in llama.cpp.
This repo contains specialized MoE-quants for MiMo-V2.5. The idea being that given the huge size of the FFN tensors compared to the rest of the tensors in the model, it should be possible to achieve a better quality while keeping the overall size of the entire model smaller compared to a similar naive quantization. To that end, the quantization type default is kept in high quality and the FFN UP + FFN GATE tensors are quanted down along with the FFN DOWN tensors.
