This is Q6_K GGUF quant with some extra params. I just thought - can I make such a cool quant Unsloth is doing? I took a look at tensors in their GGUF models and noticed differences in standard quants also took a look at llama-quantize options and noticed some cool options I can use to produce something simmilar. Made few qauants and vibe tested them on some tricky prompts to find configuration that pass them all successfuly while both ordinary Q6_K and UD-Q6_XL quants do not pass them. BTW, GGUF is very flexible format and this is what I also learned with this experiment. I thought only Exllama have flexibility but it is not true and with GGUF you also can optimise quants for your need. What is special here? All attention related layers are bf16, few first and last layers are Q8_0 and also all ffn_down_exps are Q8_0... so quite a lot Q8_0 layers. I would say it is Q7_K quant ;P Here is llama-quantize command used:

XXL GGUF quant is a joke. Nothing new here, I just used params llama-quantize already have. However I find out MoE models and especialy this one are highly sensitive to quantization. Gol was to have something smaller then Q8_0 but still performing exeptionally like if it was unquantized model... I think this quant is realy, realy good. Do you share my vibe tested view? :P Have fun with this model!