Mixed precision mainly IQ4_NL quantization of
Vortex5/Phoenix-X-26B-A4B. Precision was quanted up to Q6_K in 3 beginning and end layers, as well as global attn layers, resulting in 10/30 attn-related tensor groups being Q6_K.
IQ4_NL was chosen specifically for outlier handling. In my testing, even IQ4_XS does break MoE occasionally, unless you're building it from a QAT checkpoint.
Deliberately stepping away from mixed math/article/story/rp soup datasets, imatrix dataset is a random conversation 250000-token prune of Squish42/bluemoon-fandom-1-1-rp-cleaned.
My only contribution is compute. This is neither my merge nor my dataset. WYSIWYG. Have fun.
