This repo is being kept up for archival purposes for now. These quants were used during the development of the llama.cpp PR which added MSA support for MiniMax-M3. They are now outdated and it is recommended to use AesSedai or Unsloth's updated quant mixes. I will likely not be updating these further (even though they should be working still) as MiniMax-M3 dose not survive the 2-bit quant process very well. I would not recommend the use of these low bit quants for real world tasks.
I will keep these quants (and any more I create) up until the PR merges and the bigger players update there repos to match, then I will likely replace this repo with IK quants once ik_llama catches up.
These are up to date with the PR as of 7-10-2026.
Models:
IF16/IF32?
I am still running tests on this at the moment, but for now. The indexer tensors within the official MiniMax-M3 release are stored as both FP32 and BF16 tensors. Specifically the projection tensors (index_{q,k}_proj) are what use BF16. There is some conflicting information in the PR about what these values should be quantized too. The main description recommends F32, but an example llama-quantize command further down in the chain just uses F16.
(Once I get some proper perp testing done I will update this section with any differences I noted. I will leave both model's up regardless!)
I have done some testing via runpod on the IF16/32 issue. The repo now has a flat Q8 and Q6_K quant in it if anyone else wants to avoid generating them. I also generated a 32K context Q8 logit dump via the following command (which is also in the repo):
./llama-perplexity --flash-attn on -lv 4 --file /workspace/Models/wiki.test.raw --save-all-logits /workspace/Models/MiniMax-M3-Q8_0-IF32-32768ctx-wiki.test.raw.bin --batch-size 2048 --ubatch-size 2048 -c 32768 --fit on --no-mmap --model /workspace/Models/MiniMax-M3-Q8_0.gguf
(Sadly, runpod ate the command output by randomly crashing the SSH session, so I don't have the final PPL for the Q8 file as I lost the entire command output...)
Regardless, I went on to run a perplexity test on my UD-IQ2_M-IF32/16 quants and did find a slight difference between the two:
(Full outputs in the repo.)
IF16:
Mean KLD: 0.457160 ± 0.002882
IF32:
Mean KLD: 0.455649 ± 0.002860
The difference is within the error margin, but it is there. So while I can't say with certainty the exact importance of the F32 conversion, I think it is safe to say that it dose have an effect.
UD-IQ2_M_IF16/IF32:
These are clones of Unsloth's UD-IQ2_M mix, with FP16 and FP32 indexer tensors respectively. They behave fairly well compared to the original quant from Unsloth and are likely the highest 'quality' 2-bit quant you can get right now until they update their mix once PR #24098 lands or ik_llama implements it's own support and allows for better 2-bit SOTA quant types.
Mean KLD: 0.488235 ± 0.002577
IQ2_M_IF16:
This is a flat automatic IQ2_M quant directly from llama-quantize without any tensor overrides. I just made this one to compare against the UD quant.
NOT RECOMENDED FOR ACTUALL USE!
