This repository contains a BF16 GGUF conversion of JetBrains/Mellum2-12B-A2.5B-Thinking.

These GGUF files were created with the experimental Mellum2 implementation in adrianhoehne/llama.cpp. Use the default branch cached-experts-v2; it includes the Mellum2 support from add-mellum-2-support. These files are not compatible with upstream llama.cpp and are not expected to become compatible with upstream llama.cpp unless Mellum2 support is implemented there separately.

The original model is a JetBrains Mellum2 Mixture-of-Experts thinking model with 12B total parameters, 2.5B active parameters, 64 experts, and 8 active experts per token. It is post-trained for reasoning and emits reasoning inside <think>...</think> blocks before the final answer. See the original JetBrains model card for full architecture details, training notes, evaluations, and license information.

The Q6_K and Q4_K_M files were produced with llama.cpp's quantizer. Mellum2's expert down-projection tensors use 896 columns, so K-quant formats cannot be applied to those 28 tensors directly; llama.cpp applies its normal fallback quantization for them.

These files require the Mellum2 implementation from adrianhoehne/llama.cpp. Use the default branch cached-experts-v2, which includes add-mellum-2-support. Do Do not expect them to load with upstream llama.cpp.

Scores are pass@1. Both models were evaluated locally via llama.cpp with greedy decoding (temperature=0). Result is Mellum2 minus Qwen2.5-Coder. BigCodeBench was evaluated on the local Hard/Instruct subset with --execution local --no_gt.

The original model is released under the Apache 2.0 license. This GGUF conversion follows the same license.