Mellum2-12B-A2.5B-Thinking is a reasoning-oriented code generation model developed by JetBrains and optimized for software engineering workflows, code completion, code generation, code understanding, and structured reasoning tasks. This repository contains GGUF quantized variants of the model optimized for efficient local inference using llama.cpp.

The model is designed to assist developers with programming-related tasks while incorporating enhanced reasoning capabilities for code analysis, problem solving, and technical workflows. The quantized formats significantly reduce memory requirements while preserving strong coding and reasoning performance, enabling practical deployment across consumer hardware and local development environments.

Model Overview

Model Name: Mellum2-12B-A2.5B-Thinking
Base Model: JetBrains/Mellum2-12B-A2.5B-Thinking
Architecture: Mixture-of-Experts (MoE) Transformer
Parameter Count: 12 Billion Total Parameters / ~2.5 Billion Active Parameters
Modalities: Text
Primary Languages: English
Developer: JetBrains
License: Apache 2.0

Quantization Formats

This repository provides various GGUF quantized versions of the Mellum2-12B-A2.5B-Thinking model, optimized for efficient local inference using llama.cpp. Below are the details of the available quantization formats.

IQ3_M

Size reduction of approx 75.71% (5.50 GB) compared to 16-bit (22.64 GB)
Aggressive 3-bit quantization optimized for maximum memory efficiency
Suitable for low-memory deployment environments and CPU-based inference
Enables practical execution of coding and reasoning workloads on constrained hardware
Complex code generation, repository-level reasoning, and advanced analytical tasks may experience reduced output fidelity compared to higher-precision formats

IQ4_NL

Size reduction of approx 71.25% (6.51 GB) compared to 16-bit (22.64 GB)
Advanced 4-bit non-linear quantization designed to better preserve coding capability and reasoning quality
Better suited for software engineering workflows, code analysis, and structured problem-solving tasks
Designed to reduce quantization loss while maintaining generation quality
May require slightly higher computational overhead during inference

IQ4_XS

Size reduction of approx 72.31% (6.27 GB) compared to 16-bit (22.64 GB)
Balanced 4-bit quantization focused on efficient inference and dependable coding performance
Provides a practical balance between memory efficiency, generation quality, and runtime speed
Suitable for code completion, code explanation, debugging assistance, and technical workflows
Maintains stable performance across most practical software engineering workloads

Training Background (Original Model)

Mellum2-12B-A2.5B-Thinking is trained with an emphasis on software engineering, code understanding, technical reasoning, and developer productivity workflows.

Pretraining

Large-scale training across programming languages, software repositories, and technical documentation
Focus on code understanding, code generation, and software engineering tasks
Optimized for downstream coding and reasoning workloads

Post-Training

Enhanced through reasoning-oriented and instruction-following training
Improved for structured problem-solving and code analysis tasks
Optimized for practical developer workflows and technical assistance

Key Capabilities

Code Generation Supports generation of code across multiple programming languages and development workflows.
Code Understanding Assists with explaining, analyzing, and understanding existing codebases.
Technical Reasoning Performs structured reasoning for debugging, implementation planning, and software design tasks.
Developer Assistance Supports coding productivity through code completion, refactoring assistance, and technical guidance.
Efficient Local Deployment Quantized variants enable practical local inference on consumer hardware.

Usage Example

Using llama.cpp

bash

./llama-cli \
  -m SandlogicTechnologies/Mellum2-12B-A2.5B-Thinking_IQ4_NL.gguf \
  -p "Implement a Python function that performs binary search and explain the time complexity."

Recommended Usecases

Code Generation Workflows Generate functions, scripts, and application components across multiple programming languages.
Code Review and Analysis Assist with understanding, reviewing, and improving existing code.
Developer Productivity Tools Support IDE integrations, local coding assistants, and software engineering workflows.
Technical Reasoning Tasks Help with debugging, architecture discussions, and implementation planning.
Research and Experimentation Evaluate local coding models and reasoning-focused software engineering workflows.

Acknowledgments

These quantized models are based on the original work by the JetBrains development team.

Special thanks to:

The JetBrains team for developing and releasing the Mellum2-12B-A2.5B-Thinking model.
Georgi Gerganov and the llama.cpp open-source community for enabling efficient quantization and inference via the GGUF format.

Contact

For questions, feedback, or support, please reach out at support@sandlogic.com or visit https://www.sandlogic.com/

SandLogicTechnologies/Mellum2-12B-A2.5B-Thinking-GGUF

Mellum2-12B-A2.5B-Thinking