[!TIP] KV-cache quantization without any fork (recommended, 2026): upstream llama.cpp/Ollama now cover this natively — use -ctk q8_0 -ctv q8_0 (~half KV memory, negligible quality loss: perplexity +0.002–0.05) or -ctk q4_0 -ctv q4_0 (~quarter memory, ≈7.6% perplexity increase). In Ollama: OLLAMA_KV_CACHE_TYPE=q8_0 with OLLAMA_FLASH_ATTENTION=1. Keep K and V types symmetric to stay on the fast fused Flash-Attention path. Since April 2026, mainline llama.cpp also applies Hadamard rotation to KV activations (PR #21038), which greatly improves low-bit KV quality (opt-out: LLAMA_ATTN_ROT_DISABLE=1).

The RotorQuant/TurboQuant fork flow below is experimental/legacy: the TurboQuant llama.cpp PR was closed without merging (June 2026) and the fork is unmaintained relative to mainline. It is NOT required to use this model.

MiniMax-M2.7-TurboQuant-MLX-3bit

MLX 3-bit quantized variant of MiniMaxAI/MiniMax-M2.7 with the legacy TurboQuant KV-cache fork (superseded by upstream llama.cpp KV options), optimized for Apple Silicon.

Overview

MiniMax-M2.7 is a massive 256-expert Mixture-of-Experts (MoE) model with 8 experts active per token, totaling approximately 456 billion parameters. This variant combines 3-bit MLX weight quantization with TurboQuant KV-cache quantization for deployment on Apple Silicon hardware.

TurboQuant uses asymmetric per-channel quantization on the KV cache, optimized for throughput and long-context generation. At 3-bit, quality degradation becomes more noticeable -- consider RotorQuant if quality is paramount.

Property	Value
Architecture	MoE (256 experts, 8 active/token)
Total Parameters	~456B
Layers	62
Hidden Size	3072
Attention Heads	48
Weight Quantization	3-bit (MLX)
KV-Cache Quantization	TurboQuant
Estimated Size	~170 GB
Base Model	MiniMaxAI/MiniMax-M2.7

Quickstart

pip install mlx-lm

python

from mlx_lm import load, generate

model, tokenizer = load("majentik/MiniMax-M2.7-TurboQuant-MLX-3bit")

prompt = "What is a Comprehensive Geriatric Assessment?"
messages = [{"role": "user", "content": prompt}]
text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)

response = generate(
    model,
    tokenizer,
    prompt=text,
    max_tokens=512,
)
print(response)

TurboQuant vs RotorQuant

Feature	TurboQuant	RotorQuant
Technique	Asymmetric per-channel KV quantization	Rotation-based KV quantization (Hadamard transform)
Throughput	Higher throughput, lower latency	Slightly lower throughput
Quality	Good quality preservation	Better quality preservation at low bit-widths
Best For	High-throughput serving, long contexts	Quality-sensitive tasks, research

Memory Estimates (Apple Silicon)

Variant	Estimated Size	Minimum Unified Memory
MLX 8-bit	~456 GB	512 GB (Mac Studio M2/M3/M4 Ultra)
MLX 5-bit	~280 GB	384 GB
MLX 4-bit	~225 GB	256 GB
MLX 3-bit	~170 GB	192 GB
MLX 2-bit	~110 GB	128 GB

Note: 3-bit quantization requires Apple Silicon with 192 GB+ unified memory, such as a Mac Studio with M2/M3/M4 Ultra.

Quant trade-off (MLX lane)

Bits	Approx size	Use case	Recommendation
2-bit	~119 GB	Aggressive quantization	Very low-RAM Macs
3-bit	~164 GB	Lossy but small	Low-RAM Macs
4-bit	~192 GB	Balanced default	Recommended for most Macs
5-bit	~228 GB	Higher fidelity	Quality-sensitive
6-bit	~274 GB	Approaching FP16 quality	High-fidelity
8-bit	~347 GB	Near-lossless reference	Fidelity-critical work

(Current variant — 3bit — is bolded.)

Variants in this family

(Showing 12 sibling variants under majentik/minimax-m2.7-*. The current variant — TurboQuant-MLX-3bit — is bolded.)

Variant	Runtime	Approx size	Use case
TurboQuant-MLX-3bit	mlx-lm	~1.2 GB	Apple Silicon, small
TurboQuant-MLX-4bit	mlx-lm	~1.7 GB	Apple Silicon balanced
TurboQuant-MLX-5bit	mlx-lm	~2.1 GB	Apple Silicon, higher fidelity

About the RotorQuant / TurboQuant labels

RotorQuant and TurboQuant are this project's release labels, not distinct quantization algorithms — for any given tier, both brand repos carry byte-identical weights produced with the standard MLX / llama.cpp quantizers. No brand-specific speedup is claimed or measured.

majentik/MiniMax-M2.7-TurboQuant-MLX-3bit