This GGUF was quantized from Mistral's official BF16 GGUF using llama.cpp (build 537). The BF16 weights were dequantized to F32 during quantization, then quantized to MXFP4 format.

MXFP4 (Microscaling FP4) uses block-wise quantization with shared exponents per block.

Files

File	Size	Description
`ministral-3-8b-instruct-2512-mxfp4.gguf`	~3.5 GB	MXFP4 quantized model weights
`mmproj-ministral-3-8b-instruct-2512-f16.gguf`	~693 MB	Vision projector (F16, unquantized)

Usage

llama.cpp

bash

# Server mode with OpenAI-compatible API
llama-server \
  -m ministral-3-8b-instruct-2512-mxfp4.gguf \
  --mmproj mmproj-ministral-3-8b-instruct-2512-f16.gguf \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8080

LM Studio

Download both files from this repository
Load the main GGUF file in LM Studio
Load the mmproj file for vision support
Set GPU offload layers to maximum

Architecture

Parameters: 8B (dense transformer)
Layers: 40
Hidden dimension: 4096
Attention heads: 32 (8 KV heads for GQA)
Vision encoder: 24-layer Pixtral ViT
Context: 262K (YaRN scaled)
Vocabulary: Mistral Tekken tokenizer

Hardware Requirements

Minimum: 6 GB VRAM for text-only
Recommended: 8 GB VRAM for full GPU offload

License

Apache 2.0 — same as the base model.

FreedomAISVR/Ministral-3-8B-Instruct-2512-MXFP4-GGUF

Ministral 3 8B Instruct-2512 — MXFP4 GGUF

About the Model

Quantization