This GGUF was quantized from the FP8_E4M3 source weights using llama.cpp (build 537). The source safetensors were dequantized to F16 during conversion, then quantized to NVFP4 format.

NVFP4 (Microscaling FP4) uses block-wise quantization with shared exponents per block, providing better precision than standard FP4 for the same memory footprint.

Files

File	Size	Description
`ministral-3-14b-instruct-2512-nvfp4.gguf`	~6.9 GB	NVFP4 quantized model weights
`mmproj-ministral-3-14b-instruct-2512-f16.gguf`	~878 MB	Vision projector (F16, unquantized)

Usage

llama.cpp

bash

# Server mode with OpenAI-compatible API
llama-server \
  -m ministral-3-14b-instruct-2512-nvfp4.gguf \
  --mmproj mmproj-ministral-3-14b-instruct-2512-f16.gguf \
  -ngl 99 \
  --host 0.0.0.0 \
  --port 8080

# Direct inference
llama-cli \
  -m ministral-3-14b-instruct-2512-nvfp4.gguf \
  --mmproj mmproj-ministral-3-14b-instruct-2512-f16.gguf \
  -ngl 99 \
  -p "Write a Python function to compute fibonacci numbers"

LM Studio

Download both files from this repository
Load the main GGUF file in LM Studio
Load the mmproj file for vision support
Set GPU offload layers to maximum

Architecture

Parameters: 14B (dense transformer)
Layers: 40
Hidden dimension: 5120
Attention heads: 32 (8 KV heads for GQA)
Vision encoder: 24-layer Pixtral ViT
Context: 131K (native), 393K (extended)
Vocabulary: Mistral Tekken tokenizer

Hardware Requirements

Minimum: 8 GB VRAM for text-only, 10 GB for vision
Recommended: 16 GB VRAM for full GPU offload
Disk: ~7.8 GB for model + mmproj

Quantization Details

Metric	Value
Source format	FP8_E4M3 (safetensors)
Intermediate	F16 GGUF
Output format	NVFP4
Approximate BPW	~4.6
Quantized with	llama.cpp build 537

License

Apache 2.0 — same as the base model.

FreedomAISVR/Ministral-3-14B-Instruct-2512-NVFP4-GGUF

Ministral 3 14B Instruct-2512 — NVFP4 GGUF

About the Model

Quantization