You can deploy this model and start earning money today!

liodon-ai/
Vikhr-Nemo-12B-Instruct-R-21-09-24-imatrix-GGUF

Name: Vikhr-Nemo-12B-Instruct-R-21-09-24-imatrix-GGUF
Availability: InStock

mistralai/Mistral-Nemo-Instruct-2407

text-generation

Vikhrmodels/Vikhr-Nemo-12B-Instruct-R-21-09-24

Views

No views yet

Vikhr-Nemo-12B-Instruct-R-21-09-24 — iMatrix GGUF

GGUF quantizations of Vikhrmodels/Vikhr-Nemo-12B-Instruct-R-21-09-24, published by Liodon AI.

Quick Start

llama.cpp

llama-cli -hf liodon-ai/Vikhr-Nemo-12B-Instruct-R-21-09-24-imatrix-GGUF:Q4_K_M

Ollama

ollama run hf.co/liodon-ai/Vikhr-Nemo-12B-Instruct-R-21-09-24-imatrix-GGUF:Q4_K_M

LM Studio / Jan — search liodon-ai/Vikhr-Nemo-12B-Instruct-R-21-09-24-imatrix-GGUF and pick your quant.

Quants

Quant	Size	VRAM est.	Notes
`IQ2_M`	4.44 GB	~5 GB	2-bit, iMatrix — smallest usable
`IQ3_M`	5.72 GB	~7 GB	3-bit, iMatrix — great quality/size tradeoff
`IQ4_XS`	6.74 GB	~8 GB	4-bit extra-small, iMatrix
`Q4_K_M`	7.48 GB	~9 GB	4-bit, iMatrix-calibrated (recommended)
`Q5_K_M`	8.73 GB	~10 GB	5-bit, iMatrix-calibrated
`Q6_K`	10.06 GB	~12 GB	6-bit, iMatrix-calibrated, near-lossless
`Q8_0`	13.02 GB	~15 GB	8-bit, essentially lossless

What is iMatrix?

Standard quantization treats all weights equally. iMatrix runs 128 calibration chunks through the full-precision model to find which weights matter most, then allocates more precision where it counts. At Q2/Q3/Q4 this means noticeably better coherence and instruction-following — same file size, better output.

Calibration: 2M tokens of WikiText-103.

Also see plain (non-iMatrix) quants: liodon-ai/Vikhr-Nemo-12B-Instruct-R-21-09-24-GGUF

Source

Model: Vikhrmodels/Vikhr-Nemo-12B-Instruct-R-21-09-24
License: other

Quantized by Liodon AI