GGUF quantizations of NVIDIA's Nemotron-3-Nano 4B Instruct model, packaged by DuoNeural for local inference with llama.cpp, Ollama, LM Studio, and compatible runtimes.

Available Quantizations

File	Size	Quality	Recommended Use
`nemotron-3-nano-4b-Q4_K_M.gguf`	~2.5 GB	Good	General use, best balance
`nemotron-3-nano-4b-Q5_K_M.gguf`	~3.0 GB	Better	Higher quality, more RAM
`nemotron-3-nano-4b-Q8_0.gguf`	~4.3 GB	Best	Near-lossless, 8+ GB RAM
`nemotron-3-nano-4b-f16.gguf`	~8.0 GB	Reference	Full precision, 10+ GB RAM

Usage

Ollama

ollama run hf.co/DuoNeural/NVIDIA-Nemotron-3-Nano-4B-GGUF:Q4_K_M

llama.cpp

./llama-cli -m nemotron-3-nano-4b-Q4_K_M.gguf -p "Your prompt here" -n 512

LM Studio

Search DuoNeural/NVIDIA-Nemotron-3-Nano-4B-GGUF in the model browser and select your preferred quantization.

About the Base Model

NVIDIA's Nemotron-3-Nano 4B Instruct is a compact, efficient instruction-following model from NVIDIA's Nemotron family. Built on the NemotronH architecture, it delivers strong reasoning and chat performance at the 4B parameter scale. Suitable for local deployment on consumer hardware.

Notes

Quantized from the official NVIDIA release using llama.cpp's convert_hf_to_gguf.py
All quantizations validated for correct output format
F16 is the reference quantization; Q4_K_M is recommended for most users

DuoNeural

DuoNeural is an open AI research lab — human + AI in collaboration.

Platform	Link
HuggingFace	huggingface.co/DuoNeural
Website	duoneural.com
GitHub	github.com/DuoNeural
X / Twitter	@DuoNeural
Email	duoneural@proton.me
Newsletter	duoneural.beehiiv.com
Support	buymeacoffee.com/duoneural

DuoNeural Research Publications

Title	DOI
Nano-CTM: Ternary Continuous Thought Machines with Thought-Space Self-Prediction for Efficient Iterative Reasoning	10.5281/zenodo.19775622
Recurrence as World Model: CTM Learns Implicit Belief States in Partially Observable Physical Environments	10.5281/zenodo.19810620
Per-Object Slot Decomposition for Scalable Neural World Modeling: When Does Attention Beat Mean-Field?	10.5281/zenodo.19846804

Open access, CC BY 4.0. Authored by Archon, Jesse Caldwell, Aura — DuoNeural.

Research Team

Jesse — Vision, hardware, direction
Archon — Lab Director, post-training, abliteration, experiments
Aura — Research AI, literature synthesis, novel proposals

Subscribe to the lab newsletter at duoneural.beehiiv.com for model drops before they go anywhere else.