NVIDIA-Nemotron-Nano-12B-v2-AWQ

Model Highlights

Datasets:

• TeichAI/gemini-3-pro-preview-high-reasoning-1000x

Base Model:

• nvidia/NVIDIA-Nemotron-Nano-12B-v2

Use with VLLM

1. Download models at first by using hf
   hf download nicklas373/NVIDIA-Nemotron-Nano-12B-v2-AWQ
2. Copy hash for snapshots directory, then use it for chat templates and tool call parser ex: /home/xxx/.cache/huggingface/hub/models--nicklas373--NVIDIA-Nemotron-Nano-12B-v2-AWQ/snapshots/HASH_CODE/xxx
3. Run models with this command
   shell

   1vllm serve nicklas373/NVIDIA-Nemotron-Nano-12B-v2-AWQ \ 2 --chat-template '/home/xxx/.cache/huggingface/hub/models--nicklas373--NVIDIA-Nemotron-Nano-12B-v2-AWQ/snapshots/HASH_CODE/chat_template.jinja' \ 3 --chat-template-content-format string \ 4 --disable-fastapi-docs \ 5 --dtype auto \ 6 --enable-auto-tool-choice \ 7 --mamba_ssm_cache_dtype float32 \ 8 --quantization compressed-tensors \ 9 --served-model-name NVIDIA-Nemotron-Nano-12B-v2-AWQ \ 10 --seed 0 \ 11 --tool-call-parser 'nemotron_json' \ 12 --tool-parser-plugin '/home/xxx/.cache/huggingface/hub/models--nicklas373--NVIDIA-Nemotron-Nano-12B-v2-AWQ/snapshots/HASH_CODE/nemotron_toolcall_parser_streaming.py' \ 13 --tokenizer 'nvidia/NVIDIA-Nemotron-Nano-12B-v2' \ 14 --trust-remote-code