Native llama.cpp conversions of NVIDIA and Unsloth Qwen3.6 35B A3B NVFP4 checkpoints with their bundled MTP blocks preserved. The newest artifact comes from unsloth/Qwen3.6-35B-A3B-NVFP4-Fast.

Files

File	Source	Layout	Size
`qwen3.6-35b-a3b-unsloth-nvfp4-fast-mtp-gguf.gguf`	Unsloth NVFP4 Fast	All expert layers native NVFP4, bundled MTP	21.33 GiB
`qwen3.6-35b-a3b-unsloth-nvfp4-mtp-gguf.gguf`	Unsloth NVFP4	Native NVFP4 expert FFNs, source FP8 tensors stored as Q8_0, bundled MTP	24.34 GiB
`qwen3.6-35b-a3b-nvfp4-mtp.gguf`	NVIDIA NVFP4	Native NVFP4 with bundled MTP	22.21 GiB

Fast GGUF SHA256: ABE31A8082E44E5C03D0F08595715584AEADEA8C2A76C6F9F066A4ECBA64E78A.

The Fast source stores every expert layer as NVFP4. The earlier Unsloth source uses a mixed compressed-tensors layout; its conversion keeps packed NVFP4 expert tensors native and stores source FP8 weights as Q8_0. Both expose their MTP layer to recent llama.cpp builds through draft-mtp.

RTX 5090 Result

Windows 11, RTX 5090, 200k context, q4_0 target/draft KV, draft-mtp n=2, no-thinking, identical BookContext fixtures, and one measured 1,024-token completion per point. These are llama.cpp decode-only rates after the separately reported prompt prefill.

GGUF	Runtime	10k decode	10k prefill	200k decode	200k prefill	VRAM after
Unsloth Fast	b10068	135.23 tok/s	2.16s	84.08 tok/s	52.84s	25,138 MiB
Unsloth mixed	b9851	151.16 tok/s	2.37s	93.20 tok/s	50.41s	26.7 GiB
NVIDIA source	b9851	150.94 tok/s	2.39s	90.24 tok/s	51.35s	24.6 GiB

The Fast artifact is about 3.0 GiB smaller than the earlier mixed Unsloth conversion and leaves roughly 7 GiB free with the tested 200k server. However, Unsloth's upstream Fast claim is based on vLLM concurrency throughput. That advantage did not transfer to this native Windows llama.cpp, single-request test: the earlier mixed conversion remains faster at both prompt lengths.

Benchmark CSV: benchmarks/qwen36-35b-a3b-unsloth-nvfp4-fast-mtp-gguf-benchmark-20260719.csv.

Quick Start - Fast

bat

llama-server \
  --model qwen3.6-35b-a3b-unsloth-nvfp4-fast-mtp-gguf.gguf \
  --alias qwen36-35b-a3b-unsloth-nvfp4-fast-mtp-gguf \
  --host 0.0.0.0 \
  --port 39202 \
  --device CUDA0 \
  --gpu-layers all \
  --gpu-layers-draft all \
  --ctx-size 200000 \
  --cache-type-k q4_0 \
  --cache-type-v q4_0 \
  --cache-type-k-draft q4_0 \
  --cache-type-v-draft q4_0 \
  --flash-attn on \
  --parallel 1 \
  --cont-batching \
  --jinja \
  --reasoning off \
  --spec-type draft-mtp \
  --spec-draft-n-max 2 \
  --spec-draft-p-min 0.0

Companion Scripts

Portable Windows source download, conversion, official llama.cpp b10068 runtime installation, Hermes setup, launcher, and benchmark scripts are in neko-legends/nvidia-local-llm-profiles. Models are stored relative to the checkout parent rather than in a machine-specific drive path.

Apache 2.0, inherited from the source releases.

neko-legends/Qwen3.6-35B-A3B-NVFP4-MTP-GGUF

Qwen3.6 35B A3B NVFP4 MTP GGUFs

Files

RTX 5090 Result

Quick Start - Fast

Companion Scripts

neko-legends/
Qwen3.6-35B-A3B-NVFP4-MTP-GGUF