llama-server \
  -hf ggml-org/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-GGUF:Q4_K_M \
  -hfd apolo13x/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-DFlash-GGUF \
  --spec-type draft-dflash \
  -ngl all \
  -ngld all \
  -fa on \
  --temp 1.0 \
  --top-p 0.95

NVIDIA recommends temperature 1.0 and top-p 0.95.

This DFlash GGUF was obtained with llama.cpp b10373 by running:

bash

python3 convert_hf_to_gguf.py \
  dflash-hf \
  --target-model-dir target-meta \
  --outtype bf16 \
  --outfile dflash-NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4.gguf