You can deploy this model and start earning money today!

SergiioB/
Nemotron-3.5-Lightning-30B-A3B-DFlash-BF16

Name: Nemotron-3.5-Lightning-30B-A3B-DFlash-BF16
Availability: InStock

nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DFlash

finetune

other

Views

No views yet

Nemotron-3.5-Lightning DFlash — local BF16 reconstruction (Intel XPU)

⚠️ Draft only. Not a standalone chat model. Pair with the GPTQ INT4 G64-sym target.

This is a local NVFP4 → BF16 reconstruction of nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4-DFlash so vLLM XPU can load the official DFlash speculator. NVIDIA ships the draft as ModelOpt NVFP4 (E2M1 + E4M3 scales). The pinned XPU image does not consume that packing for this draft.

Publisher: SergiioB.

📖 Cookbook — setup, patches, methodology, evidence

Repo: intel-arc-pro-b70-inference-cookbook
Nemotron speed card + capacity ladder: NEMOTRON-DFLASH-B70.md
Model + launch recipe: NEMOTRON-B70.md
Image + patch matrix: IMAGE-AND-PATCH-MATRIX.md
Copy-paste launch lines: FULL-SETUP-COMMANDS.md

Launch with the GPTQ target

bash

vllm serve /path/to/gptq-target \
  --quantization gptq --dtype float16 \
  --max-model-len 16384 --max-num-seqs 1 \
  --no-enable-prefix-caching --async-scheduling \
  --speculative-config '{"method":"dflash","model":"/path/to/this-draft","num_speculative_tokens":7}'

Set VLLM_XPU_ENABLE_XPU_GRAPH=1. On B70 also apply the cookbook patches (native grouped-topk v2 + SSU B8/W4).

Conversion

Attention, embeddings, and norms were already BF16 and were copied.
MLP + fc NVFP4 tensors were dequantized: E2M1 low-nibble-first × float8_e4m3fn × F32 weight_scale_2, group 16, linear (not swizzled). Matches vLLM nvfp4_emulation_utils decode.
quantization_config was removed from config.json. If a leftover hf_quant_config.json is present, rename it — vLLM will otherwise treat the draft as NVFP4 and fail on XPU.
19 matrices dequantized, 51 tensors copied. See CONVERSION.txt.

This is not a published official BF16 draft. It is a reconstruction.

Measured with the GPTQ target on Arc Pro B70

Real measurements from the publisher's card: isolated C1, n=5 medians, cache off, 150 W cap, DFlash n_spec=7. Commands and raw logs are in the cookbook (links above):

representative decode 186.61 t/s at exact p2048/g128 (174.60–201.83)
p8192/g128 157.92 t/s (1.81× vs matched no-spec 87.25 on that cell)
p8192/g1 cold input 7160 t/s (prompt tokens / client TTFT — not isolated engine prefill)
window acceptance 1830/3521 = 52.0%

Do not headline the p512 194.6 median (family range 140–220).

License

OpenMDW-1.1, same as the NVIDIA DFlash source.

SergiioB/Nemotron-3.5-Lightning-30B-A3B-DFlash-BF16