Property	Value
Method	NVFP4 (W4A16 float)
Tool	nvidia-modelopt `mtq.quantize`
Config	`NVFP4_DEFAULT_CFG`
Vision encoder	BF16 (not quantized)
MTP head	BF16 (not quantized)
Calibration	allenai/c4 (128 samples, seq_len 512)
Hardware	NVIDIA B200 (Blackwell)

NVFP4 is a native Blackwell format (E2M1, 4-bit float). The model is intended for inference on B200 GPUs with TensorRT-LLM or vLLM with NVFP4 support.

Usage

python

from transformers import AutoProcessor, AutoModelForImageTextToText
import torch

model = AutoModelForImageTextToText.from_pretrained(
    "Shashwat42/Qwen3.6-27B-VLM-NVFP4",
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
processor = AutoProcessor.from_pretrained("Shashwat42/Qwen3.6-27B-VLM-NVFP4", trust_remote_code=True)

Note: Requires NVFP4-capable hardware (NVIDIA Blackwell B200) for full performance.

Files

model-00001-of-00002.safetensors — NVFP4 weights shard 1 (~9.3 GB)
model-00002-of-00002.safetensors — NVFP4 weights shard 2 (~4.3 GB)
hf_quant_config.json — nvidia-modelopt quantization config
Processor / tokenizer files from base model