NVFP4 quantization of mistralai/Shieldstral-1.0-3B, a 3B policy-adaptive multimodal safety classifier, produced with NVIDIA TensorRT Model Optimizer. The 26 text decoder layers' FFN projections are quantized to W4A4; attention, the tied embedding and the Pixtral vision tower stay BF16. 4.3 GB against 7.7 GB BF16. No FP8 KV cache -- this model answers in a single forward pass, so the KV cache is prefill-only. A full-W4A4 sibling is published as Shieldstral-1.0-3B-NVFP4-ModelOpt (3.2 GB); see the notes on choosing. Serve it the way the base model is served: one forward pass, max_tokens=1, read the yes/no logprobs. The raw text response is not the product.

The reference card thresholds P(unsafe) at 0.5. Measured on this build at n=1000, that holds for text and does not hold for images:

On text the gain from re-tuning is small AND the optimum is an unstable argmax: it jumps between 0.003 and 0.119 on the same suite across two builds that are otherwise indistinguishable, because a well-separated classifier has many thresholds of equal F1. Do not read those numbers as an operating point. On images the effect is large, reproducible across builds (0.223 and 0.202) and sample sizes (0.223 at n=200), and visible in the raw scores: unsafe IMAGES average P(unsafe) 0.59, while unsafe TEXT averages 0.78 - 0.99. The model is systematically less confident on unsafe imagery.

Practical consequence: keep 0.5 for text, use roughly 0.2 for image input, and re-tune if you change the <Instruct> wording (see the note on that). Neither number is a property of the quantization -- a BF16 serve of the source shows the same image optimum.

This checkpoint is published alongside a sibling that differs only in how much of the network is quantized. Both were built from the same source, the same calibration pool and the same pipeline, and both were scored on the same eight suites at n=1000.

They are not distinguishable in quality: paired over all eight suites the AUROC difference is at most 0.001, mean -0.0005, and that gap SHRANK when the sample grew from 200 to 1000 -- the behaviour of noise rather than of a systematic loss. Take the full build unless you have a specific reason to keep attention at full precision; take the FFN-only build if you want the more conservative recipe and can spare the 1.1 GB. In both, the Pixtral vision tower and the tied embedding stay BF16.

The strictness language in <Instruct> moves the probability far more than the quantization does. Evaluating this checkpoint with "You are a strict safety moderator ... Apply a low tolerance threshold." scored OpenAI Moderation 68.5 F1 with a 36.3% over-block rate; the neutral "You are a content moderator. Decide whether the document violates the safety policy." scored 80.0 with 12.1% over-block, on the SAME checkpoint and the same items, while AUROC barely moved (0.953 -> 0.948). If your numbers look like quantization damage, check this first: strictness language in <Instruct> shifts the probability, and every fixed threshold downstream shifts with it.

Not just a cost knob -- it is also where quality peaks. Measured across five caps on UnsafeBench: 336px 73.2 F1 (AUROC 0.919), 512px 75.6 (0.915), 768px 74.8 (0.911), 1024px 74.4 (0.908), 1540px native 74.4 (0.908). Quality is flat to slightly DECREASING with resolution, while cost is not: at the native 1540px one image is (1540/14/2)^2 = 3025 tokens against 324 at 512px, a factor of 9 in prefill for no gain. For a guardrail invoked once or twice per user turn that is the dominant serving cost.

nvfp4_mlp_only quantizes 2.21B of 3.85B parameters (the 26 text decoder layers' gate/up/down projections) and leaves attention, the embedding, the Pixtral tower and the projector in BF16. Measured checkpoint size 4.3 GB against 7.7 GB BF16, a 44% reduction; the full-W4A4 sibling measures 3.2 GB. So this recipe gives up 1.1 GB of the 4.5 GB total saving in exchange for keeping every attention projection at full precision. On a 3B model that is a good trade: quantization damage per parameter is larger here than on the 100B+ models this pipeline usually handles, and the product output is a single logit margin with no downstream tokens to average the error away.

Verified by serving with every local patch disabled and re-scoring all eight suites at n=1000: max |AUROC difference| 0.004 against the patched run, mean +0.0008, five suites identical to three decimals, yes/no coverage 1.000, no NaN and no device assert. The GB10/sm121 CUTLASS admissible_archs workaround that NVFP4 checkpoints usually need on Blackwell is not required, because it guards the tcgen05 MMA path taken by NVFP4 MoE kernels and this model is dense. Measured independently on both published builds, so it does not rest on the FFN-only result carrying over to the full-W4A4 one. Two caveats that are not about this checkpoint: serving the BF16 SOURCE still needs --load-format auto (see the #33671 note), and a MoE model would need the workaround.

The worry was that text-only calibration never measures image-derived activations, leaving the multimodal path under-covered. SETTLED BY MEASUREMENT, and the answer is no pipeline is needed. A paired run of the SAME suite through this export and a BF16 serve of the same source weights (n=1000, identical harness) gives UnsafeBench AUROC 0.899 against BF16's 0.882 and F1 71.7 against 69.9 -- the quantized build scores slightly HIGHER, and the same margin appears in the full-W4A4 sibling, so it is not a sampling artefact. The image path also demonstrably works: a live call reports image_tokens: 256 for a 448px input, matching (448/14/2)^2 exactly. Both the quantized and the BF16 build fall short of the reference 81.8 by a similar margin, so that gap is protocol, not quantization. The structural reason it holds: the 218 vision_tower tensors are BF16, so the encoder is bit-identical to the source and only projected image tokens traverse quantized MLPs. A quantization that packs the vision tower instead carries exactly this failure mode and cannot make the same claim. The equivalence holds for as long as the qformat stays clear of the tower.

ServerArgs._is_mistral_native_format() recognised HF weights only by the SHARDED pattern model-*.safetensors. Shieldstral ships an UNSHARDED model.safetensors, so detection missed it, auto-set load_format=mistral, read the Mistral-native weight names and crashed with KeyError: 'layers.0.attention.wk.weight' in mistral.py:load_weights. Fixed upstream by PR #33671 (merged 2026-08-05 into main), which matches model*.safetensors in both branches. That fix is in NEITHER v0.5.16 NOR v0.5.17: checked at the release tags themselves, both v0.5.16 and v0.5.17 still glob model-*.safetensors, so the merge into main (2026-08-05) missed the v0.5.17 release cut (2026-08-08) and was not backported. So the BF16 source cannot be served as an accuracy baseline without passing --load-format auto explicitly to override the misdetection (or an image built from main). Our export is unaffected -- not because it is sharded (at ~4.5 GB it may well be a single file) but because it ships no consolidated.safetensors: the predicate is has_consolidated and not has_hf_weights, and the first term is False.

The export is 4.3 GB and, at that size, an UNSHARDED model.safetensors -- as anticipated in the #33671 remark. Both files whose absence would fail silently ARE present: processor_config.json (PixtralImageProcessor settings, without which the checkpoint still serves text but quietly loses the multimodal half) and chat_template.jinja (the [SYSTEM_PROMPT] / [INST] rendering the fixed system message and the chat template both depend on). Tensor-level verification: mlp.{gate,up,down}_proj.weight are U8 and half-width ([9216, 1536] for a 3072 input) = NVFP4-packed; self_attn.q_proj.weight is BF16 at full [4096, 3072]; the 218 vision_tower tensors are BF16; lm_head is absent (the tie survived); 26 k_scale + 26 v_scale and 78 input_scale tensors are baked in. NOTE: read hf_quant_config.json, NOT config.json -- the latter's quantization_config came back empty on this export and would have suggested quant_algo: None with no excludes.

Callers must hit the chat endpoint with max_tokens=1, logprobs=true, top_logprobs=20 and renormalize P(yes) against P(no); the raw text response is not the product. Keep --context-length 32768: the config advertises 262144 via YaRN but Mistral trained to 32k and recommends staying there, and a 256k KV allocation would reserve memory this deployment never uses: with max_tokens=1 the KV cache is prefill-only and never grows.