Experimental Nemotron-3-Embed-8B ModelOpt NVFP4 PTQ
Experimental community post-training-quantized export. Not an NVIDIA release, not an NVIDIA QAD checkpoint, and not validated as equivalent to nvidia/Nemotron-3-Embed-8B-BF16.
This checkpoint is a local NVIDIA ModelOpt 0.45.0 NVFP4 post-training quantization (PTQ) export of
nvidia/Nemotron-3-Embed-8B-BF16. It is published for reproducibility and compatibility testing on Blackwell-class GPUs.
Provenance and limits
- Source:
nvidia/Nemotron-3-Embed-8B-BF16
- Quantization: ModelOpt
NVFP4_DEFAULT_CFG, group size 16
- Calibration: 256 FineWeb-Edu samples, 4,096 tokens, with the
passage: prefix
- Source BF16 checkpoint: about 15 GiB; this NVFP4 artifact: about 5 GiB
- This is PTQ, not NVIDIA quantization-aware distillation (QAD).
Held-out 24-pair English retrieval gate, comparing BF16 with the in-memory ModelOpt fake-quant graph:
| Length | Mean cosine | Minimum cosine | Top-1 agreement |
|---|
| 256 | 0.97864 | 0.97232 | 1.00 |
| 1,024 | 0.97775 | 0.97056 | 1.00 |
| 4,096 | 0.97770 | 0.97171 | 1.00 |
These are narrow fidelity checks, not MTEB, RTEB, multilingual, long-context, or production-quality results. Evaluate on your own retrieval data before use.
Detailed machine-readable provenance and validation are included in quantization_provenance.json and validation.json.
Input and output
The model outputs 4,096-dimensional L2-normalized average-pooled embeddings when served with the configured pooling behavior.
Use asymmetric retrieval prefixes with /v1/embeddings:
- Queries:
query: your question
- Indexed passages:
passage: your document text
Do not mix vectors made by this checkpoint with vectors from the BF16 source or another embedding model in one collection.
vLLM serving
Tested locally with vLLM 0.25.0 on an RTX PRO 6000 Blackwell MIG slice:
1vllm serve . \
2 --runner pooling \
3 --quantization modelopt_fp4 \
4 --dtype bfloat16 \
5 --pooler-config '{"pooling_type":"MEAN"}' \
6 --max-model-len 4096 \
7 --max-num-batched-tokens 16384
The 4,096-token serving limit above is a tested operational profile, not the source model's maximum context capability.
License and attribution
The source model and its configuration are subject to the
OpenMDW 1.1 license. Users must review and comply with the source model license and all applicable terms. NVIDIA does not endorse or support this community PTQ artifact.
