The H3 repo ships the stock Qwen/Qwen3-VL-32B-Instruct as its text encoder — we verified this (1058 tensors, 66.7 GB, identical key structure), so this artifact is just a quantisation of the official Qwen weights, nothing H3-specific.

Why

In bf16 the encoder is 66.7 GB, which does not fit on a 5090/4090-class card. The usual workaround is accelerate.cpu_offload, which streams the whole model across PCIe on every request. In a three-stage H3 deployment that made text encoding the pipeline bottleneck.

	bf16 + cpu_offload	this artifact (NF4 resident)
text encode per request	7.4 – 8.5 s	0.09 s (text-only) / 0.5 – 1.5 s (with a keyframe)
weights on disk	66.7 GB	19 GB
VRAM while resident	1.5 GB (weights keep crossing PCIe)	18.2 GB
load time	~16 s (read 66.7 GB + quantise)	~6 s

Measured on an RTX 5090 (31.36 GiB), transformers 5.15.0, bitsandbytes 0.50.1.

Quality

prompt_embeds versus the bf16 reference: cosine 1.00003, relative L2 1.6 %.

End-to-end, generating video with the same prompt, spec and seed — only the encoder differs — frame PSNR against the bf16 run:

comparison	PSNR
same file against itself	∞
bf16 ↔ NF4, first+last-frame conditioning	24 – 29 dB
bf16 ↔ NF4, first-frame conditioning	18 – 28 dB
bf16 ↔ NF4, text-to-video	15 – 17 dB
same config, different seed	8.6 – 10.2 dB

The encoder swap sits far from the "different seed" floor, i.e. it perturbs the same clip rather than resampling a new one — and image-conditioned modes are affected least, because the reference frame anchors the composition.

Usage

python

from transformers import Qwen3VLForConditionalGeneration

text_encoder = Qwen3VLForConditionalGeneration.from_pretrained(
    "moonzerokevin/qwen3vl-32b-minimax-h3-nf4", device_map="cuda")

Then hand it to the H3 modular pipeline in place of the bf16 encoder:

python

pipe = ModularPipeline.from_pretrained(h3_path, workflow="fl2va")
pipe.load_components(names=["tokenizer", "processor", "vae", ...])  # skip text_encoder
pipe.text_encoder = text_encoder

Requires bitsandbytes (tested with 0.50.1 on sm120).

Notes

RedHatAI/Qwen3-VL-32B-Instruct-NVFP4 (compressed-tensors, ~16 GB) loads under transformers but cannot run inference there — the NVFP4 kernels live in vLLM; the forward pass fails with 'Linear' object has no attribute 'weight'. That is why this artifact uses bitsandbytes NF4 instead.
Double quantisation is on (bnb_4bit_use_double_quant=True), compute dtype bfloat16.