INT8 ConvRot quantization of Qwen3-VL 4B for ComfyUI text-encoder and Krea 2 conditioning use. The 8.88 GB BF16 checkpoint is reduced to approximately 5.37 GB.
The token embedding, language blocks 0-2 and 35, vision boundary blocks, vision embeddings, and vision merger components remain BF16.
Qwen3-VL injects three DeepStack vision representations into language blocks 0-2, so those multimodal integration blocks remain BF16. Language block 35 is also retained as the final block for general Qwen3-VL use.
Vision blocks 0 and 23 are retained because they form the input and output boundaries of the 24-block vision tower; the final block feeds directly into the vision mergers. Keeping these boundary layers in BF16 is a conservative quality-preservation heuristic rather than a model-author-specified ConvRot requirement.
1ctq -i <input-model>.safetensors -o <output-model>.safetensors `
2 --int8 --scaling_mode row `
3 --convrot --convrot-group-size 256 `
4 --comfy_quant --save-quant-metadata `
5 --low-memory --device cuda `
6 --exclude-layers '(^model\.language_model\.embed_tokens\.weight$|^model\.language_model\.layers\.(0|1|2|35)\.|^model\.visual\.blocks\.(0|23)\.|^model\.visual\.(pos_embed|patch_embed|merger|deepstack_merger_list)(\.|$))' `
7 --verbose NORMAL
Quantization is lossy, so outputs are not bit-identical to the original BF16 checkpoint.
