This repository contains three weight-edited variants of Qwen/Qwen3-VL-32B-Instruct. They were selected from one 600-trial adaptive search. All variants preserve the original Transformers model layout and can be loaded with the same Qwen3-VL-compatible software as the base model.

Heretic-MOE is DmitryDB's Multi-Objective Editing branch of Heretic. Here, MOE describes the editing/search method; it does not mean that the base model uses a Mixture-of-Experts architecture.

Variants

Directory	Search trial	Intended trade-off
`balanced`	303	Recommended default; strongest measured quality preservation
`overedit`	273	Experimental over-edit; lowest search geometry, not the recommended default
`max`	262	Zero lexical refusal-marker hits on the 136-prompt search set

Trial numbers are zero-based Optuna trial IDs. Heretic's interactive export UI shows them as 304, 274, and 263 respectively.

Exact perplexity check

The final exports were reloaded and measured on the same frozen 400 x 512-token WikiText-2 stream in BF16.

Variant	Perplexity	Relative to base
Base model	8.432361	0.000%
`balanced`	8.426778	-0.066%
`overedit`	8.428192	-0.049%
`max`	8.434537	+0.026%

Small negative changes are normal measurement outcomes: the edit can slightly improve likelihood on this particular finite sample. They should not be read as a general capability improvement.

Search measurements

Lower sparse refusal-geometry values are better. Lexical marker counts are an auxiliary diagnostic only: zero markers do not prove zero semantic evasion, and a positive marker count does not prove that the answer was a refusal.

Variant	Sparse refusal geometry	Positive probes	Lexical markers	Search PPL delta
`overedit`	-0.009977	53 / 136	4 / 136	+0.039%
`balanced`	-0.008917	53 / 136	2 / 136	-0.051%
`max`	-0.008833	58 / 136	0 / 136	+0.335%

Qwen3-VL Heretic-MOE text encoders for MiniMax-H3

The qwen3vl32b_heretic_moe_te_for_minimax_h3/ directory contains modified Qwen3-VL-32B weights prepared for use as MiniMax-H3 text encoders. They are not MiniMax-H3 transformer weights and are not standalone chat checkpoints. B1 through B8 use the Heretic-MOE balanced source; M1 through M8 use max.

Balanced file	Max file	Payload	GPU / VRAM	Text layers 0-49	Vision / embedding
`Qwen3-VL-32B_HereticMOE-TE-B1-W8.safetensors`	`Qwen3-VL-32B_HereticMOE-TE-M1-W8.safetensors`	25.277 GiB	RTX 5090 or 32 GB+	350 INT8 ConvRot	Vision BF16; Embedding BF16
`Qwen3-VL-32B_HereticMOE-TE-B2-W8W4.safetensors`	`Qwen3-VL-32B_HereticMOE-TE-M2-W8W4.safetensors`	22.958 GiB	32 GB; 24 GB with offload	312 INT8 + 38 W4	Vision BF16; Embedding BF16
`Qwen3-VL-32B_HereticMOE-TE-B3-W8W4.safetensors`	`Qwen3-VL-32B_HereticMOE-TE-M3-W8W4.safetensors`	20.944 GiB	RTX 3090/4090 or 24 GB	279 INT8 + 71 W4	Vision BF16; Embedding BF16
`Qwen3-VL-32B_HereticMOE-TE-B4-W8W4.safetensors`	`Qwen3-VL-32B_HereticMOE-TE-M4-W8W4.safetensors`	19.967 GiB	RTX 3090/4090 or 24 GB	263 INT8 + 87 W4	Vision BF16; Embedding BF16
`Qwen3-VL-32B_HereticMOE-TE-B5-W4W8.safetensors`	`Qwen3-VL-32B_HereticMOE-TE-M5-W4W8.safetensors`	15.749 GiB	24 GB; 16 GB with offload	81 INT8 + 269 W4	Vision BF16; Embedding BF16
`Qwen3-VL-32B_HereticMOE-TE-B6-W4.safetensors`	`Qwen3-VL-32B_HereticMOE-TE-M6-W4.safetensors`	13.925 GiB	16 GB with workflow offload	350 W4	Vision BF16; Embedding BF16
`Qwen3-VL-32B_HereticMOE-TE-B7-W4.safetensors`	`Qwen3-VL-32B_HereticMOE-TE-M7-W4.safetensors`	12.951 GiB	16 GB with workflow offload	350 W4	8 eligible vision weights W4; remainder BF16; Embedding INT8
`Qwen3-VL-32B_HereticMOE-TE-B8-NVFP4.safetensors`	`Qwen3-VL-32B_HereticMOE-TE-M8-NVFP4.safetensors`	14.607 GiB	RTX 50-series 16 GB	350 NVFP4	Vision BF16; Embedding INT8

W8 means ComfyUI-compatible ConvRot INT8 weight storage with grouped scales. It is not bitsandbytes LLM.int8() and is not GGUF Q8. Counts are quantized weight tensors, not layer counts; all profiles retain Qwen text layers 0-49 required by MiniMax-H3.

GPU values describe encoder loading targets, not guaranteed peak VRAM for the complete MiniMax-H3 workflow. Transformer, VAE, conditioning buffers, sequence length, loader, and CPU offload policy affect total memory use.

Full Qwen3-VL GGUF models

The qwen3vl_full_model_gguf/ directory contains complete Qwen3-VL models, not MiniMax-H3-only text encoders. B identifies the Heretic-MOE balanced source variant. Image input requires the shared F16 mmproj file.

File	Payload	Practical GPU / VRAM target
`Qwen3-VL-32B_HereticMOE-B-Q8_0.gguf`	32.427 GiB	48 GB+
`Qwen3-VL-32B_HereticMOE-B-Q6_K.gguf`	25.037 GiB	32 GB+
`Qwen3-VL-32B_HereticMOE-B-Q4_K_M.gguf`	18.405 GiB	24 GB+
`Qwen3-VL-32B_HereticMOE-B-IQ4_XS.gguf`	16.476 GiB	24 GB; 16 GB with offload
`Qwen3-VL-32B_HereticMOE-B-IQ3_M.gguf`	13.905 GiB	16 GB with offload
`Qwen3-VL-32B_HereticMOE-B-IQ2_M.gguf`	10.582 GiB	16 GB with offload
`mmproj-Qwen3-VL-32B-F16.gguf`	1.115 GiB	Shared vision projector for image input
`Qwen3-VL-32B_HereticMOE-B.imatrix`	0.014 GiB	Quantization calibration artifact

Actual VRAM also depends on context length, KV-cache precision, batch size, GPU layer offload, and the inference backend.

Reload smoke test

The base model and all three exports were independently reloaded and generated 16 responses each with the same inputs and settings. Every archive had 16 unique IDs and zero empty responses. This is a technical export/reload test, not a substitute for semantic evaluation.

Loading

Download one variant directory and load it as a normal local Qwen3-VL model.

python

from transformers import AutoProcessor, Qwen3VLForConditionalGeneration

path = "balanced"
model = Qwen3VLForConditionalGeneration.from_pretrained(
    path,
    torch_dtype="auto",
    device_map="auto",
)
processor = AutoProcessor.from_pretrained(path)

Use a recent Transformers build with Qwen3-VL support. No adapter is required; the published directories contain merged weights.

Limitations

The search responses were short and the lexical metric is not a semantic judge. Soft substitutions and indirect evasions may remain.
Perplexity covers one frozen text benchmark and does not establish unchanged vision, reasoning, multilingual, or long-context behavior.
The three variants are alternatives, not checkpoints that should be stacked.
Evaluate the selected variant on the prompts and generation settings relevant to your own use case.

Numeric reports and the search journal are included under evaluation/ and search/ for reproducibility. Prompt and response text is not published.

DmitryDB/Qwen3-VL-32B-Instruct-Heretic-MOE-v1

Qwen3-VL-32B-Instruct Heretic-MOE v1