A GGUF conversion of lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA, so the rewriter runs under llama.cpp against a quantized Qwen3.6-27B instead of the 52 GB bf16 checkpoint.

This repository contains no new training — only a format conversion. The tensor values are unchanged.

The original adapter is a PEFT LoRA for Qwen/Qwen3.6-27B, which means a 52 GB download before anything can run. Under llama.cpp the same adapter attaches to a quantized base:

Offloading fewer layers lowers VRAM further at the cost of speed, and llama.cpp brings ROCm, Metal and CPU along with CUDA.

Base quants come from unsloth/Qwen3.6-27B-GGUF.

Tensor count and alpha match the source adapter exactly.

Take Q8_0 unless you have a reason not to. It is quantised from the F16 in this repository rather than converted separately, so tensor names, their order and every metadata field are identical — only the stored values differ, by a mean of 0.001 relative across all 992 tensors and 0.004 on the worst one.

Both behave the same where it matters: with either adapter attached the rewrite comes back in the trained cut-point form, [Shot 2] At 00:06.500, while the unadapted base model writes plain ranges like (0s-4s) instead.

system_prompt.txt must hold the exact system prompt from prompt_template.py in the source repository — the adapter was trained on that wording, and changing it degrades the rewrite.

--reasoning off matters: it corresponds to the enable_thinking=False that the reference infer.py passes. Without it the model spends hundreds of tokens reasoning before starting the rewrite.

Measured on an RTX 5090 with Q4_K_M: 50 tok/s with the adapter, 78 tok/s without it. That ~35% difference is llama.cpp computing the adapter's matmuls, which is the simplest confirmation that it is actually applied.

MiniMax-H3-Prompt-Rewriter-ComfyUI picks this adapter up automatically when a GGUF base model is selected and llama-cpp-python is installed. It fetches F16 by default; name MiniMax-H3-Prompt-Rewriter-LoRA-Q8_0.gguf in the options node's adapter field to take the smaller one instead and it is downloaded from here.

convert_lora_to_gguf.py from llama.cpp, plus a one-hunk fix to conversion/qwen.py.

llama.cpp reorders the V heads of Qwen3.5's linear-attention layers. For in_proj_qkv, in_proj_z, in_proj_a and in_proj_b that reorder lands on the output dimension and a LoRA passes through untouched. For out_proj it lands on the input dimension, where LoraTorchTensor.reshape refuses — it cannot reshape the axis that lives on the A factor.

The fix is to express that one reorder as an index instead of reshape/permute/reshape. LoraTorchTensor.__getitem__ already routes a last-axis index onto A, and

exactly, so nothing is approximated and no full-rank delta is materialised. For ordinary dense tensors the two forms are bit-identical, so the base-model conversion path is unaffected.

The weights originate from lightx2v/MiniMax-H3-Prompt-Rewriter-LoRA and were converted without modifying their values. Whatever terms that repository carries apply here unchanged; this repository claims nothing additional.

Use of MiniMax-H3 itself is governed by the licence and acceptable-use terms in the official MiniMax-H3 repository.