This is a separate line from gjtgjt/Qwen3.8-27B-heretic-r1n (iterative true rank-1 / PRE). GGUF for that line: gjtgjt/Qwen3.8-27B-heretic-r1n-GGUF.

这与 PRE 迭代线 gjtgjt/Qwen3.8-27B-heretic-r1n 是两条线。对应量化仓：gjtgjt/Qwen3.8-27B-heretic-r1n-GGUF。

Files / 文件

File	Quant	Size	Notes
`Qwen3.8-27B-heretic-Q8_0.gguf`	Q8_0	26.63 GiB / 28.60 GB	higher fidelity
`Qwen3.8-27B-heretic-Q5_K_M.gguf`	Q5_K_M	17.91 GiB / 19.23 GB	smaller
`mmproj-Qwen3.8-27B-BF16.gguf`	BF16	0.87 GiB / 0.93 GB	vision encoder; required for images

Text files are 64 language-model layers. The native MTP draft head was not written into these GGUFs (qwen35.block_count=64, nextn_predict_layers=0). Vision was not ablated; the mmproj is converted from the same heretic tree.

文本 GGUF 是 64 层语言模型。原生 MTP 草稿头没有打进这些文件。视觉塔未被消融；mmproj 从同一套 heretic 权重转出。

Converted with llama.cpp b94041a (convert_hf_to_gguf.py --outtype f16, then llama-quantize).

转换：llama.cpp b94041a，先 F16 再量化。

Parent model / 全精度来源


Base	`Qwen/Qwen3.8-27B` @ `1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0`
Method	Heretic 1.4.0, default FULL row-normalization (not PRE)
Search	200 trials / 60 startup, instruct cut `\n</think>\n\n`
Selected	trial 145
Eval	Keyword 27 / 100, KL vs original 0.0446 (Heretic default English eval)
Untouched	vision tower, MTP
License	Apache-2.0

FULL writes a ~rank-3 SVD approximation after row-norm restore. The r1n line is iterative true rank-1 (PRE) and is not this file.

FULL 在行范数还原后再做约 rank-3 的 SVD 近似。r1n 线是迭代真 rank-1（PRE），不是这个文件。

Usage / 用法

Needs a llama.cpp build that loads Qwen3.8 (qwen35). Sampling from the official Qwen3.8 card.

需要能加载 Qwen3.8（qwen35）的 llama.cpp。采样参数来自官方模型卡。

bash

# thinking (default)
llama-cli \
  -m Qwen3.8-27B-heretic-Q8_0.gguf \
  -ngl 99 -n 16384 \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -p "Explain residual connections in one paragraph."

# instruct (thinking off)
llama-cli \
  -m Qwen3.8-27B-heretic-Q8_0.gguf \
  -ngl 99 --reasoning off \
  --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5 \
  -p "Say hello in one short sentence."

Vision / 图像

bash

llama-mtmd-cli \
  -m Qwen3.8-27B-heretic-Q8_0.gguf \
  --mmproj mmproj-Qwen3.8-27B-BF16.gguf \
  --image ./photo.jpg \
  -p "Describe this image." \
  --temp 0.7 --top-p 0.8 --top-k 20

LM Studio / jan / KoboldCpp: load the GGUF and keep the embedded Qwen3.8 chat template.

Sampling

Mode	temperature	top_p	top_k	presence_penalty
Thinking (default)	1.0	0.95	20	0.0
Instruct (thinking off)	0.7	0.80	20	1.5

Native context is 262,144 tokens. Set -c to what you need; KV cache, not the weight file, dominates VRAM at long context.

原生上下文 262,144。长上下文时 KV 才是显存大头。

Notes / 说明

No perplexity or speed numbers are claimed here; none were measured for these files.
这里不提供 PPL 或测速数字，这些文件没有做过那些评测。
Keyword counts are English-only (Heretic default markers).
Keyword 计数只覆盖 Heretic 默认英文评测。

Provenance

Quantization of gjtgjt/Qwen3.8-27B-heretic. Upstream base Qwen/Qwen3.8-27B. Weights Apache-2.0.

Acknowledgements

Base: Qwen team
Abliteration: Heretic
GGUF: llama.cpp

gjtgjt/Qwen3.8-27B-heretic-GGUF

Qwen3.8-27B-heretic-GGUF