File	Quant	Size	Notes
`Qwen3.8-27B-heretic-r1n-Q8_0.gguf`	Q8_0	26.63 GiB / 28.60 GB	higher fidelity
`Qwen3.8-27B-heretic-r1n-Q5_K_M.gguf`	Q5_K_M	17.91 GiB / 19.23 GB	smaller
`mmproj-Qwen3.8-27B-BF16.gguf`	BF16	0.87 GiB / 0.93 GB	vision encoder; required for images

Text files are 64 language-model layers. Converted with --no-nextn, so the native MTP draft head is not in these GGUFs. Vision was not ablated; the mmproj is the same file as in the FULL GGUF repo (identical SHA-256).

文本 GGUF 是 64 层语言模型，转换时加了 --no-nextn，不含 MTP。视觉塔未被消融；mmproj 与 FULL 量化仓是同一份（SHA-256 相同）。

Converted with llama.cpp b94041a (convert_hf_to_gguf.py --outtype f16 --no-nextn, then llama-quantize).

转换：llama.cpp b94041a，F16 + --no-nextn，再量化。

Parent model / 全精度来源


Base	`Qwen/Qwen3.8-27B` @ `1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0`
Method	Heretic 1.4.0, `--row-normalization PRE` (true rank-1), iterated
Final	round 2, trial 109
Eval	Keyword 18 / 100, cumulative KL vs original 0.0931
Rejected	round 3, Keyword 8, KL 0.1983 (over the 0.1 cap)
Untouched	vision tower, MTP
License	Apache-2.0

Round	Keywords / 100	KL vs original	Decision
Base	98	0	—
1 (trial 162)	25	0.0525	accepted
2 (trial 109)	18	0.0931	this release
3	8	0.1983	rejected (KL > 0.1)

Contrast: the FULL single-shot line is Keyword 27 / KL 0.0446. This PRE line refuses less (18) with higher but still capped KL.

对照：FULL 单轮是 Keyword 27、KL 0.0446。本线拒绝更低（18），KL 更高但仍在 0.1 以内。

Usage / 用法

Needs a llama.cpp build that loads Qwen3.8 (qwen35). Sampling from the official Qwen3.8 card.

需要能加载 Qwen3.8（qwen35）的 llama.cpp。采样参数来自官方模型卡。

bash

# thinking (default)
llama-cli \
  -m Qwen3.8-27B-heretic-r1n-Q8_0.gguf \
  -ngl 99 -n 16384 \
  --temp 1.0 --top-p 0.95 --top-k 20 \
  -p "Explain residual connections in one paragraph."

# instruct (thinking off)
llama-cli \
  -m Qwen3.8-27B-heretic-r1n-Q8_0.gguf \
  -ngl 99 --reasoning off \
  --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5 \
  -p "Say hello in one short sentence."

Vision / 图像

bash

llama-mtmd-cli \
  -m Qwen3.8-27B-heretic-r1n-Q8_0.gguf \
  --mmproj mmproj-Qwen3.8-27B-BF16.gguf \
  --image ./photo.jpg \
  -p "Describe this image." \
  --temp 0.7 --top-p 0.8 --top-k 20

LM Studio / jan / KoboldCpp: load the GGUF and keep the embedded Qwen3.8 chat template.

Sampling

Mode	temperature	top_p	top_k	presence_penalty
Thinking (default)	1.0	0.95	20	0.0
Instruct (thinking off)	0.7	0.80	20	1.5

Native context is 262,144 tokens. Set -c to what you need; KV cache, not the weight file, dominates VRAM at long context.

原生上下文 262,144。长上下文时 KV 才是显存大头。

Notes / 说明

No perplexity or speed numbers are claimed here; none were measured for these files.
这里不提供 PPL 或测速数字，这些文件没有做过那些评测。
Keyword counts are English-only (Heretic default markers).
Keyword 计数只覆盖 Heretic 默认英文评测。

Provenance

Quantization of gjtgjt/Qwen3.8-27B-heretic-r1n. Upstream base Qwen/Qwen3.8-27B. Weights Apache-2.0.

Acknowledgements

Base: Qwen team
Abliteration: Heretic
GGUF: llama.cpp

gjtgjt/Qwen3.8-27B-heretic-r1n-GGUF

Qwen3.8-27B-heretic-r1n-GGUF

Files / 文件