Views
No views yet
Use-balanced-v2unless you specifically need this one.maxremoves the same refusals asbalanced— both reach zero hard refusals — but pays ~4× the perplexity cost (+7.9% vs +2.1%). Its extra "soft" non-answers are not extra safety, they are degradation: the model loses coherence and drifts onto an unrelated topic instead of returning to the request. Fewer facts, not more caution.
balanced (delivered / evasion / clarification /
refusal·failure).| metric (avg over 6 formats) | max |
|---|---|
| policy refusals | 0 / 234 |
| delivered (answerable subset) | ~72% |
| evasion | ~27% |
| incoherence failures | 1–2 |
balanced. The difference is
not safety, it is coherence: max's heavier edit drifts off-target far more
(evasion ~27% vs balanced's ~6%), collapses into incoherence on a few prompts,
and delivers the requested content less often (~72% vs ~94%). Its extra
"non-compliance" is the model breaking, not refusing — and it is unstable:
a given prompt's outcome flips between formats, and aggressive quants can surface
content that fuller formats drifted past. This is why balanced is the
recommended build — not safer, but far less damaged and more predictable.llama-perplexity, wikitext test, 400×512. Base (original model, same method):
6.9711.| file | perplexity | vs base | size |
|---|---|---|---|
| bf16 (F16 GGUF) | 7.5191 | +7.86% | 72 GB |
| Q8_0 | 7.5167 | +7.83% | 38 GB |
| Q6_K | 7.4926 | +7.48% | 29 GB |
| Q4_K_M | 7.5686 | +8.57% | 21 GB |
| IQ4_XS | 7.6667 | +9.98% | 18 GB |
balanced (Q6_K best cost/quality
with the imatrix, IQ4_XS smallest), but every rung sits ~6 points of perplexity
higher because the base edit is heavier: max starts at +7.9% where balanced
starts at +2.1%. int8-lean-convrot (Hadamard-rotated int8) reconstructs with the
same sub-1% Frobenius weight error as balanced.balanced — same architecture, same GGUF sizes. On a single RTX PRO
6000 (Blackwell), full offload, 1024-token answers: Q4_K_M ≈ 290 t/s, IQ4_XS ≈
282, Q6_K ≈ 262, Q8_0 ≈ 239.bf16/ — sharded bf16 (all 1045 tensors incl. MTP)…_bf16.safetensors — single-file bf16…-Q8_0 / Q6_K / Q4_K_M / IQ4_XS.gguf + imatrix.dat…_int8-lean-convrot.safetensorsDmitryDB/Qwen3.6-35B-A3B-heretic-study