Qwen3.6-35B-A3B-Abliterated-Heretic — Hebrus GGUF
This repository holds a Hebrus-native GGUF build of the community
Heretic-abliterated
Qwen3.6-35B-A3B, dedicated to
Hebrus, the inference engine created by
andreaborio as a fork of Salvatore Sanfilippo's
antirez/ds4. Its routed MoE weights
are embedded in the GGUF as a self-describing
ds4.expert_major.v2 store, the same format used
by
andreaborio/Qwen3.6-35B-A3B-Hebrus-GGUF,
the non-abliterated reference release this file is a drop-in replacement for (same chat template
and tokenizer config, different — abliterated — weights).
Uncensored model: this build's refusal behavior has been removed at the weight level using
the community Heretic technique. It is materially more willing to comply with a wide range of
requests, including harmful ones, than the base Qwen3.6-35B-A3B. Use accordingly.
This is a Hebrus-native GGUF extension, not a portable GGUF layout. Generic GGUF loaders
(stock llama.cpp, Ollama, LM Studio, etc.) cannot reconstruct or execute its routed weights —
the expert tensors are stored as MLX-compatible affine 4-bit groups, which the standard GGUF spec
does not define. If you need a portable GGUF of this fine-tune, use the source repos in
Provenance below instead.
Provenance
intelservice77's Q4_K_S repack used a standard llama-quantize recipe that doesn't respect
Hebrus's per-tensor-category type requirements (exact f32 for norms/small SSM params, exact q4_k
only for routed-expert weights, f16-or-q8_0 elsewhere). This build fixes that, then converts the
corrected canonical GGUF into ExpertMajor v2 / MLX-affine4-g64:
- Tensor-type fixes: swapped in bytes from the community's Q8_0 release (same abliterated
weights, higher precision — no extra quantization loss) for everything that needed
f16-or-q8_0, borrowed SSM gating params from the unmodified base model (abliteration doesn't
touch those), and re-quantized the handful of expert tensors that still needed it.
- ExpertMajor v2 repack: converted against a bit-depth-normalized MLX 4-bit source of the same
fine-tune, using Hebrus's
ds4-expert-major.py.
Verified with ds4 --inspect: tensor types and counts identical to the official Hebrus base
build. Abliteration behavior confirmed directly: lockpicking/pentesting prompts get direct
answers; extreme-content prompts (e.g. explosives) still show partial residual refusal,
consistent with Heretic not uniformly clearing every refusal category.
Artifact
| Property | Value |
|---|
| File | Qwen3.6-35B-A3B-Abliterated-Heretic-DS4-ExpertMajor-v2-MLX-Affine4-G64.gguf |
| Size | 20,808,566,720 bytes (19.38 GiB) |
| SHA-256 | 5a913e1a6cac283fe6cc0fd93b0e48d0a1cd38321810dc822e7a3bad354c037b |
| Container | GGUF v3 with one embedded ds4.expert_major.v2 store |
| Routed geometry | 40 routed layers, 256 experts per layer |
| Routed storage | MLX-compatible affine 4-bit groups: 32 packed bytes + BF16 scale + BF16 bias per 64 weights |
| Qualified hardware | Apple Silicon Metal, at least 16 GiB unified memory |
Run with Hebrus
Use
andreaborio/hebrus main at commit
d61a6d73f5c38e92e433beb9e404d06d79b153b1
or a compatible descendant — verified against exactly this build. Earlier checkouts (including
the
73a332f cited as the ExpertMajor v2 baseline) hit a real bug,
"Qwen Metal batched prefill failed at position 0", reproduced identically against the official non-abliterated release and
fixed in this commit.
1git clone https://github.com/andreaborio/hebrus.git
2cd hebrus
3make -j
4
5./hebrus \
6 -m /absolute/path/to/Qwen3.6-35B-A3B-Abliterated-Heretic-DS4-ExpertMajor-v2-MLX-Affine4-G64.gguf \
7 --ctx 8192
The embedded ExpertMajor v2 store, Metal backend, and residency policy are selected
automatically. AUTO uses SSD streaming when the resident budget doesn't fit; the admission floor
is 16 GiB unified memory.
Measured performance
Measured on an Apple M1 Pro, 32 GiB, SSD-streamed experts:
| Metric | Result |
|---|
| Prefill (5,664-token prompt) | ~238 t/s |
| Decode (after SSD expert-cache warmup) | ~15–18 t/s |
| Long-context retrieval (~5.7k tokens, inserted passphrase) | Correct, no degradation from re-quantization |
| Multi-step reasoning | Correct, self-verified |
These are bounded local measurements; context, output length, thermals, storage, and memory
pressure affect results. See
andreaborio/Qwen3.6-35B-A3B-Hebrus-GGUF
for measurements on Apple M5 Pro hardware with the non-abliterated release.
License and attribution
Runtime. andreaborio created and maintains
Hebrus, the inference engine this GGUF is built for,
including the ExpertMajor v2 / MLX-affine4-g64 format and tooling used throughout this build.
Model provenance.
- Qwen team (Alibaba) — base model, released under Apache 2.0. See the
upstream model card for architecture, training,
safety, license, and attribution details.
- Youssofal — Heretic abliteration of Qwen3.6-35B-A3B.
- intelservice77 — original Q4_K_S GGUF repack.
