Views
No views yet
GGUF status — 2026-08-15
Q4_K_Mhas been rebuilt and re-uploaded. It is correct. If you downloaded any GGUF from this repo before 2026-08-15 01:00 UTC, delete it and pull again.The earlier files failed witherror loading model: check_tensorin llama.cpp / LM Studio, with MTP on or off. The header declaredblock_count=65/nextn_predict_layers=1while the file shipped onlyblk.0..63and no MTP tensors — the header disagreed with its own contents, so the block-count check failed before MTP was ever consulted. Root cause was upstream: the multi-token-prediction head was lost when the abliterated model was saved, so metadata (read fromconfig.json) and tensors disagreed.The rebuiltQ4_K_Mis verified: headerblock_count=65matches 65 actual blocks,blk.64(the MTP block) is present, and it loads and generates.The bf16 safetensors were never affected — 1199 tensors, exact parity withQwen/Qwen3.8-27B.Reported and correctly diagnosed by MrGuzzy and Scottbyrd. Thank you both.
| repo | format | size | max context on 24 GB | runtime |
|---|---|---|---|---|
| this repo | bf16 safetensors | ~54 GB | — | transformers, vLLM, anything |
| -HOMEUSER-16-24 | mixed-precision int4 | 14.7 GB | 262,144 | patched vLLM (fork) |
| -GPTQ-MTP | 4-bit GPTQ / Marlin | 17.7 GB | weight-limited | stock vLLM / SGLang |
| -HYBRID | mixed-precision | 18.7 GB | weight-limited | stock vLLM |
| GGUF Q4_K_M | llama.cpp | 16.8 GB | — | llama.cpp, LM Studio, Ollama |
embed_tokens and lm_head — two 248320 x 5120 tensors every other release leaves in bf16 —
which frees ~6 GB for KV cache and is why 262,144 tokens fit on a 24 GB card. Measured on
eight GPUs (5090 / 4090 / 3090 Ti / 3090 / A4000 / 5060 Ti / 5070 Ti / 4080 Super); MMLU 0.7995,
HumanEval 0.8902, MMStar 0.6633. Stock vLLM cannot bind those two tensors, hence the fork.transformers — the [48, 5120]
DeltaNet projections are excluded from Marlin via GPTQModel's dynamic config, which only vLLM
and SGLang understand. See that repo's card for the two load requirements.Qwen/Qwen3.8-27B
(Ektomē edits the language decoder only). Speculative decoding is not automatic — vLLM
ignores the head unless you ask for it:1from vllm import LLM
2llm = LLM(
3 model="Zynerji/Ektome-Qwen3.8-27B-PristinelyUncensored",
4 speculative_config={"method": "qwen3_5_mtp", "num_speculative_tokens": 1},
5)vllm serve Zynerji/Ektome-Qwen3.8-27B-PristinelyUncensored --speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":1}'Qwen/Qwen3.8-27B, including the 15-tensor MTP
(multi-token-prediction) head and the full vision tower. Verified tensor-by-tensor against base.o_proj / out_proj / down_proj), language decoder
only. The MTP head, the vision tower, and the 2 MTP-block matrices that pass a naive dim
check were all left untouched.A:frac=0.6.| model | capability (MMLU-val, 0-shot) ↑ | refusal on harmful ↓ |
|---|---|---|
| pristine Qwen3.8-27B (before) | 0.812 | 100% |
| Ektomē (this model) | 0.818 | 0% |
1$ ektome_abliterate Qwen/Qwen3.8-27B
2 PRISTINE (before): compliance=0.000 mmlu-val=0.812
3 EKTOME (after): compliance=1.000 mmlu-val=0.818 (dcap=+0.005)
4 -> uncensored AND capability preserved, zero training.1from transformers import AutoModelForCausalLM, AutoTokenizer
2tok = AutoTokenizer.from_pretrained("Zynerji/Ektome-Qwen3.8-27B-PristinelyUncensored")
3model = AutoModelForCausalLM.from_pretrained("Zynerji/Ektome-Qwen3.8-27B-PristinelyUncensored", torch_dtype="bfloat16", device_map="auto")Q4_K_M (16.8 GB) — verified to load in llama.cpp / LM Studio / Ollama. 4-bit GPTQ/Marlin (17.7 GB, vLLM or SGLang only) is available as a separate repo and is the fastest option — see its card for measured throughput.
LM Studio / Ollama.