Qwythos-9B-Claude-Mythos-5-1M — SHQ8 (Selective Hybrid Quants)

Note on upstream v2: After these quants were made, the upstream release was updated to v2 with tokenizer normalization, embedded chat template improvements, and MTP variants. These quants are based on the original v1 source — they work fine with current llama.cpp for inference, but if you need the updated chat template or MTP support, grab the latest GGUFs from empero-ai/Qwythos-9B-Claude-Mythos-5-1M. MTP quants now available: wepiqx/Qwythos-9B-Claude-Mythos-5-1M-MTP-SHQ8-GGUF — same SHQ8 method with built-in MTP draft head for speculative decoding.

Note: File names contain Q5_K_M for HF parser compatibility only. These are not pure Q5_K_M — they're selective hybrid quants using Q8_0, Q6_K, IQ4_XS, Q5_K, and F16 across different tensor types. See each section for the exact per-tensor map.

Selective hybrid quantizations for Empero's Qwythos-9B-Claude-Mythos-5-1M — a full-parameter reasoning fine-tune of Qwen3.5-9B with 1M context and vision support.

Uses the exact same SHQ8 method and formulas as Qwable-9B-Claude-Fable-5-SHQ8-GGUF — same architecture, same imatrix, same quantization strategies. This is not a new SHQ9 series; it's the proven SHQ8 approach applied to a different fine-tune.

Status: Complete

SHQ8-v2 vs stock quants: Despite the Q5_K_M name tag, SHQ8-v2 is actually better than a pure Q5_K_M — Q8_0 on critical attention paths + Q6_K on mid layers more than compensates for IQ4_XS on low-importance tensors. A stock Q5_K_M would be ~5.3 GB with PPL ~7.9 on Qwythos; v2 is 7.65 at nearly the same size. And vs Q3_K_M (~4.1 GB, PPL ~8.5+): v2 is only 1.6 GB larger but dramatically better quality. If you need the smallest possible file, consider v2 before dropping to low-bit quants.

Architecture

Imatrix

SHQ8-OptA (Quality Champion)

~/llm-tools/llama.cpp/build/bin/llama-quantize \
  --imatrix /mnt/everything/qwen/source/Qwable-9B-Claude-Fable-5.imatrix.gguf \
  --output-tensor-type Q5_K \
  --token-embedding-type Q4_K \
  --tensor-type "output_norm.*=Q8_0" \
  --tensor-type "blk\.\d+\.attn_gate=Q8_0" \
  --tensor-type "blk\.\d+\.attn_qkv=Q8_0" \
  --tensor-type "blk\.\d+\.ssm_alpha=Q8_0" \
  --tensor-type "blk\.\d+\.ssm_beta=Q8_0" \
  --tensor-type "blk\.31\.ffn_down=Q8_0" \
  --tensor-type ".*attn_q_norm.*=Q8_0" \
  --tensor-type ".*attn_k_norm.*=Q8_0" \
  --tensor-type ".*ssm_conv1d.*=Q8_0" \
  --tensor-type ".*ssm_dt.*=F16" \
  --tensor-type ".*ssm_a=F16" \
  --tensor-type ".*attn_norm.*=F16" \
  --tensor-type ".*post_attention_norm.*=F16" \
  --tensor-type ".*ssm_norm.*=F16" \
  /mnt/everything/qwen/source/Qwythos-9B-Claude-Mythos-5-1M-BF16.gguf \
  /mnt/everything/qwen/output/Qwythos-9B-Claude-Mythos-5-1M-SHQ8-OptA.gguf \
  Q5_K_M

• PPL 7.4831 — beats Q6_K (7.5876) by −0.105 PPL at 10% smaller size
• Size 6,303 MiB vs Q6_K 7,008 MiB (10% smaller, 705 MiB saved)
• Same formula, same size, same quality transfer from Qwable

SHQ8-v2 (Compact Champion)

• Q8_0 — blk.0 (L0) + blk.26-31 attn_gate/qkv/ssm_alpha/beta, blk.31 ffn_down/attn_output, norms
• Q6_K — blk.1-25 attn_gate/qkv/ssm_alpha/beta
• IQ4_XS — ssm_out, attn_output, ffn_down (low importance)
• F16 — ssm_dt, ssm_a, all norms

• PPL 7.6542 vs Q6_K 7.5876 (+0.067, within 2× SE)
• Size 5,726 MiB vs Q6_K 7,008 MiB (18% smaller, 1,282 MiB saved)
• Larger gap on Qwythos than Qwable (+0.067 vs +0.017) — IQ4_XS slightly more costly here

Speed

Coding Examples

"I'm a dev, my audience is youth. I like a creative/tech style. Write the full website code. This HTML will be our foundation."

SHQ8-OptA — mythos-SHQ8_temp-0.6.html

• Full hero, skills, projects, about, contact, and footer sections
• Animated grid background, floating elements, glow buttons
• Mobile menu, smooth scroll, active nav highlight, form with validation
• Stagger entrance animations for cards
• External deps: Google Fonts, Font Awesome

SHQ8-v2 — mythos-SHQ8-v2_temp-0.6.html

• Hero, projects, skills, and footer — focused layout
• Cleaner CSS, fewer sections, parallax effect
• No Font Awesome — lightweight approach
• External deps: Google Fonts only

At temp 0.6, both quants produce professional code. OptA goes for completeness (more sections, more features), v2 for conciseness. The reasoning fine-tune shows its quality — no repetition, no broken markup at this temperature.

Usage

Recommended sampling: Start with temperature 0.6, top_k 20, top_p 0.95, min_p 0. If you encounter looping or over-thinking, set repeat_penalty to 1.05 — this solves both issues without touching temperature. Be cautious with high temperatures — this is a reasoning fine-tune and can get unstable above 1.2.

Personal note from wepiqx: I've found that top_p 1.0 + min_p 0.05 often produces noticeably better results than top_p 0.95 + min_p 0. The wider top_p pool with a minimum probability threshold gives the model more creative freedom while still filtering out low-probability garbage. Give it a try.

llama.cpp

VRAM usage: ~7.4 GiB / 8 GiB at -c 65536 with flash-attn + q8_0 KV cache. The Gated DeltaNet architecture is memory-efficient even at long context.

From wepiqx: This fine-tune can over-engineer at times, but overall it's a solid model. I'd love to see an MTP variant released — with MTP I could make an even better quantization with specialized draft model support for faster inference.

Ollama

1. Create a Modelfile:

FROM ./M-SHQ8-OptA-Q5_K_M.gguf

TEMPLATE """{{ .System }}
{{ .Prompt }}"""

PARAMETER num_ctx 8192
PARAMETER temperature 0.6
PARAMETER top_k 20
PARAMETER top_p 0.95
PARAMETER min_p 0

1. Build and run:

If the model loops or over-thinks, set repeat_penalty to 1.05 — this solves both issues.

LM Studio

1. Open LM Studio
2. Drag M-SHQ8-OptA-Q5_K_M.gguf into the app
3. Set GPU Offload to 99 layers
4. Enable flash-attention
5. Set context length to 8192+
6. Set sampling: temperature 0.6, top_k 20, top_p 0.95, min_p 0, repeat_penalty 1.0 (off)
7. If the model loops or over-thinks, set repeat_penalty to 1.05 — this solves both issues.
8. Start chatting

⚠️ Crucial Security & Safety Note (Uncensored Nature)

• For Users/Developers: This model does not surface safety boilerplate. It is critical to verify any specific identifiers, source code, or clinical data before execution or practical application.
• For Deployments: If you are using these SHQ8 quants in user-facing production applications, it is highly recommended to implement your own application-level moderation, review pipelines, or safety alignment layers depending on your target audience.

Files

Key Takeaways

1. OptA formula transfers perfectly — same architecture + same imatrix = same optimal quantization. PPL improved over Q6_K as expected.
2. Qwythos baseline is higher than Qwable (7.5876 vs 7.4394 at Q6_K) — 500M reasoning tokens and different eval domain (Wikitext-2).
3. v2 has a wider gap on Qwythos (+0.067 vs +0.017 on Qwable) — IQ4_XS tensors impose ~0.05 PPL extra cost on this fine-tune.
4. OptA remains the recommendation — best PPL at comfortable 6.3 GB. v2 only if 5.7 GB is required.

References

• Qwythos-9B-Claude-Mythos-5-1M
• Qwable-9B-Claude-Fable-5-SHQ8-GGUF — full methodology, importance analysis, PPL results (applies to Qwythos 1:1)