Qwen3.5-122B-A10B (uncensored by HauhauCS) quantized with APEX I-Compact — a MoE-aware mixed-precision strategy that applies layer-wise precision gradients. Edge layers get higher precision, middle layers get aggressive compression. Quantized from Q8_K_P using the APEX project.

📊 Benchmark Results

Measurements from APEX project on 8×RTX PRO 6000 Blackwell (768 GB VRAM). Perplexity on wikitext-2-raw (ctx 512). Accuracy via llama.cpp (400 tasks each).

Profile	Size	PPL	HellaSwag	Wino	MMLU	ARC	t/s
Q8_0 (ref)	121 GB	4.819	85.5%	77.3%	44.19	57.19	85.5
APEX I-Balanced	83.4 GB	4.831	85.5%	77.8%	43.86	57.86	96.7
APEX I-Compact ★	55.1 GB	4.978	84.5%	77.5%	44.06	57.86	106.3
APEX I-Mini	44.9 GB	5.306	84.0%	75.3%	42.83	56.52	110.0

★ This quantization. I-Compact achieves 84.5% HellaSwag and 57.86 ARC at 55% less size than Q8_0, fastest standard APEX profile at 106 t/s. Quantized from Q8_K_P (137 GB → 55.1 GB).

🔬 Quantization Strategy

APEX I-Compact applies layer-wise mixed-precision with MoE-aware tensor classification. Edge layers (L0–4, L43–47) get higher precision, middle layers (L10–29) get more aggressive compression. I-variants use diverse imatrix calibration (chat, code, reasoning, tool-calling, agentic traces) for better real-world accuracy.

Component	Edge (L0-4, L43-47)	Middle (L10-29)	Role
Routed Experts (exps)	Q3_K_M	Q3_K_S	256 experts, 8 active
Shared Experts (shexp)	Q4_K_S	Q4_K_S	Always active
Attention (QKV)	Q4_K_S	Q3_K_M	Per-layer attention
Router (gate_inp)	F32	F32	Precision-critical

Router weights kept in F32 (lossless) to preserve routing accuracy. Shared experts at Q4_K_S across all layers for stable token processing.

🏗️ Architecture

Base Model	HauhauCS/Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive
Parameters	122B total, ~10B active per token
Experts	256 routed + 1 shared (8 active per token)
Architecture	Hybrid: Gated DeltaNet linear attention + softmax attention (3:1)
Layers	48 (12 × (3 DeltaNet-MoE + 1 Attention-MoE))
Context	262K native
Modalities	Text, Image, Video (natively multimodal)
Vocabulary	248K tokens, 201 languages
Uncensored	Aggressive variant — 0/465 refusals
Quantized From	Q8_K_P (137 GB → 55.1 GB, 3.88 BPW effective)

⚙️ Recommended Settings

Thinking mode (default):

General	temp=1.0, top_p=0.95, top_k=20, min_p=0, presence_penalty=1.5
Coding	temp=0.6, top_p=0.95, top_k=20, min_p=0, presence_penalty=0

Non-thinking mode:

General	temp=0.7, top_p=0.8, top_k=20, min_p=0, presence_penalty=1.5
Reasoning	temp=1.0, top_p=1.0, top_k=40, min_p=0, presence_penalty=2.0

Use --jinja flag with llama.cpp. Thinking mode is on by default — disable with --chat-template-kwargs '{"enable_thinking":false}'. Vision support requires mmproj file.

📝 Usage

Works with llama.cpp, LM Studio, Jan, koboldcpp, and other GGUF-compatible runtimes.

# Text only llama-cli -m Qwen3.5-122B-A10B-Uncensored-APEX-Compact.gguf \ --jinja -c 131072 -ngl 99

# With vision llama-cli -m Qwen3.5-122B-A10B-Uncensored-APEX-Compact.gguf
--mmproj mmproj-Qwen3.5-122B-A10B-Uncensored-f16.gguf
--jinja -c 131072 -ngl 99

For CPU inference: this 55 GB model fits in 64 GB+ RAM systems. For GPU offload, adjust -ngl based on available VRAM.

🔗 Credits

Base model: HauhauCS/Qwen3.5-122B-A10B-Uncensored-HauhauCS-Aggressive (uncensored)
Quantization: APEX — Adaptive Precision for EXpert Models by mudler / LocalAI team
Runtime: Built on llama.cpp
Original model: Qwen/Qwen3.5-122B-A10B

SC117/Qwen3.5-122B-A10B-Uncensored-APEX-Compact-GGUF

⚡ Qwen3.5-122B-A10B Uncensored — APEX I-Compact GGUF

SC117/
Qwen3.5-122B-A10B-Uncensored-APEX-Compact-GGUF