Huihui-Qwen3.6-35B-A3B-abliterated MXFP4 MOE

Original Model

• Base model: huihui-ai/Huihui-Qwen3.6-35B-A3B-abliterated
• Architecture: Qwen3.6 MoE (Mixture of Experts)
• Base version: bf16
• License: apache-2.0

Quantized Model

• File: Huihui-Qwen3.6-35B-A3B-abliterated-bf16_MXFP4_MOE.gguf
• Format: GGUF (llama.cpp)
• Quantization: MXFP4 (4-bit Mixed Precision Floating Point for MoE)
• Type: MoE-optimized

Specifications

About MXFP4

• Uses block floating point representation (per-block scaling)
• Maintains precision in sensitive tensors
• Optimizes expert weights while keeping bf16 in critical layers
• Compatible with llama.cpp and GGUF runners

Tensor Types Preserved in BF16

• Token embeddings (token_embd.weight)
• Normalization layers (attn_norm, ssm_norm, ffn_norm, etc.)
• MoE gate weights (ffn_gate_inp, ffn_gate_inp_shexp)
• Expert selection vectors
• Biases and normalization parameters
• Full attention layers

How to Use with llama.cpp

Prerequisites

Basic Usage

• Use the F32 version of the mmproj file for optimal results. Recommended quality ranking: F32 > BF16 > F16.

Usage Warning

This is an abliterated model - Safety filtering has been significantly reduced. Use with caution.

Downloads

• Original model on Hugging Face