This repo contains a local MXFP4_MOE GGUF conversion of HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive, prepared with a patched local llama.cpp toolchain.

Files

hauhau_qwen36_35b_a3b_mxfp4_gguf.gguf: text-only GGUF export in robinshao-style MXFP4 layout
llama.cpp.src-patched.zip: patched local llama.cpp source snapshot used for conversion and serving
build_result.json: conversion and quantization summary
api-benchmark.json: basic local API benchmark output

Quantization Notes

Source FP16 weights: khoi11/Qwen3.6-35B-A3B-Uncensored-HauhauCS-FP16
Upstream model lineage: HauhauCS/Qwen3.6-35B-A3B-Uncensored-HauhauCS-Aggressive
Converted to GGUF with local convert_hf_to_gguf.py --outtype f16 --no-mtp
Vision tensors are not included in the final GGUF
MTP tensors are not included in the final GGUF
Quantized with local llama-quantize using MXFP4_MOE --pure --output-tensor-type q8_0 --token-embedding-type q8_0

Tensor Layout

json

{
  "tensor_count": 733,
  "type_counts": {
    "Q8_0": 2,
    "F32": 301,
    "MXFP4": 430
  },
  "output_bytes": 19041835392
}

Local Deployment Notes

This file was validated locally with patched llama.cpp on Windows using:

bash

llama-server -m hauhau_qwen36_35b_a3b_mxfp4_gguf.gguf \
  --jinja \
  --device CUDA0 \
  --gpu-layers all \
  --ctx-size 110592 \
  --batch-size 768 \
  --ubatch-size 128 \
  --flash-attn on

Local API benchmark on an RTX 4000 Ada at ctx-size=110592 observed roughly 103-105 tok/s decode speed for short requests.

Caveats

This is a text-only export, not the original multimodal package.
Native training context metadata remains 262144, but the local validation run used 110592.
Behavior quality and multilingual stability may vary by prompt and runtime configuration.

robinshao/hauhau_qwen36_35b_a3b_mxfp4_gguf

hauhau_qwen36_35b_a3b_mxfp4_gguf

Files

Quantization Notes

Tensor Layout

Local Deployment Notes

Caveats

robinshao/
hauhau_qwen36_35b_a3b_mxfp4_gguf