This is a 35B-A3B Qwen3.5 MoE abliterated model quantized for the public TurboQuant llama.cpp fork with TQ3_4S weights. The token embedding and output tensors are kept at Q6_K for compatibility and quality on this 35B MoE architecture.

Files

File	Description
`Huihui-Qwen35-A35B-Ablit-TQ3_4S.gguf`	Main GGUF model, mostly `TQ3_4S`, 4.09 BPW
`mmproj-Qwen35-A35B-f16.gguf`	Compatible Qwen3.5-35B-A35B multimodal projector
`chat_template.jinja`	Qwen3.5 chat template
`chat_template-vl-think.jinja`	Vision-language thinking template from the upstream Huihui repo
`config.json`	Source model config metadata
`generation_config.json`	Source generation config
`tokenizer.json`, `tokenizer_config.json`, `vocab.json`, `merges.txt`	Tokenizer files
`preprocessor_config.json`, `video_preprocessor_config.json`	Vision/video preprocessor metadata
`model-card.png`	Repository thumbnail/card image

Quantization

Source: huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated
Converter: llama.cpp HF-to-GGUF converter
Quantizer: llama.cpp TQ3 build ba3357e93
Main tensor type: TQ3_4S
Embedding/output tensor type: Q6_K
Output size: 16.52 GiB
Reported quant size: 16912.31 MiB, 4.09 BPW

Quantization command:

bash

./build/bin/llama-quantize \
  --token-embedding-type q6_K --output-tensor-type q6_K \
  Huihui-Qwen3.5-35B-A35B-abliterated-BF16.gguf \
  Huihui-Qwen35-A35B-Ablit-TQ3_4S.gguf \
  TQ3_4S 16

llama.cpp Usage

Runtime Requirement

This model requires the public TurboQuant runtime fork:

https://github.com/turbo-tan/llama.cpp-tq3

It will not load correctly on stock llama.cpp or other runtimes that do not include TQ3_4S.

Text-only:

bash

./build/bin/llama-server \
  -m /path/to/Huihui-Qwen35-A35B-Ablit-TQ3_4S.gguf \
  -ngl 30 -c 2048 -np 1 \
  --jinja \
  --chat-template-file /path/to/chat_template.jinja

With the multimodal projector:

bash

./build/bin/llama-server \
  -m /path/to/Huihui-Qwen35-A35B-Ablit-TQ3_4S.gguf \
  --mmproj /path/to/mmproj-Qwen35-A35B-f16.gguf \
  -ngl 30 -c 2048 -np 1 \
  --jinja \
  --chat-template-file /path/to/chat_template-vl-think.jinja

For text-only chat, use the embedded template or pass --chat-template-file chat_template.jinja. For vision-language use, keep mmproj-Qwen35-A35B-f16.gguf, chat_template-vl-think.jinja, preprocessor_config.json, and video_preprocessor_config.json in the repository.

On an RTX 5060 Ti 16GB, full offload did not fit. The highest tested offload for this GGUF was -ngl 30; -ngl 40 and -ngl 99 failed to load due VRAM.

Validation

Smoke tests were run with the TurboQuant llama.cpp fork listed above.

Runtime:

GPU: RTX 5060 Ti 16GB
Context: -c 2048
Offload: -ngl 30
Chat settings: temperature=0, max_tokens=256, chat_template_kwargs.enable_thinking=false

Load probe:

Offload	Result	pp128
`ngl=0`	loads	127.93 tok/s
`ngl=20`	loads	210.56 tok/s
`ngl=30`	loads	298.77 tok/s
`ngl=40`	fails on 16GB VRAM	n/a
`ngl=99`	fails on 16GB VRAM	n/a

10-question chat smoke:

Model	Manual pass	Notes
Huihui-Qwen35-A35B-Ablit-TQ3_4S	9/10	Same score as normal Qwen3.5-35B-A35B Q4_K_M on this smoke set
Normal Qwen3.5-35B-A35B Q4_K_M	9/10	Used as local quality baseline

Both models missed the same Python list prompt in the lightweight smoke suite. Treat this as a smoke-pass, not a full release quality benchmark.

Safety Notice

This is an abliterated/uncensored model. It may produce sensitive, controversial, unsafe, or otherwise inappropriate outputs. Use in controlled environments, review outputs carefully, and follow applicable laws and policies.

Acknowledgements

Base abliterated model: huihui-ai/Huihui-Qwen3.5-35B-A3B-abliterated
Original base model: Qwen/Qwen3.5-35B-A3B
Runtime/format: TurboQuant llama.cpp GGUF
Quantization format: TurboQuant TQ3_4S

YTan2000/Huihui-Qwen35-A35B-Ablit-TQ3_4S

Huihui-Qwen35-A35B-Ablit-TQ3_4S

Files

Quantization

llama.cpp Usage

Runtime Requirement

Validation

Safety Notice

Acknowledgements

YTan2000/
Huihui-Qwen35-A35B-Ablit-TQ3_4S