This repository is designed for users who want to quickly download the recommended quantization without having to select from a larger multi-file repository.

📦 Looking for other quantizations? If you need different sizes (like Q3_K_M, Q8_0, etc.), please visit the main repository: [https://huggingface.co/Abiray/Qwen3.6-35B-A3B-heretic-GGUF).

Usage with llama.cpp

You can run this model using the latest versions of llama.cpp (built via cmake).

1. CLI Inference

To run inference via the command line:

./build/bin/llama-cli -m Qwen3.6-35B-A3B-heretic-Q4_K_M.gguf -p "Explain the concept of quantum entanglement." -n 512

Abiray/Qwen3.6-35B-A3B-heretic-Q4_K_M-GGUF

Qwen3.6-35B-A3B-heretic Q4_K_M GGUF

Usage with llama.cpp

1. CLI Inference

Abiray/
Qwen3.6-35B-A3B-heretic-Q4_K_M-GGUF