This repository contains quantized
GGUF formats of the
Google Gemma 4 31B Instruction-Tuned model, published and heavily optimized by
CQ Systems.
⚡ Mac Neo & Apple Silicon Compatibility
These files are optimized for local inference on professional hardware. Due to the 31-billion parameter size of this model, it is recommended to run these on machines with substantial unified memory (such as high-end MacBook Pros, Mac Studios, or Mac Pros).
By utilizing Apple Silicon's advanced unified memory matrix and native processing capabilities via Metal, these quantizations deliver fast, on-device reasoning and frontier-level cognition without relying on cloud infrastructure.
📦 Available Files
We provide multiple versions of the model, allowing you to scale based on your available unified memory (RAM/VRAM) constraints and latency requirements.
| Filename | Size | Quantization | Recommendation / Notes |
|---|
| CQ-Google-Gemma-4-31B-f16-Q4_K.gguf | 18.7 GB | Q4_K | 🏆 Recommended. Excellent balance of reasoning quality and speed. Requires a machine with at least 32GB of total unified memory to run comfortably with context. |
| CQ-Gemma-4-31B-q8_0.gguf | 32.6 GB | Q8_0 | High-fidelity 8-bit integer quantization. Near-uncompressed reasoning quality. Requires 64GB+ of unified memory. |
| CQ-Google-Gemma-4-31B-f16.gguf | 61.4 GB | F16 | Baseline uncompressed 16-bit float. Highest absolute fidelity but immense memory footprint. Useful for research, fine-tuning, or requantizing. Requires 96GB+ RAM. |
🚀 How to Use
Using llama.cpp
You can run this model directly in your terminal using the compiled llama.cpp CLI tool. Ensure you have built llama.cpp with Metal support for Macs.
# Basic chat launch with the recommended Q4_K model
./llama-cli -m CQ-Google-Gemma-4-31B-f16-Q4_K.gguf \
-c 8192 \
-n 1024 \
-p "<|think|>\nAnalyze the system architecture.\n<|channel>thought\n"
Using LM Studio / Ollama
- Download the .gguf file matching your system's capabilities.
- LM Studio: Drag and drop the downloaded file into your LM Studio model folder, or use the local import feature.
- Ollama: Create a Modelfile with the line FROM ./CQ-Google-Gemma-4-31B-f16-Q4_K.gguf, then run ollama create cq-gemma-31b -f Modelfile.
(Note: While the base Gemma 4 31B supports audio and image inputs natively, standard GGUF text-generation primarily supports text out-of-the-box. Multimodal capabilities in GGUF require an accompanying mmproj encoder file).
🧠 About the Base Model: Gemma 4 31B
Gemma 4 is a family of open models built by Google DeepMind. The 31B variant is the flagship heavyweight model in the family, designed to handle highly complex reasoning, advanced coding, and nuanced logic comparable to frontier closed-source models.
- Total Parameters: 31B Dense
- Context Length: 128K tokens
- Supported Modalities (Base Model): Text, Image, Audio
- System Prompts: Gemma 4 uses standard system, assistant, and user roles natively.
Thinking Mode Configuration
Gemma 4 models are designed as highly capable reasoners. To enable the built-in reasoning capabilities, include the <|think|> token at the start of your system prompt.
When thinking is enabled, the model will output its internal reasoning followed by the final answer using this structural formatting:
<|channel>thought\n**[Internal reasoning]<channel|>[Final answer]**
📜 License & Acknowledgements
- Base Model: Google DeepMind
- License: Gemma License
- Quantization & Optimization: CQ Systems Team
For full benchmark details, evaluations, ethical considerations, and safety metrics, please refer to the original
Google Gemma 4 Documentation.
