This is a custom-quantized version of Gemma4-31B, specifically optimized to obtain the highest possible local byte-intelligence ratio with 24GB+ RAM consumer laptops or computers.

🧠 Why this model is different

Unlike a standard quant, this model was processed using a custom Importance Matrix (imatrix). The training data for the imatrix was hand-curated to preserve:

Incredible reasoning: Inclusion of custom coding examples built with frontier models provides high retention of very specific and sharp architectural reasoning skills
Logical Flow: Inclusion of llama.cpp source code, logic puzzles, and historical writing in the imatrix training to ensure the model stays coherent at low bitrates.
High Speed: Built using llama.cpp specifically for local-first AI and edge computing setups like apple silicon with minimum 24GB RAM

🛠 Quantization Details

Base Model: Gemma4-31B
Quantization: IQ3_M
Format: GGUF
Size: ~14.42 GB
Context Length: 262144 tokens

⚙️ Recommended Inference Settings

Optimize for balance between creativity and coherence:

--repeat-penalty: 1.1 – 1.4 (Sweet spot! Pushes away from familiar loops. >1.5 causes "robot-speak".)
--repeat-last-n: 128 – 256 (Larger window ensures the model doesn't forget recent repetitions.)
--temperature: 0.7 – 0.8 (Prevents over-committing to safe/repetitive tokens.)
--top-p: 0.90 (Trims low-probability hallucinations without killing creativity.)
--min-p: 0.05 – 0.1 (Optional: Prunes very low-probability tokens if your backend supports it.)

📈 Perplexity Benchmarks

I attempted to run perplexity tests as normal, but this model is strange. The numbers for perplexity were incredibly high (lower is better) so I am choosing not to include the data...