Bamboo-gemma3-12b-gguF
This is finetuned on Reverend Insanity Universe. After using it myself, Q8_0 gives the best result.
You can talk to it and it understands the world of RI. The way power works. Why people do what they do. The ideology behind Fang Yuan's choices. The logic of Gu cultivation, clan politics, the food chain of the strong and weak. It doesn't just know facts, it reasons in the framework of this world.
What it is NOT good at:
Reciting the exact canon story accurately. If you want a plot summary of specific chapters, use a wiki. This model was trained on the ideology, tone, and mechanics of the world, not on being a plot database.
What it IS good at:
Ask it why a character made a certain decision, how the power structure of a clan works, what the logic of survival of the fittest looks like in practice in this world, or just roleplay a conversation where someone actually speaks and thinks the way RI characters do. That's where it shines.
This repository contains GGUF quantized versions of a fine-tuned Gemma 3 12B model.
Model Details
- Base Model: google/gemma-3-12b-it
- Fine-tuning: Custom instruction tuning
- Quantization: Multiple GGUF formats for different use cases
Available Quantizations
| Quantization | File Size | Use Case |
|---|
| Q8_0 | ~13 GB | Highest quality, minimal loss |
| Q5_K_M | ~8.5 GB | Excellent quality/size balance (RECOMMENDED) |
| Q4_K_M | ~7 GB | Good quality, smaller size, most popular |
Usage
With llama.cpp
1# Download model
2huggingface-cli download rihuwa/Bamboo-gemma3-12b-gguF gemma-3-12b-merged-Q5_K_M.gguf --local-dir ./models
3
4# Run inference
5./llama-cli -m ./models/gemma-3-12b-merged-Q5_K_M.gguf -p "Your prompt here" -n 256
With Ollama
1# Create Modelfile
2echo 'FROM ./gemma-3-12b-merged-Q5_K_M.gguf' > Modelfile
3
4# Create model
5ollama create my-gemma3-model -f Modelfile
6
7# Run
8ollama run my-gemma3-model
With LM Studio, GPT4All, etc.
Simply download the GGUF file and load it in your preferred application.
Quantization Details
- Q8_0: 8-bit quantization, highest quality
- Q5_K_M: 5-bit with K-quant method, medium size
- Q4_K_M: 4-bit with K-quant method, smaller size
License
This model inherits the Gemma license from the base model.
Citation
1@misc{Bamboo_gemma3_12b_gguF,
2 author = {rihuwa},
3 title = {Bamboo-gemma3-12b-gguF},
4 year = {2025},
5 publisher = {HuggingFace},
6 url = {https://huggingface.co/rihuwa/Bamboo-gemma3-12b-gguF}
7}
