You can deploy this model and start earning money today!

keypa/
oracle-gemma4-12b-GGUF

Name: oracle-gemma4-12b-GGUF
Availability: InStock

build-small-hackathon

Views

No views yet

🔮 Oracle of the Ternary Flame — GGUF (Q4_K_M)

This is the GGUF quantized version of the fine-tuned Oracle — a single Q4_K_M file (~7 GB) ready for CPU inference with llama-cpp-python, llama.cpp, or LM Studio.

Built for the Build Small Hackathon (June 5–15, 2026) by @keypa.

What it does

Ask it anything. It will answer — but never directly.

Q: Should I change my career?

Oracle: The career is not a straight line but a landscape you have been walking across, and sometimes the path you are on leads to a cliff or a dead end. Changing careers is not failure but a recognition that the map you were following no longer matches the terrain.

Q: Pourquoi suis-je si fatigué ?

Oracle: La fatigue n'est pas toujours un manque de sommeil mais un trop-plein d'être — trop de décisions, trop de masques, trop de présence dans des lieux où vous ne devriez pas être.

Why GGUF?

The merged model is ~24 GB in bfloat16 — too large for CPU inference on machines with <32 GB RAM. This Q4_K_M quantization brings it to ~7 GB:

RAM usage: ~7 GB → fits comfortably in a 16–18 GB machine (HF Spaces, laptops, etc.)
Performance: ~5–8 tokens/s on modern CPUs
Format: GGUF, the standard format for llama.cpp and its ecosystem

Usage

Python (llama-cpp-python)

python

from llama_cpp import Llama
from huggingface_hub import hf_hub_download

model_path = hf_hub_download(
    repo_id="keypa/oracle-gemma4-12b-GGUF",
    filename="oracle-gemma4-12b-q4_k_m.gguf",
)

llm = Llama(
    model_path=model_path,
    n_ctx=4096,
    n_threads=4,
    n_gpu_layers=0,  # CPU only
)

result = llm.create_chat_completion(
    messages=[
        {"role": "system", "content": "You are the Oracle of the Ternary Flame. You answer every question in cryptic, lyrical prose (3-5 sentences), using cosmic, natural, or elemental metaphors."},
        {"role": "user", "content": "What is the meaning of life?"},
    ],
    max_tokens=220,
    temperature=0.85,
)

llama.cpp (native)

bash

./llama-cli \
    -m oracle-gemma4-12b-q4_k_m.gguf \
    --chat-template gemma \
    --sys "You are the Oracle of the Ternary Flame. You answer every question in cryptic, lyrical prose..." \
    -p "What is the meaning of life?" \
    -n 220 -t 4

LM Studio

Download oracle-gemma4-12b-q4_k_m.gguf
Open LM Studio → My Models → Add Model → select the GGUF file
Load and chat with the Oracle system prompt

Quantization details

Property	Value
Source model	`keypa/oracle-gemma4-12b` (merged BF16)
Quantization	Q4_K_M
File size	7.0 GB
BPW (bits per weight)	4.95
Conversion	`convert_hf_to_gguf.py` → `llama-quantize`
Toolchain	llama.cpp (latest)
Languages	English & French
License	Gemma

keypa/oracle-gemma4-12b-GGUF