You can deploy this model and start earning money today!

Zambizi/
slim-rpcache-unsloth-gemma-4-12b-gguf-mtp-mmporj

Name: slim-rpcache-unsloth-gemma-4-12b-gguf-mtp-mmporj
Availability: InStock

Views

No views yet

Gemma 4 12B RunPod Cache Repo

This repository exists to support RunPod model caching for a custom llama.cpp Serverless worker.

Related worker repository:

REPLACE_WITH_PUBLIC_GITHUB_REPO_URL

Why this repo exists

The original model repository contains many quantized variants of the same model.

RunPod model caching currently hydrates multi-quant Hugging Face repos inefficiently for this use case, which can cause long first-time cache downloads on a new host.

This slim repo avoids that by containing only the files needed at runtime for this endpoint.

Files in this repo

This repo should contain only these 3 files:

gemma-4-12b-it-UD-Q8_K_XL.gguf
mtp-gemma-4-12b-it.gguf
mmproj-F16.gguf

What this is used for

This repo is meant to be used in two places:

The RunPod endpoint Model cache field
The worker environment variable:

LLAMA_CACHED_MODEL=Zambizi/slim-rpcache-unsloth-gemma-4-12b-gguf-mtp-mmporj

The worker then resolves these exact runtime files from RunPod's host-local Hugging Face cache:

env

LLAMA_CACHED_GGUF_PATH=gemma-4-12b-it-UD-Q8_K_XL.gguf
LLAMA_CACHED_MTP_PATH=mtp-gemma-4-12b-it.gguf
LLAMA_CACHED_MMPROJ_PATH=mmproj-F16.gguf

What this repo is NOT

This is not:

the application code repo
a training repo
a network volume replacement
a general-purpose model distribution repo

It is only a minimal artifact repo for fast and predictable RunPod cache hydration.

Why not use a network volume?

RunPod model caching does not require a network volume.

That matters because network volumes can restrict data-center placement and reduce available GPU pool options. This slim repo keeps the deployment compatible with RunPod's normal host-local cached model flow.

Source model

The files in this repo come from:

unsloth/gemma-4-12b-it-GGUF

Credits:

Original GGUF, MTP, and multimodal projector files are provided by Unsloth.
This slim cache repo is only a repackaging layer for RunPod cache hydration convenience.

Specifically:

gemma-4-12b-it-UD-Q8_K_XL.gguf
mtp-gemma-4-12b-it.gguf
mmproj-F16.gguf

Recommended worker args

The matching worker profile is:

LLAMA_SERVER_CMD_ARGS=--alias gemma-4-12b-it --ctx-size 262144 -ngl 999 -np 1 -fa auto --jinja --reasoning on --reasoning-budget 1024 --spec-type draft-mtp --spec-draft-n-max 4 --cache-reuse 256 --no-warmup --temp 1.0 --top-p 0.95 --top-k 64 --mmap

Notes

Keep the filenames exactly the same unless you also update the worker env vars.
This repo is optimized for one specific RunPod deployment target.
If you later switch quant level, replace the GGUF file and update LLAMA_CACHED_GGUF_PATH accordingly.