You can deploy this model and start earning money today!

lym00/
gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test

Name: gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test
Availability: InStock

Views

No views yet

[!NOTE] AI Disclosure
This file was generated by an AI Agent, but tested and verified by a human.

Drafter/Assistant: https://huggingface.co/google/gemma-4-E2B-it-qat-q4_0-unquantized-assistant/tree/main

llama.cpp support: https://github.com/ggml-org/llama.cpp/pull/24282

Asked Hermes to convert the Gemma 4 E2B Assistant safetensors to GGUF.

Tested with llama.cpp

ini

[HauhauCS/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive]
alias = HauhauCS/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive
model = /root/.cache/llama.cpp/HauhauCS/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive-Q4_K_P.gguf
mmproj = /root/.cache/llama.cpp/HauhauCS/Gemma-4-E2B-Uncensored-HauhauCS-Aggressive/mmproj-Gemma-4-E2B-Uncensored-HauhauCS-Aggressive-f16.gguf
spec-draft-model = /root/.cache/llama.cpp/mtp/gemma-4-E2B-it-qat-assistant-q4_0.gguf
temperature = 1.0
top-k = 64
top-p = 0.95
spec-type = draft-mtp,ngram-mod,ngram-map-k4v
spec-draft-n-max = 3
spec-draft-p-min = 0.50
spec-draft-prio = 2
spec-draft-prio-batch = 2
spec-ngram-mod-n-match = 24
spec-ngram-mod-n-min = 48
spec-ngram-mod-n-max = 64
spec-ngram-map-k4v-size-n = 8
spec-ngram-map-k4v-size-m = 24
spec-ngram-map-k4v-min-hits = 2

Now getting peak 76 tokens/second with Gemma 4 E2B + MTP (baseline was around ~40 t/s)

lym00/gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test

lym00/
gemma-4-E2B-it-qat-q4_0-unquantized-assistant-gguf-test