You can deploy this model and start earning money today!

MrOvkill/
gemma-2-inference-endpoint-GGUF

Name: gemma-2-inference-endpoint-GGUF
Availability: InStock

Views

No views yet

Gemma 2 - Inference Endpoint

NOTICE: This model does, in fact run on inference endpoints. Just click deploy, unlike with regular GGUF models. The model is no longer stored, merely linked. Enjoy <3

Code Sample ( One-Shot )

javascript

{
  "inputs": "A plain old prompt with nothing else"
}

Multi turn coming soon...

Hello! I wrote a simple container that allows for easy running of llama-cpp-python with GGUF models. My goal here was a cheap way to play with Gemma, but then I thought maybe i'd share just in case it's helpful. I'll probably make a bunch of these, so if you have any requests for GGUF or otherwise quantized Llama.cpp models to become inference endpoints, please feel free to reach out!

Files

I used the excellent quant by lmstudio-ai/gemma-2b-it-GGUF,

My email is newp@justkidding.net

Just kidding, it's sam att samuellmeyers DOT... com

MrOvkill/gemma-2-inference-endpoint-GGUF

Gemma 2 - Inference Endpoint

NOTICE: This model does, in fact run on inference endpoints. Just click deploy, unlike with regular GGUF models. The model is no longer stored, merely linked. Enjoy <3

Multi turn coming soon...

Files

MrOvkill/
gemma-2-inference-endpoint-GGUF