gemma-4-31b-larkspur-v0.5-Q6_K – AI Model by finis-est | AlphaNeural AI

You can deploy this model and start earning money today!

finis-est/
gemma-4-31b-larkspur-v0.5-Q6_K

like

0

trashpanda-org/gemma-4-31b-larkspur-v0.5

endpoints_compatible

Views

No views yet

Gemma 4 31B Larkspur v0.5 — Q6_K GGUF

Q6_K quantization of trashpanda-org/gemma-4-31b-larkspur-v0.5.

Quant Details

Property	Value
Source	`trashpanda-org/gemma-4-31b-larkspur-v0.5`
Quant	Q6_K (6.56 BPW)
Size	~24 GB
Format	GGUF (llama.cpp)
Original Precision	bf16

Usage

Load with any llama.cpp-compatible runtime (llama.cpp, KoboldCpp, ollama, LM Studio, etc.):

llama-cli -m gemma-4-31b-larkspur-v0.5-Q6_K.gguf -p "Your prompt here"

Notes

Quantized from the bf16 source weights using llama.cpp's convert_hf_to_gguf.py → llama-quantize
Q6_K preserves near-original quality at ~41% of the bf16 size