Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
gemma-7b-it-gguf – AI Model by iAkashPaul | AlphaNeural AI
You can deploy this model and start earning money today!
iAkashPaul
/
gemma-7b-it-gguf
like
0
endpoints_compatible
gemma
gguf
template
other
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Gemma 7B Instruct GGUF
Contains Q4 & Q8 quantized GGUFs for
google/gemma
Perf
Variant
Device
Perf
Q4
RTX 2070S
22 tok/s
M1 Pro 10-core GPU
28 tok/s
Q8
RTX 2070S
7 tok/s (could only offload 23/29 layers to GPU)
M1 Pro 10-core GPU
17 tok/s
