Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
gemma-2-9b-it-abc-notation-bnb-4bit – AI Model by Seeker38 | AlphaNeural AI
You can deploy this model and start earning money today!
Seeker38
/
gemma-2-9b-it-abc-notation-bnb-4bit
like
0
safetensors
Seeker38/gemma-2-9b-it-abc-notation
finetune
8-bit
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
This is a converted weight from
gemma-2-9b-it-abc-notation
model in
unsloth 4-bit dynamic quant
using this
collab notebook
.
About this Conversion
This conversion uses
Unsloth
to load the model in
4-bit
format and force-save it in the same
4-bit
format.
How 4-bit Quantization Works
The actual
4-bit quantization
is handled by
BitsAndBytes (bnb)
, which works under
Torch
via
AutoGPTQ
or
BitsAndBytes
.
Unsloth
acts as a wrapper, simplifying and optimizing the process for better efficiency.
This allows for reduced memory usage and faster inference while keeping the model compact.
