Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Impish_LLAMA_3B-bnb-4bit – AI Model by huggingkot | AlphaNeural AI
You can deploy this model and start earning money today!
huggingkot
/
Impish_LLAMA_3B-bnb-4bit
like
0
safetensors
SicariusSicariiStuff/Impish_LLAMA_3B
finetune
8-bit
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
This is a converted weight from
Impish_LLAMA_3B
model in
unsloth 4-bit dynamic quant
using this
collab notebook
.
About this Conversion
This conversion uses
Unsloth
to load the model in
4-bit
format and force-save it in the same
4-bit
format.
How 4-bit Quantization Works
The actual
4-bit quantization
is handled by
BitsAndBytes (bnb)
, which works under
Torch
via
AutoGPTQ
or
BitsAndBytes
.
Unsloth
acts as a wrapper, simplifying and optimizing the process for better efficiency.
This allows for reduced memory usage and faster inference while keeping the model compact.
