Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Mistral-7B-Instruct-v0.2-AQLM-2Bit-1x16 – AI Model by alpindale | AlphaNeural AI
You can deploy this model and start earning money today!
alpindale
/
Mistral-7B-Instruct-v0.2-AQLM-2Bit-1x16
like
0
transformers
pytorch
mistral
text-generation
conversational
autotrain_compatible
text-generation-inference
endpoints_compatible
aqlm
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Took 42 hours to quantize on 4xA40s, at a batch size of 128. I could've went higher, but hindsight. At that batch size, it was using about 25-30 GiB per GPU, utilization remained at 100%.
