Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Mistral-7B-Instruct-v0.2-AQLM-2Bit-2x8 – AI Model by ISTA-DASLab | AlphaNeural AI
You can deploy this model and start earning money today!
ISTA-DASLab
/
Mistral-7B-Instruct-v0.2-AQLM-2Bit-2x8
like
0
transformers
safetensors
mistral
text-generation
finetuned
conversational
text-generation-inference
2401.06118
autotrain_compatible
endpoints_compatible
8-bit
aqlm
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Official
AQLM
quantization of
mistralai/Mistral-7B-Instruct-v0.2
.
For this quantization, we used 2 codebooks of 8 bits.
Results:
Model
Quantization
MMLU (5-shot)
Model size, Gb
mistralai/Mistral-7B-Instruct-v0.2
None
0.5912
14.5
2x8
0.4384
2.3
