Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Mixtral-8x7B-Instruct-v0_1-AQLM-2Bit-1x16-hf – AI Model by ISTA-DASLab | AlphaNeural AI
You can deploy this model and start earning money today!
ISTA-DASLab
/
Mixtral-8x7B-Instruct-v0_1-AQLM-2Bit-1x16-hf
like
0
transformers
safetensors
mixtral
text-generation
conversational
2401.06118
autotrain_compatible
text-generation-inference
endpoints_compatible
aqlm
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Official
AQLM
quantization of
mistralai/Mixtral-8x7B-Instruct-v0.1
.
For this quantization, we used 1 codebook of 16 bits.
Selected evaluation results for this model:
Model
AQLM scheme
WinoGrande
PiQA
HellaSwag
ArcE
ArcC
Model size, Gb
Hub link
Mixtral-8x7B-Instruct-v0.1 (THIS)
1x16
0.7593
0.8043
0.6179
0.7768
0.4793
12.6
Link
To learn more about the inference, as well as the information on how to quantize models yourself, please refer to the
official GitHub repo
.
