Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Meta-Llama-3-70B-Instruct-AQLM-2Bit-1x16 – AI Model by ISTA-DASLab | AlphaNeural AI
You can deploy this model and start earning money today!
ISTA-DASLab
/
Meta-Llama-3-70B-Instruct-AQLM-2Bit-1x16
like
0
transformers
safetensors
llama
text-generation
facebook
meta
llama-3
conversational
text-generation-inference
2401.06118
autotrain_compatible
endpoints_compatible
aqlm
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Official
AQLM
quantization of
meta-llama/Meta-Llama-3-70B-Instruct
.
For this quantization, we used 1 codebook of 16 bits.
Results (measured with
lm_eval==4.0
):
Model
Quantization
MMLU (5-shot)
ArcC
ArcE
Hellaswag
Winogrande
PiQA
Model size, Gb
meta-llama/Meta-Llama-3-70B
-
0.7980
0.6160
0.8624
0.6367
0.8183
0.7632
141.2
1x16
0.7587
0.4863
0.7668
0.6159
0.7481
0.7537
21.9
