Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Meta-Llama-3.1-70B-Instruct-AQLM-PV-2Bit-1x16 – AI Model by ISTA-DASLab | AlphaNeural AI
You can deploy this model and start earning money today!
ISTA-DASLab
/
Meta-Llama-3.1-70B-Instruct-AQLM-PV-2Bit-1x16
like
0
transformers
safetensors
llama
text-generation
aqlm
facebook
meta
llama-3
conversational
text-generation-inference
2401.06118
2405.14852
meta-llama/Llama-3.1-70B-Instruct
quantized
autotrain_compatible
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Official
AQLM
quantization of
meta-llama/Meta-Llama-3.1-70B-Instruct
finetuned with
PV-Tuning
.
For this quantization, we used 1 codebook of 16 bits and groupsize of 8.
Results:
Model
Quantization
MMLU (5-shot)
ArcC
ArcE
Hellaswag
PiQA
Winogrande
Model size, Gb
fp16
0.8213
0.6246
0.8683
0.6516
0.8313
0.7908
141
1x16g8
0.7814
0.5478
0.8270
0.6284
0.8036
0.7814
21.9
