Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
CalliopeDS-v2-L2-13B-exl2 – AI Model by Doctor-Shotgun | AlphaNeural AI
You can deploy this model and start earning money today!
Doctor-Shotgun
/
CalliopeDS-v2-L2-13B-exl2
like
0
llama
llama-2
text-generation
en
llama2
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
CalliopeDS-v2-L2-13B-exl2
Exllama v2 quant of
Doctor-Shotgun/CalliopeDS-v2-L2-13B
Branches:
main: measurement.json calculated at 2048 token calibration rows on PIPPA
4.0bpw-h6: 4 decoder bits per weight, 6 head bits
ideal for 12gb GPUs, or 16gb GPUs with NTK extended context or CFG
6.0bpw-h6: 6 decoder bits per weight, 6 head bits
ideal for 16gb GPUs, or 24gb GPUs with NTK extended context or CFG
8bit-32g-h8: all tensors 8bit 32g, 8 head bits
experimental quant, this is with exllamav2 monkeypatched to quantize all tensors to 8bit 32g
similar in size to old GPTQ 8bit no groupsize, recommend 24gb GPU
