Beta
Explore
Marketplace
Neural Labs
Chat
Wallet
Docs
Euryale-L2-70B-2.1BPW-exllama2 – AI Model by lmganon123 | AlphaNeural AI
You can deploy this model and start earning money today!
lmganon123
/
Euryale-L2-70B-2.1BPW-exllama2
like
0
transformers
safetensors
llama
text-generation
autotrain_compatible
text-generation-inference
endpoints_compatible
us
Views
No views yet
Model card
Files and Versions
Community
API
Deploy
Quantized :
https://huggingface.co/Sao10K/Euryale-L2-70B
With :
https://github.com/turboderp/exllamav2
Fits into 24GB of ram with 4096 context. Unfortunately it seems to be dumbed down a bit too much by compression. Files also include measurement.json that can be used to speed up quantization process for other BPW size.
Measurement done with default parameters and
https://huggingface.co/datasets/wikitext/tree/refs%2Fconvert%2Fparquet/wikitext-103-raw-v1/test
license: other
