Made with Exllamav2 0.3.1 with default dataset.
These quants require RTX GPU on Windows or RTX/ROCm GPU on Linux and can be used with TabbyAPI or Text-Generation-WebUI.
They require to be fully loaded into your GPU, if you have to rely on system RAM then GGUF quants is a better choice.
