Small perplexity test:
before quantization - 'mean_perplexity': 10.241
after quantization - 'mean_perplexity': 10.379
As this is a hastily thrown together quant with no prior experience in quants, use
https://huggingface.co/TheBloke version if he releases a quant for this model.
