kennylam/

CantoneseLLM-6B-preview202402-exl2

哩個係用 /hon9kon9ize/CantoneseLLM-6B-preview202402 生成嘅exl2量化模型。
This is a quantizated model from /hon9kon9ize/CantoneseLLM-6B-preview202402 in exl2 format.
這是一個由 /hon9kon9ize/CantoneseLLM-6B-preview202402 生成的exl2量化模型。

哩度係main branch, 只係放EvLlamaV2量化果陣用到嘅measurement.json檔案，請響下面揀量化程度。
You are currently at the main branch, which provides only measurement.json used in the ExLlamaV2 quantization. Please take a look of your choices in following table of branches.
這裡是main branch, 只提供EvLlamaV2量化時所用到的measurement.json檔案，請在下面選擇量化程度。。

8.0bpw-h8 8 bits per weight.

6.0bpw-h6 6 bits per weight.

5.0bpw-h6 4 bits per weight.

4.0bpw-h6 4 bits per weight.

3.0bpw-h6 3 bits per weight.