EXL2 quants of alpindale/goliath-120b (
https://huggingface.co/alpindale/goliath-120b), to be used on exllamav2.
Update 06/01/2024: Updated with new quant method after some time, thanks for the measurement
hereCalibration dataset is a cleaned, fixed pippa RP dataset, which does affect the results (in favor) for RP usage. You can find the calibration dataset
here.I've added a measurement.json file on the main branch if you want to do your own quants.
An auto-regressive causal LM created by combining 2x finetuned
Llama-2 70B into one.
Please check out the quantized formats provided by
@TheBloke and
@Panchovix:
Both Vicuna and Alpaca will work, but due the initial and final layers belonging primarily to Xwin, I expect Vicuna to work the best.
The models used in the merge are
Xwin and
Euryale.
1- range 0, 16
2 Xwin
3- range 8, 24
4 Euryale
5- range 17, 32
6 Xwin
7- range 25, 40
8 Euryale
9- range 33, 48
10 Xwin
11- range 41, 56
12 Euryale
13- range 49, 64
14 Xwin
15- range 57, 72
16 Euryale
17- range 65, 80
18 Xwin
Coming soon.
Credits goes to
@chargoddard for developing the framework used to merge the model -
mergekit.
Special thanks to
@Undi95 for helping with the merge ratios.
