An auto-regressive causal LM created by combining 2x finetuned
Llama-2 70B into one.
Please check out the quantized formats provided by
@TheBloke and
@Panchovix:
Both Vicuna and Alpaca will work, but due the initial and final layers belonging primarily to Xwin, I expect Vicuna to work the best.
The models used in the merge are
Xwin and
Euryale.
1- range 0, 16
2 Xwin
3- range 8, 24
4 Euryale
5- range 17, 32
6 Xwin
7- range 25, 40
8 Euryale
9- range 33, 48
10 Xwin
11- range 41, 56
12 Euryale
13- range 49, 64
14 Xwin
15- range 57, 72
16 Euryale
17- range 65, 80
18 Xwin
Coming soon.
Credits goes to
@chargoddard for developing the framework used to merge the model -
mergekit.
Special thanks to
@Undi95 for helping with the merge ratios.
