SLERP (Spherical Linear Interpolation) via
mergekit.
1slices:
2 - sources:
3 - model: TheDrummer/Rocinante-X-12B-v1
4 layer_range: [0, 12]
5
6 - sources:
7 - model: TheDrummer/Rocinante-X-12B-v1
8 layer_range: [12, 28]
9 parameters:
10 weight: 0.80
11 - model: WokeAI/Tankie-DPE-12B-SFT-v2
12 layer_range: [12, 28]
13 parameters:
14 weight: 0.20
15
16 - sources:
17 - model: TheDrummer/Rocinante-X-12B-v1
18 layer_range: [28, 40]
19
20merge_method: slerp
21base_model: TheDrummer/Rocinante-X-12B-v1
22
23parameters:
24 t:
25 - value: 0.20
26
27dtype: bfloat16
Inherits from base models. Please refer to the respective model cards for full license details.
Shared for personal research use only.