This is a merge of pre-trained language models created using
mergekit.
I don't know anything about merges, so this may be a stupid method, but I was curious how the models would be merged if I took this approach.
This model was merged using the SLERP merge method.
This model is a model that first merges Model
Orochi with Model
dolphin with a 0.15 SLERP option, and then merges Model
BagelMIsteryTour with a 0.2 SLERP option based on the merged model.
1base_model:
2 model:
3 path: ../maid-yuzu-v7-base
4dtype: bfloat16
5merge_method: slerp
6parameters:
7 t:
8 - value: 0.2
9slices:
10- sources:
11 - layer_range: [0, 32]
12 model:
13 model:
14 path: ../maid-yuzu-v7-base
15 - layer_range: [0, 32]
16 model:
17 model:
18 path: ycros/BagelMIsteryTour-v2-8x7B
