A merge of Cydonia came out and while I'm happy with Cydonia as a current daily-driver kind of model, I'd be remiss if I didn't keep an eye out for something new that might be fun.
A merge of Cydonia and Mistral Small Gutenberg.
This will hopefully make it and even better story teller.
Mistral or ChatML format.
Much appreciation to the original model creators The Drummer and nbeerbower.
This is a merge of pre-trained language models created using
mergekit.
This model was merged using the SLERP merge method.
1base_model: nbeerbower/Mistral-Small-Gutenberg-Doppel-22B
2dtype: bfloat16
3merge_method: slerp
4parameters:
5 t:
6 - filter: self_attn
7 value: [0.0, 0.5, 0.3, 0.7, 1.0]
8 - filter: mlp
9 value: [1.0, 0.5, 0.7, 0.3, 0.0]
10 - value: 0.5
11slices:
12- sources:
13 - layer_range: [0, 56]
14 model: TheDrummer/Cydonia-22B-v1
15 - layer_range: [0, 56]
16 model: nbeerbower/Mistral-Small-Gutenberg-Doppel-22B
