Trying a new merge technique (sce). Early results are interesting..
This is a merge of pre-trained language models created using
mergekit.
This model was merged using the
SCE merge method using
Undi95/PsyMedRP-v1-20B as a base.
1merge_method: sce
2models:
3 - model: Undi95/MXLewd-L2-20B
4 - model: Elfrino/PsyMedLewdPass
5 - model: Elfrino/XwinXtended-20B
6base_model: Undi95/PsyMedRP-v1-20B
7tokenizer:
8 source: base
9parameters:
10 select_topk: 0.8
11dtype: float32
12out_dtype: bfloat16
13normalize: true