This is a merge of pre-trained language models created using
mergekit.
This model was merged using the della merge method using
TheDrummer/Theia-21B-v1 as a base.
1models:
2 - model: athirdpath/model034
3 parameters:
4 weight: 0.35
5 - model: athirdpath/modeldpo_008
6 parameters:
7 weight: 0.30
8 - model: athirdpath/model028
9 parameters:
10 weight: 0.35
11merge_method: della
12base_model: TheDrummer/Theia-21B-v1
13tokenizer_source: athirdpath/model034
14parameters:
15 density: 0.7
16 lambda: 1.1
17 epsilon: 0.2
