This is a merge of pre-trained language models created using
mergekit.
This model was merged using the
DARE TIES merge method using
01-ai/Yi-34B-200K as a base.
1
2models:
3 - model: 01-ai/Yi-34B-200K
4 # No parameters necessary for base model
5 - model: abacusai/Smaugv0.1
6 parameters:
7 density: 0.53
8 weight: 0.3
9 - model: jondurbin/bagel-34b-v0.2
10 parameters:
11 density: 0.53
12 weight: 0.3
13 - model: NousResearch/Nous-Hermes-2-Yi-34B
14 parameters:
15 density: 0.53
16 weight: 0.4
17merge_method: dare_ties
18base_model: 01-ai/Yi-34B-200K
19parameters:
20 int8_mask: true
21dtype: bfloat16
22
