This is a merge of pre-trained language models created using
mergekit.
This model was merged using the breadcrumbs_ties merge method using
tavtav/eros-7b-test as a base.
1slices:
2- sources:
3 - model: tavtav/eros-7b-test
4 layer_range: [0, 32]
5 - model: Nexusflow/Starling-LM-7B-beta
6 layer_range: [0, 32]
7 parameters:
8 weight: [1, 0.686, 0.37185, 0.686, 1]
9 density: [0.9, 0.7, 0.9] # density gradient
10 gamma: [0.01, 0.03, 0.02, 0.01] # weight gradient
11tokenizer_source: base
12merge_method: breadcrumbs_ties
13base_model: tavtav/eros-7b-test
14parameters:
15 normalize: true
16dtype: bfloat16
17name: scringle
18
