This is a merge of pre-trained language models created using
mergekit.
This model was merged using the della_linear merge method using
flammenai/Mahou-1.3-mistral-nemo-12B as a base.
1models:
2 - model: flammenai/Mahou-1.3-mistral-nemo-12B
3 parameters:
4 weight: 0.5
5 density: 0.8
6 - model: nbeerbower/mistral-nemo-gutenberg-12B-v3
7 parameters:
8 weight: 0.5
9 density: 0.8
10merge_method: della_linear
11base_model: flammenai/Mahou-1.3-mistral-nemo-12B
12parameters:
13 epsilon: 0.05
14 lambda: 1
15 int8_mask: true
16dtype: bfloat16
17tokenzer_source: union
18
