A test merge to balance the coherency of Behemoth v1, Drummer's Uoh on v1.1 and Magnum's prose. Unsure if task_arithmetic works.
This is a merge of pre-trained language models created using
mergekit.
This model was merged using the
task arithmetic merge method using
TheDrummer/Behemoth-123B-v1 as a base.
1models:
2 - model: TheDrummer/Behemoth-123B-v1
3 parameters:
4 weight: 0.425
5 - model: TheDrummer/Behemoth-123B-v1.1
6 parameters:
7 weight: 0.425
8 - model: anthracite-org/magnum-v4-123b
9 parameters:
10 weight: 0.15
11merge_method: task_arithmetic
12base_model: TheDrummer/Behemoth-123B-v1
13dtype: bfloat16