This is a merge of pre-trained language models created using
mergekit.
This model was merged using the passthrough merge method.
1slices:
2- sources:
3 - layer_range: [0, 16]
4 model: MarinaraSpaghetti/NemoMix-Unleashed-12B
5- sources:
6 - layer_range: [8, 24]
7 model: DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-MADNESS
8- sources:
9 - layer_range: [17, 32]
10 model: MarinaraSpaghetti/NemoMix-Unleashed-12B
11- sources:
12 - layer_range: [25, 40]
13 model: DavidAU/MN-GRAND-Gutenberg-Lyra4-Lyra-12B-MADNESS
14merge_method: passthrough
15dtype: bfloat16
