This is a merge of pre-trained language models created using
mergekitty.
I like Skyfall 4.2, it had some nice new prose, felt more emotionally smart, but it had some 'rough' edges for me that the previous version didn't.
This merge is exploring a mix of both to try and bring the new prose and emotional smarts to the older model with it's sharper scene and spatial memory.
I tried various percentages of each model and also different techniques, such as merging in increasing intensity across the layers, and this one turned out my favourite.
This model was merged using the SLERP merge method.
1slices:
2 - sources:
3 - model: TheDrummer/Skyfall-31B-v4.1
4 layer_range: [0, 54]
5 - model: TheDrummer/Skyfall-31B-v4.2
6 layer_range: [0, 54]
7merge_method: slerp
8base_model: TheDrummer/Skyfall-31B-v4.1
9parameters:
10 t: 0.4
11 nuslerp_flatten: false
12dtype: bfloat16
13