This is lumikabra. It's based on
Mistral-Large-Instruct-2407 , merged with Magnum-v2-123B, Luminum-v0.1-123B and Tess-3-Mistral-Large-2-123B.
I shamelessly took this idea from
FluffyKaeloky. Like him, i always had my troubles with each of the current large mistral based models.
Either it gets repetitive, shows too many GPTisms, is too horny or too unhorny. RP and storytelling is always a matter of taste, and i found myself swiping too often for new answers or even fixing them when I missed a little spice or cleverness.
Luminum was a great improvement, mixing a lot of desired traits, but I still missed some spice, another sauce.
So i took Luminum, added magnum again and also Tess for knowledge and structure.
This is a second version with another mixture of the same sauce. It is different than v0.1, not worse not better, just a little different. Again, I believe it is just a matter of taste, which answers one prefers and like the most.
This model was merged using
mergekit with the della_linear merge method using mistralai_Mistral-Large-Instruct-2407 as a base.
1models:
2 - model: anthracite-org_magnum-v2-123b
3 parameters:
4 weight: 0.24
5 density: 0.5
6 - model: FluffyKaeloky_Luminum-v0.1-123B
7 parameters:
8 weight: 0.34
9 density: 0.8
10 - model: migtissera_Tess-3-Mistral-Large-2-123B
11 parameters:
12 weight: 0.24
13 density: 0.9
14merge_method: della_linear
15base_model: mistralai_Mistral-Large-Instruct-2407
16parameters:
17 epsilon: 0.05
18 lambda: 1
19 int8_mask: true
20dtype: bfloat16
