My second merge! Yayay! Technically my third but we'll ignore the first failed model :(
This model is just headache to do, it really is! It wasted me $20 trying to fit darkc0de/BlackXorDolphTronGOAT because it's a float32 dtype!
I didn't know I can downcast the model into dfloat16 first until at the very end there.
But, this model has the downcast in the df16 directory for your pleasure.
Still testing and awaiting GGUF. Please kindly give your patience. Thank you <3
Philosophy
Mind
Fusion of darkc0de's fusions merge, and Acree's DeepSeek distill abliterated.
I think this is a good base. The Xortron is the top performing at UGI leaderboard and Arcee has the DeepSeek distill and updated world information.
Heart
Karcher with all 2503 base, using updated Dan's and Eurydice, and reintroduce Pantheon.
I feel like this will improve the roleplay. All has the models have unique characteristics of their own, and hopefully the Karcher mean can find a nice center.
Soul
Stock on Sleep's Omega Gaslight and Broken Tutu, since both based on Cydonia 2.1 and have the BlackSheep model in their merge.
I'm horny...
Merge Details
This is a merge of pre-trained language models created using mergekit.
Merge Method
This model was merged using multi-stage model merging method:
Mind was merged using the Arcee Fusion merge method using huihui-ai/Arcee-Blitz-abliterated as a base.
Heart was merged using the Karcher Mean merge method.
Soul was merged using the Model Stock merge method using TheDrummer/Cydonia-24B-v2.1 as a base.
Fially, this model was merged using the DELLA merge method using Mind as a base.
The partial merge results (Mind, Heart, and Soul) are available inside the intermediates directory.
The following YAML configuration was used to produce this model (using mergekit-multi):
yaml
1name: Mind
2merge_method: arcee_fusion
3dtype: bfloat16
4tokenizer:5source: union
6chat_template: auto
7base_model: huihui-ai/Arcee-Blitz-abliterated
8models:9-model: darkc0de/BlackXorDolphTronGOAT
10---11name: Heart
12merge_method: karcher
13tokenizer:14source: union
15chat_template: auto
16parameters:17max_iter:100018models:19-model: PocketDoc/Dans-PersonalityEngine-V1.3.0-24b
20-model: aixonlab/Eurydice-24b-v3.5
21-model: Gryphe/Pantheon-RP-1.8-24b-Small-3.122---23name: Soul
24merge_method: model_stock
25tokenizer:26source: union
27chat_template: auto
28base_model: TheDrummer/Cydonia-24B-v2.1
29models:30-model: ReadyArt/Broken-Tutu-24B
31-model: ReadyArt/Omega-Darker-Gaslight_The-Final-Forgotten-Fever-Dream-24B
32---33merge_method: della
34tokenizer:35source: union
36chat_template: auto
37base_model: Mind
38models:39-model: Mind
40-model: Heart
41parameters:42weight:0.643-model: Soul
44parameters:45weight:0.446parameters:47density:0.748epsilon:0.249lamda:1.1