merge
This is a merge of pre-trained language models created using
mergekit.
Merge Details
This version uses Halu Blackroot which has fewer GPTisms than Jamet Blackroot as well as being more easily tamed in terms of leaning towards romantic/NSFW scenarios compared to Jamet Blackroot, making this model more suitable for slow-burns.
Note: Personally, I don't think this model feels quite right, MopeyMule doesn't seem to synergize as well with Halu as I had hoped so it still feels a bit too positive, but I'll leave this up for people that prefer Halu over Jamet
The goal of this merge was to make an RP model better suited for role-plays with heavy themes such as but not limited to:
- Mental illness
- Self-harm
- Trauma
- Suicide
I hated how RP models tended to be overly positive and hopeful with role-plays involving such themes,
but thanks to
failspy/Llama-3-8B-Instruct-MopeyMule this problem has been lessened considerably.
I haven't done extensive testing yet, but the bit of testing I did do has shown noticeably good results in lessening how positive the bot is in its writing style.
If you're an enjoyer of savior/reverse savior type role-plays like myself, then this bot is for you.
Merge Method
This model was merged using several Task Arithmetic merges and then tied together with a Model Stock merge.
Models Merged
The following models were included in the merge:
-
Casual-Autopsy/Umbral-1
-
Casual-Autopsy/Penumbral
-
Casual-Autopsy/Umbral-3
Secret Sauce
The following YAML configurations were used to produce this model:
Umbral-1
1slices:
2- sources:
3 - model: Sao10K/L3-8B-Stheno-v3.2
4 layer_range: [0, 32]
5 - model: failspy/Llama-3-8B-Instruct-MopeyMule
6 layer_range: [0, 32]
7 parameters:
8 weight: 0.25
9merge_method: task_arithmetic
10base_model: Sao10K/L3-8B-Stheno-v3.2
11dtype: bfloat16
12
Penumbral
1slices:
2- sources:
3 - model: Hastagaras/Halu-8B-Llama3-Blackroot
4 layer_range: [0, 32]
5 - model: failspy/Llama-3-8B-Instruct-MopeyMule
6 layer_range: [0, 32]
7 parameters:
8 weight: 0.15
9merge_method: task_arithmetic
10base_model: Hastagaras/Halu-8B-Llama3-Blackroot
11dtype: bfloat16
Umbral-3
1slices:
2- sources:
3 - model: grimjim/Llama-3-Oasis-v1-OAS-8B
4 layer_range: [0, 32]
5 - model: failspy/Llama-3-8B-Instruct-MopeyMule
6 layer_range: [0, 32]
7 parameters:
8 weight: 0.35
9merge_method: task_arithmetic
10base_model: grimjim/Llama-3-Oasis-v1-OAS-8B
11dtype: bfloat16
Penumbral-Mind-RP-8B
1models:
2 - model: Casual-Autopsy/Umbral-1
3 - model: Casual-Autopsy/Penumbral
4 - model: Casual-Autopsy/Umbral-3
5merge_method: model_stock
6base_model: Casual-Autopsy/Umbral-1
7dtype: bfloat16
