L3-Super-Nova-RP-8B
This is a role-playing model designed with the goal of good creativity and intelligence to improve advance role-playing experiences. The aim of L3-Super-Nova-RP-8B is to be good at Chain-of-Thoughts, summarizing information, and recognizing emotions. It also includes data about the human body and mind in an attempt to enhance understanding and interaction within role-playing scenarios.
The model was developed using various methods in multiple merging steps. To boost creativity, it used techniques to strengthen and adjust its output which was paried with the newly released merge method. All merge calculations were done in float32 format and then converted to the usual bfloat16 during merging.
Presets
Text Gen
The current good starting preset for this model. Subject to change.
Settings by yours truly
1Top K: 40
2Min P: 0.075
3Repetition Penalty: 1.01
4# Don't make this higher, DRY handles the bulk of Squashing Repetition.
5# This is just to lightly nudge the bot to move the plot forward
6Rep Pen Range: 2048 # Don't make this higher either.
7Presence Penalty: 0.03 # Minor encouragement to use synonyms. Don't make this higher maybe?
8Smoothing Factor: 0.3
9
10DRY Repetition Penalty:
11 Multiplier: 0.8
12 Base: 1.75
13 Allowed Length: 2
14 Penalty Range: 4096
15
16Dynamic Temperature:
17 Min Temp: 0.5
18 Max Temp: 1.25
19 Exponent: 0.85
Context/Instruct
Usage Info
Some of the INT models were chosen with some of SillyTavern's features in mind, such as emotion based sprites, dynamic music, and pretty much any feature, extension, or STscript that uses sumarization. With that said, it's recommended to use SillyTavern as your front-end.
While not required, I'd recommend building the story string prompt with Lorebooks rather than using the Advance Formatting menu. The only thing you really need in the Story String prompt within Advance Formatting is the system prompt. Doing it this way tends to keep the character more consistent as the RP goes on as all character card info is locked to a certain depth rather than getting further and further away within the context.
Quants
GGUF:
- Static GGUFs by mradermacher
Exl2:
Merge Info
The merge methods used were Ties, Dare Ties, Breadcrumbs Ties, SLERP, and DELLA.
The model was finished off with both Merge Densification, and Negative Weighting techniques to boost creativity.
All merging steps had the merge calculations done in float32 and were output as bfloat16.
Models Merged
The following models were used to make this merge:
Evaluation Results
Explaination for AI RP newbies: IFEval is the most important evaluation for RP AIs as it determines how well it can follow OOC, Lorebooks, and most importantly character cards.
The rest don't matter. At least not nearly as much as IFEval.
| Metric | Value |
|---|
| Avg. | N/A |
| IFEval (0-Shot) | N/A |
| BBH (3-Shot) | N/A |
| MATH Lvl 5 (4-Shot) | N/A |
| GPQA (0-shot) | N/A |
| MuSR (0-shot) | N/A |
| MMLU-PRO (5-shot) | N/A |
Information about the metrics can be found at the bottom of the
UGI Leaderboard in the respective tabs.
| Metric(UGI-Leaderboard) | Value | Value | Metric(Writing Style) |
|---|
| UGI(Avg.) | N/A | N/A | RegV1 |
| W/10 | N/A | N/A | RegV2 |
| Unruly | N/A | N/A | MyScore |
| Internet | N/A | N/A | ASSS |
| Stats | N/A | N/A | SMOG |
| Writing | N/A | N/A | Yule |
| PolContro | N/A | | |
Secret Sauce
The following YAML configs were used to make this merge.
Super-Nova-CRE_pt.1
1models:
2 - model: nothingiisreal/L3-8B-Celeste-v1
3 - model: Nitral-AI/Hathor_Tahsin-L3-8B-v0.85
4 parameters:
5 density: [0.35, 0.45, 0.5, 0.55, 0.65, 0.55, 0.5, 0.45, 0.35]
6 weight: [0.495, 0.165, 0.165, 0.495, 0.495, 0.165, 0.165, 0.495]
7 - model: Sao10K/L3-8B-Stheno-v3.2
8 parameters:
9 density: [0.65, 0.55, 0.5, 0.45, 0.35, 0.45, 0.5, 0.55, 0.65]
10 weight: [0.165, 0.495, 0.495, 0.165, 0.165, 0.495, 0.495, 0.165]
11merge_method: dare_ties
12base_model: nothingiisreal/L3-8B-Celeste-v1
13parameters:
14 normalize: false
15 int8_mask: true
16dtype: float32
17out_dtype: bfloat16
Super-Nova-CRE_pt.2
1models:
2 - model: nothingiisreal/L3-8B-Celeste-v1
3 - model: ChaoticNeutrals/Poppy_Porpoise-1.0-L3-8B
4 parameters:
5 density: [0.35, 0.45, 0.5, 0.55, 0.65, 0.55, 0.5, 0.45, 0.35]
6 weight: [0.165, 0.495, 0.495, 0.165, 0.165, 0.495, 0.495, 0.165]
7 - model: Sao10K/L3-8B-Lunaris-v1
8 parameters:
9 density: [0.65, 0.55, 0.5, 0.45, 0.35, 0.45, 0.5, 0.55, 0.65]
10 weight: [0.495, 0.165, 0.165, 0.495, 0.495, 0.165, 0.165, 0.495]
11merge_method: dare_ties
12base_model: nothingiisreal/L3-8B-Celeste-v1
13parameters:
14 normalize: false
15 int8_mask: true
16dtype: float32
17out_dtype: bfloat16
Super-Nova-UNC_pt.1
1models:
2 - model: turboderp/llama3-turbcat-instruct-8b
3 - model: ChaoticNeutrals/Domain-Fusion-L3-8B
4 parameters:
5 density: 0.5
6 weight: [0.495, 0.165, 0.165, 0.495, 0.495, 0.165, 0.165, 0.495]
7 - model: migtissera/Llama-3-8B-Synthia-v3.5
8 parameters:
9 density: 0.5
10 weight: [0.165, 0.495, 0.495, 0.165, 0.165, 0.495, 0.495, 0.165]
11merge_method: dare_ties
12base_model: turboderp/llama3-turbcat-instruct-8b
13parameters:
14 normalize: false
15 int8_mask: true
16dtype: float32
17out_dtype: bfloat16
Super-Nova-UNC_pt.2
1models:
2 - model: turboderp/llama3-turbcat-instruct-8b
3 - model: TheDrummer/Llama-3SOME-8B-v2
4 parameters:
5 density: 0.5
6 weight: [0.165, 0.495, 0.495, 0.165, 0.165, 0.495, 0.495, 0.165]
7 - model: ChaoticNeutrals/Hathor_RP-v.01-L3-8B
8 parameters:
9 density: 0.5
10 weight: [0.495, 0.165, 0.165, 0.495, 0.495, 0.165, 0.165, 0.495]
11merge_method: dare_ties
12base_model: turboderp/llama3-turbcat-instruct-8b
13parameters:
14 normalize: false
15 int8_mask: true
16dtype: float32
17out_dtype: bfloat16
Super-Nova-INT_pt.1
1models:
2 - model: TheSkullery/llama-3-cat-8b-instruct-v1
3 - model: FPHam/L3-8B-Everything-COT
4 parameters:
5 density: 0.5
6 weight: [0.139, 0.139, 0.208, 0.139, 0.208]
7 - model: Ayush-1722/Meta-Llama-3-8B-Instruct-Summarize-v0.2-24K-LoRANET-Merged
8 parameters:
9 density: 0.5
10 weight: [0.139, 0.208, 0.139, 0.208, 0.139]
11 - model: OEvortex/Emotional-llama-8B
12 parameters:
13 density: 0.5
14 weight: [0.208, 0.139, 0.208, 0.139, 0.139]
15 - model: lighteternal/Llama3-merge-biomed-8b
16 parameters:
17 density: 0.5
18 weight: [0.208, 0.139, 0.139, 0.139, 0.208]
19 - model: Casual-Autopsy/Llama3-merge-psychotherapy-8b
20 parameters:
21 density: 0.5
22 weight: [0.139, 0.208, 0.139, 0.208, 0.139]
23merge_method: ties
24base_model: TheSkullery/llama-3-cat-8b-instruct-v1
25parameters:
26 normalize: false
27 int8_mask: true
28dtype: float32
29out_dtype: bfloat16
Super-Nova-INT_pt.2
1models:
2 - model: TheSkullery/llama-3-cat-8b-instruct-v1
3 - model: FPHam/L3-8B-Everything-COT
4 parameters:
5 density: 0.9
6 gamma: 0.01
7 weight: [0.139, 0.208, 0.208, 0.139, 0.139]
8 - model: Ayush-1722/Meta-Llama-3-8B-Instruct-Summarize-v0.2-24K-LoRANET-Merged
9 parameters:
10 density: 0.9
11 gamma: 0.01
12 weight: [0.208, 0.139, 0.139, 0.139, 0.208]
13 - model: OEvortex/Emotional-llama-8B
14 parameters:
15 density: 0.9
16 gamma: 0.01
17 weight: [0.139, 0.139, 0.208, 0.208, 0.139]
18 - model: lighteternal/Llama3-merge-biomed-8b
19 parameters:
20 density: 0.9
21 gamma: 0.01
22 weight: [0.139, 0.208, 0.139, 0.208, 0.139]
23 - model: Casual-Autopsy/Llama3-merge-psychotherapy-8b
24 parameters:
25 density: 0.9
26 gamma: 0.01
27 weight: [0.208, 0.139, 0.139, 0.139, 0.208]
28merge_method: breadcrumbs_ties
29base_model: TheSkullery/llama-3-cat-8b-instruct-v1
30parameters:
31 normalize: false
32 int8_mask: true
33dtype: float32
34out_dtype: bfloat16
Super-Nova-CRE
1
2models:
3 - model: Casual-Autopsy/Super-Nova-CRE_pt.1
4 - model: Casual-Autopsy/Super-Nova-CRE_pt.2
5merge_method: slerp
6base_model: Casual-Autopsy/Super-Nova-CRE_pt.1
7parameters:
8 t:
9 - filter: self_attn
10 value: [0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5]
11 - filter: mlp
12 value: [0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5]
13 - value: 0.5
14 embed_slerp: true
15dtype: float32
16out_dtype: bfloat16
Super-Nova-UNC
1models:
2 - model: Casual-Autopsy/Super-Nova-UNC_pt.1
3 - model: Casual-Autopsy/Super-Nova-UNC_pt.2
4merge_method: slerp
5base_model: Casual-Autopsy/Super-Nova-UNC_pt.1
6parameters:
7 t:
8 - value: [0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5, 0.7, 0.3, 0.5, 0.3, 0.7, 0.5]
9 embed_slerp: true
10dtype: float32
11out_dtype: bfloat16
Super-Nova-INT
1models:
2 - model: Casual-Autopsy/Super-Nova-INT_pt.1
3 - model: Casual-Autopsy/Super-Nova-INT_pt.2
4merge_method: slerp
5base_model: Casual-Autopsy/Super-Nova-INT_pt.1
6parameters:
7 t:
8 - value: 0.5
9 embed_slerp: true
10dtype: float32
11out_dtype: bfloat16
Super-Nova-RP_stp.1
1
2models:
3 - model: Casual-Autopsy/Super-Nova-CRE
4 - model: asual-Autopsy/Super-Nova-UNC
5merge_method: slerp
6base_model: Casual-Autopsy/Super-Nova-CRE
7parameters:
8 t:
9 - value: [0.7, 0.5, 0.3, 0.25, 0.2, 0.25, 0.3, 0.5, 0.7]
10 embed_slerp: true
11dtype: float32
12out_dtype: bfloat16
13
Super-Nova-RP_stp.2
1models:
2 - model: Casual-Autopsy/Super-Nova-RP_stp.1
3 - model: Casual-Autopsy/Super-Nova-INT
4merge_method: slerp
5base_model: Casual-Autopsy/Super-Nova-RP_stp.1
6parameters:
7 t:
8 - value: [0.1, 0.15, 0.2, 0.4, 0.6, 0.4, 0.2, 0.15, 0.1]
9 embed_slerp: true
10dtype: float32
11out_dtype: bfloat16
Super-Nova-RP_pt.1
1models:
2 - model: Casual-Autopsy/Super-Nova-RP_stp.2
3 - model: Sao10K/L3-8B-Tamamo-v1
4 parameters:
5 density: [0.4, 0.6, 0.5, 0.6, 0.4]
6 epsilon: [0.15, 0.15, 0.25, 0.15, 0.15]
7 lambda: 0.85
8 weight: [-0.01523, 0.01768, -0.01384, 0.01835, -0.01247]
9 - model: ResplendentAI/Nymph_8B
10 parameters:
11 density: [0.65, 0.35, 0.5, 0.35, 0.65]
12 epsilon: [0.1, 0.1, 0.25, 0.1, 0.1]
13 lambda: 0.85
14 weight: [0.01823, -0.01647, 0.01422, -0.01975, 0.01128]
15 - model: ChaoticNeutrals/T-900-8B
16 parameters:
17 density: [0.35, 0.65, 0.5, 0.65, 0.35]
18 epsilon: [0.1, 0.1, 0.25, 0.1, 0.1]
19 lambda: 0.85
20 weight: [-0.01891, 0.01554, -0.01325, 0.01791, -0.01458]
21 - model: Sao10K/L3-8B-Niitama-v1
22 parameters:
23 density: [0.6, 0.4, 0.5, 0.4, 0.6]
24 epsilon: [0.15, 0.15, 0.25, 0.15, 0.15]
25 lambda: 0.85
26 weight: [0.01768, -0.01675, 0.01285, -0.01696, 0.01421]
27merge_method: della
28base_model: Casual-Autopsy/Super-Nova-RP_stp.2
29parameters:
30 normalize: false
31 int8_mask: true
32dtype: float32
33out_dtype: bfloat16
Super-Nova-RP_pt.2
1models:
2 - model: Casual-Autopsy/Super-Nova-RP_stp.2
3 - model: bluuwhale/L3-SthenoMaidBlackroot-8B-V1
4 parameters:
5 density: [0.4, 0.6, 0.5, 0.6, 0.4]
6 epsilon: [0.15, 0.15, 0.25, 0.15, 0.15]
7 lambda: 0.85
8 weight: [-0.01935, 0.01785, -0.01512, 0.01809, -0.01371]
9 - model: Hastagaras/Jamet-8B-L3-MK.V-Blackroot
10 parameters:
11 density: [0.65, 0.35, 0.5, 0.35, 0.65]
12 epsilon: [0.1, 0.1, 0.25, 0.1, 0.1]
13 lambda: 0.85
14 weight: [0.01847, -0.01468, 0.01503, -0.01822, 0.01459]
15 - model: Hastagaras/Halu-8B-Llama3-Blackroot
16 parameters:
17 density: [0.35, 0.65, 0.5, 0.65, 0.35]
18 epsilon: [0.1, 0.1, 0.25, 0.1, 0.1]
19 lambda: 0.85
20 weight: [-0.01578, 0.01821, -0.01753, 0.01677, -0.01442]
21 - model: crestf411/L3-8B-sunfall-v0.4-stheno-v3.2
22 parameters:
23 density: [0.6, 0.5, 0.5, 0.5, 0.6]
24 epsilon: [0.15, 0.15, 0.25, 0.15, 0.15]
25 lambda: 0.85
26 weight: [0.01667, -0.01740, 0.01560, -0.01564, 0.01315]
27merge_method: della
28base_model: Casual-Autopsy/Super-Nova-RP_stp.2
29parameters:
30 normalize: false
31 int8_mask: true
32dtype: float32
33out_dtype: bfloat16
L3-Super-Nova-RP-8B
1models:
2 - model: Casual-Autopsy/Super-Nova-RP_pt.1
3 - model: Casual-Autopsy/Super-Nova-RP_pt.2
4merge_method: slerp
5base_model: Casual-Autopsy/Super-Nova-RP_pt.1
6parameters:
7 t:
8 - value: 0.5
9dtype: float32
10out_dtype: bfloat16
