Perfectly fits 16k context on 4bit with 8GB of vram. TabbyAPI recommended, ooba doesnt like this model for some reason.
Lightly tested with both Alpaca and ChatML prompts. Works with temperature 1.0 and minP 0.01, but feel free to vary it up. Tested to 8K context.
This model has a tendency to lean into revealing character interiority when generating narrative, which some people might find interesting. I found the model good with not only following the character card but also taking strong hints from the first message. This experimental model may occasionally reveal context, unfortunately.
This is a merge of pre-trained language models created using
mergekit.
Available quants:
GGUF (exl2 quant of BF16 model was broken, and withdrawn)
This model was merged using the
DARE TIES merge method using
SanjiWatsuki/Kunoichi-7B as a base. Each of the models had strengths I liked to varying degrees, leading to weights and densities being adjusted in aesthetic proportion.
1models:
2 - model: SanjiWatsuki/Kunoichi-7B
3 # no parameters necessary for base model
4 - model: KatyTheCutie/LemonadeRP-4.5.3
5 parameters:
6 weight: 0.3
7 density: 0.4
8 - model: core-3/kuno-royale-v2-7b
9 parameters:
10 weight: 0.3
11 density: 0.4
12 - model: SanjiWatsuki/Kunoichi-DPO-v2-7B
13 parameters:
14 weight: 0.4
15 density: 0.8
16merge_method: dare_ties
17base_model: SanjiWatsuki/Kunoichi-7B
18parameters:
19 int8_mask: true
20 normalize: true
21dtype: bfloat16
22
