This is a merge of pre-trained language models created using
mergekit.
An experimental merge of LoRa adapter to further boost Shisa-K-12B roleplaying capabilities with essence of PocketDoc/Dans-SakuraKaze-V1.0.0-12b.
Can occasionaly output japanese symbols, potentially mitigated by lowering TOP_P to 90 and increasing MIN_P to 0.1.
Uses ChatML.
Oh, and I am planning to use this model as layer range for next KansenSakura update
This model was merged using the
Linear merge method using ./retokenized_SHK as a base.
1merge_method: linear
2base_model: ./retokenized_SHK
3models:
4 - model: ./retokenized_SHK
5 parameters:
6 weight: 0.0
7 - model: ./retokenized_SHK+./lora_Dans-SakuraKaze-V1.0.0-12b-64d
8 parameters:
9 weight: 1.0
10dtype: bfloat16
11out_dtype: bfloat16
12tokenizer_source: Retreatcost/KansenSakura-Radiance-RP-12b