This is a merge of pre-trained language models created using
mergekit by @softwareweaver. Use the prompt format that Mistral Large uses.
Control vectors allow fine-tuned control over LLMs, enabling more precise/targeted text generation.
More info
https://huggingface.co/jukofyork/creative-writing-control-vectors-v3.0
Please add your own generations to the community tab. This allows others to evaluate the model outputs before downloading it.
This model was merged using the della_linear merge method using
mistralai/Mistral-Large-Instruct-2407 as a base.
1models:
2 - model: TheDrummer/Behemoth-123B-v1
3 parameters:
4 weight: 0.25
5 density: 0.9
6 - model: schnapper79/lumikabra-123B_v0.4
7 parameters:
8 weight: 0.3
9 density: 0.9
10merge_method: della_linear
11base_model: mistralai/Mistral-Large-Instruct-2407
12parameters:
13 epsilon: 0.05
14 lambda: 1
15 int8_mask: true
16dtype: bfloat16