This is a merge of pre-trained language models created using
mergekit.
A blend of the following models, using multiple types of merge methods (karcher, nuslerp, multislerp, modelstock, dare ties):
-
TheDrummer/Anubis-70B-v1.1
-
TheDrummer/Fallen-Llama-3.3-70B-v1
-
nvidia/Llama-3.1-Nemotron-70B-Instruct-HF
-
deepcogito/cogito-v1-preview-llama-70B
-
tdrussell/Llama-3-70B-Instruct-Storywriter
-
Sao10K/L3.3-70B-Euryale-v2.3
-
Sao10K/L3-70B-Euryale-v2.1
-
Sao10K/70B-L3.3-Cirrus-x1
-
Sao10K/70B-L3.3-mhnnn-x1
-
Sao10K/L3.1-70B-Hanami-x1
-
Doctor-Shotgun/L3.3-70B-Magnum-Diamond
-
Delta-Vector/Austral-70B-Winton
-
LatitudeGames/Wayfarer-Large-70B-Llama-3.3
-
ArliAI/Llama-3.3-70B-ArliAI-RPMax-v1.4
-
nbeerbower/Llama3.1-Gutenberg-Doppel-70B
-
ReadyArt/Forgotten-Safeword-70B-v5.0
-
Envoid/Llama-3-TenyxChat-DaybreakStorywriter-70B
This model was merged using the
DARE TIES merge method using BruhzWater/Eden-L3.3-70b-0.1 as a base.
1models:
2 - model: /workspace/cache/models--bruhzair--prototype-0.4x195/snapshots/a1cb4161a717ebf8052ce09dccaedf2dde2a7a9f
3 parameters:
4 weight: 0.15
5 density: 0.4
6 - model: /workspace/prototype-0.4x204
7 parameters:
8 weight: 0.20
9 density: 0.4
10 - model: /workspace/prototype-0.4x208
11 parameters:
12 weight: 0.15
13 density: 0.4
14 - model: /workspace/prototype-0.4x210
15 parameters:
16 weight: 0.15
17 density: 0.4
18 - model: /workspace/prototype-0.4x197
19 parameters:
20 weight: 0.5
21 density: 0.4
22merge_method: dare_ties
23base_model: /workspace/prototype-0.4x197
24parameters:
25 normalize: false
26dtype: bfloat16
27pad_to_multiple_of: 8
28int8_mask: true
29tokenizer:
30 source: base
