Views
No views yet
| Number of experts per token | Link of the branch |
|---|---|
| 2 | Main |
| 3 | 3-experts-per-token |
| 4 | 4-experts-per-token |
| 6 | 6-experts-per-token |
| 8 | 8-experts-per-token |
GPT4 Correct User: {user}<|end_of_turn|>GPT4 Correct Assistant: {asistant}<|end_of_turn|><|im_start|>system
{system}<|im_end|>
<|im_start|>user
{user}<|im_end|>
<|im_start|>assistant
{asistant}<|im_end|>Below is an instruction that describes a task. Write a response that appropriately completes the request.
### Instruction:
{instruction}
### Response: Let's think step by step.1base_model: openchat/openchat-3.5-0106
2gate_mode: hidden
3dtype: bfloat16
4
5experts:
6 - source_model: openchat/openchat-3.5-0106
7 positive_prompts: # General (Mistral finetune)
8 - "chat"
9 - "assistant"
10 - "tell me"
11 - "explain"
12
13 - source_model: teknium/OpenHermes-2.5-Mistral-7B
14 positive_prompts: # General (Mistral finetune)
15 - "interact"
16 - "converse"
17 - "respond"
18 - "express"
19
20 - source_model: jondurbin/bagel-dpo-7b-v0.1
21 positive_prompts: # Science (Mistral finetune)
22 - "science"
23 - "biology"
24 - "chemistry"
25 - "physics"
26 - "Newton's laws"
27 - "scientific method"
28 - "periodic table"
29 - "photosynthesis process"
30
31 - source_model: meta-math/MetaMath-Mistral-7B
32 positive_prompts: # Math (Mistral finetune)
33 - "reason"
34 - "math"
35 - "mathematics"
36 - "solve"
37 - "count"
38
39 - source_model: cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser
40 positive_prompts: # Uncensored (Mistral finetune)
41 - "dolphin"
42 - "uncensored"
43 - "unbiased"
44 - "unfiltered"
45 - "unrestricted"
46 - "offensive"
47
48 - source_model: beowolx/CodeNinja-1.0-OpenChat-7B
49 positive_prompts: # Code (openchat-3.5-1210 finetune)
50 - "code"
51 - "script"
52 - "python"
53 - "javascript"
54 - "programming"
55 - "algorithm"
56
57 - source_model: senseable/WestLake-7B-v2
58 positive_prompts: # Roleplay (Unknown finetune)
59 - "storywriting"
60 - "write"
61 - "scene"
62 - "story"
63 - "character"
64 - "act as"
65 - "you are"
66
67 - source_model: snorkelai/Snorkel-Mistral-PairRM-DPO
68 positive_prompts: # Question Answering (? Mistral-7B-Instruct-v0.2 finetune ?)
69 - "what happens"
70 - "what is"
71 - "what can"
72 - "why"
73 - "who"
74 - "can a"| Metric | Value |
|---|---|
| Avg. | 70.89 |
| AI2 Reasoning Challenge (25-Shot) | 65.02 |
| HellaSwag (10-Shot) | 85.24 |
| MMLU (5-Shot) | 64.96 |
| TruthfulQA (0-shot) | 62.65 |
| Winogrande (5-shot) | 80.66 |
| GSM8k (5-shot) | 66.79 |