Views
No views yet
| Name | Quant method | Size |
|---|---|---|
| Beyonder-4x7b.Q2_K.gguf | Q2_K | 8.24GB |
| Beyonder-4x7b.IQ3_XS.gguf | IQ3_XS | 9.21GB |
| Beyonder-4x7b.IQ3_S.gguf | IQ3_S | 9.73GB |
| Beyonder-4x7b.Q3_K_S.gguf | Q3_K_S | 9.72GB |
| Beyonder-4x7b.IQ3_M.gguf | IQ3_M | 9.92GB |
| Beyonder-4x7b.Q3_K.gguf | Q3_K | 10.79GB |
| Beyonder-4x7b.Q3_K_M.gguf | Q3_K_M | 5.62GB |
| Beyonder-4x7b.Q3_K_L.gguf | Q3_K_L | 3.79GB |
| Beyonder-4x7b.IQ4_XS.gguf | IQ4_XS | 12.15GB |
| Beyonder-4x7b.Q4_0.gguf | Q4_0 | 12.69GB |
| Beyonder-4x7b.IQ4_NL.gguf | IQ4_NL | 12.81GB |
| Beyonder-4x7b.Q4_K_S.gguf | Q4_K_S | 12.8GB |
| Beyonder-4x7b.Q4_K.gguf | Q4_K | 13.61GB |
| Beyonder-4x7b.Q4_K_M.gguf | Q4_K_M | 13.61GB |
| Beyonder-4x7b.Q4_1.gguf | Q4_1 | 14.09GB |
| Beyonder-4x7b.Q5_0.gguf | Q5_0 | 15.48GB |
| Beyonder-4x7b.Q5_K_S.gguf | Q5_K_S | 15.48GB |
| Beyonder-4x7b.Q5_K.gguf | Q5_K | 15.96GB |
| Beyonder-4x7b.Q5_K_M.gguf | Q5_K_M | 10.3GB |
| Beyonder-4x7b.Q5_1.gguf | Q5_1 | 16.88GB |
| Beyonder-4x7b.Q6_K.gguf | Q6_K | 9.15GB |
| Beyonder-4x7b.Q8_0.gguf | Q8_0 | 23.9GB |
1base_model: openchat/openchat-3.5-1210
2gate_mode: hidden
3experts:
4 - source_model: openchat/openchat-3.5-1210
5 positive_prompts:
6 - "chat"
7 - "assistant"
8 - "tell me"
9 - "explain"
10 negative_prompts:
11 - "storywriting"
12 - "mathematics"
13 - "reasoning"
14 - "code"
15 - "programming"
16 - source_model: beowolx/CodeNinja-1.0-OpenChat-7B
17 positive_prompts:
18 - "code"
19 - "python"
20 - "javascript"
21 - "programming"
22 - "algorithm"
23 negative_prompts:
24 - "chat"
25 - "assistant"
26 - "storywriting"
27 - "mathematics"
28 - "reasoning"
29 - source_model: maywell/PiVoT-0.1-Starling-LM-RP
30 positive_prompts:
31 - "storywriting"
32 - "write"
33 - "scene"
34 - "story"
35 - "character"
36 negative_prompts:
37 - "chat"
38 - "assistant"
39 - "code"
40 - "programming"
41 - "mathematics"
42 - "reasoning"
43 - source_model: WizardLM/WizardMath-7B-V1.1
44 positive_prompts:
45 - "reason"
46 - "math"
47 - "mathematics"
48 - "solve"
49 - "count"
50 negative_prompts:
51 - "chat"
52 - "assistant"
53 - "code"
54 - "programming"
55 - "storywriting"1!pip install -qU transformers bitsandbytes accelerate
2
3from transformers import AutoTokenizer
4import transformers
5import torch
6
7model = "mlabonne/Beyonder-4x7b"
8
9tokenizer = AutoTokenizer.from_pretrained(model)
10pipeline = transformers.pipeline(
11 "text-generation",
12 model=model,
13 model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
14)
15
16messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
17prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
18outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
19print(outputs[0]["generated_text"])A Mixture of Experts (MoE) is a neural network architecture that combines the strengths of multiple expert networks to make predictions. It leverages the idea of ensemble learning, where multiple models work together to improve performance. In each MoE, a gating network is used to select the most relevant expert for the input. The final output is a weighted combination of the expert outputs, determined by the gating network's predictions.