Views
No views yet
1base_model: Locutusque/TinyMistral-248M-v2-Instruct
2gate_mode: hidden
3dtype: bfloat16
4experts:
5 - source_model: M4-ai/TinyMistral-248M-v2-cleaner
6 positive_prompts:
7 - "versatile"
8 - "helpful"
9 - "factual"
10 - "integrated"
11 - "adaptive"
12 - "comprehensive"
13 - "balanced"
14 negative_prompts:
15 - "specialized"
16 - "narrow"
17 - "focused"
18 - "limited"
19 - "specific"
20
21 - source_model: Locutusque/TinyMistral-248M-Instruct
22 positive_prompts:
23 - "creative"
24 - "chat"
25 - "discuss"
26 - "culture"
27 - "world"
28 - "expressive"
29 - "detailed"
30 - "imaginative"
31 - "engaging"
32 negative_prompts:
33 - "sorry"
34 - "cannot"
35 - "factual"
36 - "concise"
37 - "straightforward"
38 - "objective"
39 - "dry"
40
41 - source_model: jtatman/tinymistral-v2-pycoder-instuct-248m
42 positive_prompts:
43 - "analytical"
44 - "accurate"
45 - "logical"
46 - "knowledgeable"
47 - "precise"
48 - "calculate"
49 - "compute"
50 - "solve"
51 - "work"
52 - "python"
53 - "javascript"
54 - "programming"
55 - "algorithm"
56 - "tell me"
57 - "assistant"
58 negative_prompts:
59 - "creative"
60 - "abstract"
61 - "imaginative"
62 - "artistic"
63 - "emotional"
64 - "mistake"
65 - "inaccurate"
66
67 - source_model: Locutusque/TinyMistral-248M-v2-Instruct
68 positive_prompts:
69 - "instructive"
70 - "clear"
71 - "directive"
72 - "helpful"
73 - "informative"
74 negative_prompts:
75 - "exploratory"
76 - "open-ended"
77 - "narrative"
78 - "speculative"
79 - "artistic"
801!pip install -qU transformers bitsandbytes accelerate
2
3from transformers import AutoTokenizer
4import transformers
5import torch
6
7model = "222gate/TinyMistral-248Mx4-MOE"
8
9tokenizer = AutoTokenizer.from_pretrained(model)
10pipeline = transformers.pipeline(
11 "text-generation",
12 model=model,
13 model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
14)
15
16messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
17prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
18outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
19print(outputs[0]["generated_text"])| Metric | Value |
|---|---|
| Avg. | 30.08 |
| AI2 Reasoning Challenge (25-Shot) | 29.52 |
| HellaSwag (10-Shot) | 25.71 |
| MMLU (5-Shot) | 24.82 |
| TruthfulQA (0-shot) | 48.66 |
| Winogrande (5-shot) | 51.78 |
| GSM8k (5-shot) | 0.00 |