Views
No views yet
1models:
2 - model: mistralai/Mistral-7B-v0.1
3 - model: samir-fama/SamirGPT-v1
4 parameters:
5 density: 0.53
6 weight: 0.4
7 - model: abacusai/Slerp-CM-mist-dpo
8 parameters:
9 density: 0.53
10 weight: 0.3
11 - model: EmbeddedLLM/Mistral-7B-Merge-14-v0.2
12 parameters:
13 density: 0.53
14 weight: 0.3
15merge_method: dare_ties
16base_model: mistralai/Mistral-7B-v0.1
17parameters:
18 int8_mask: true
19dtype: bfloat16
201!pip install -qU transformers bitsandbytes accelerate
2
3from transformers import AutoTokenizer
4import transformers
5import torch
6
7model = "mayacinka/West-Ramen-7Bx4"
8
9tokenizer = AutoTokenizer.from_pretrained(model)
10pipeline = transformers.pipeline(
11 "text-generation",
12 model=model,
13 model_kwargs={"torch_dtype": torch.float16, "load_in_4bit": True},
14)
15
16messages = [{"role": "user", "content": "Explain what a Mixture of Experts is in less than 100 words."}]
17prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
18outputs = pipeline(prompt, max_new_tokens=256, do_sample=True, temperature=0.7, top_k=50, top_p=0.95)
19print(outputs[0]["generated_text"])| Metric | Value |
|---|---|
| Avg. | 73.46 |
| AI2 Reasoning Challenge (25-Shot) | 69.71 |
| HellaSwag (10-Shot) | 87.05 |
| MMLU (5-Shot) | 65.07 |
| TruthfulQA (0-shot) | 63.24 |
| Winogrande (5-shot) | 81.61 |
| GSM8k (5-shot) | 73.01 |