• Model creator: https://huggingface.co/fblgit/
• Original model: https://huggingface.co/fblgit/una-cybertron-7b-v1-fp16/

Original model description:

• juanako
• UNA datasets:
• fblgit/tree-of-knowledge
• Open-Orca/SlimOrca-Dedup
• HuggingFaceH4/ultrafeedback_binarized model-index:
• name: una-cybertron-7b-v1-fp16 results:
  • task: type: text-generation name: Text Generation dataset: name: AI2 Reasoning Challenge (25-Shot) type: ai2_arc config: ARC-Challenge split: test args: num_few_shot: 25 metrics:
    • type: acc_norm value: 68.43 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: HellaSwag (10-Shot) type: hellaswag split: validation args: num_few_shot: 10 metrics:
    • type: acc_norm value: 85.42 name: normalized accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: MMLU (5-Shot) type: cais/mmlu config: all split: test args: num_few_shot: 5 metrics:
    • type: acc value: 63.34 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: TruthfulQA (0-shot) type: truthful_qa config: multiple_choice split: validation args: num_few_shot: 0 metrics:
    • type: mc2 value: 63.28 source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: Winogrande (5-shot) type: winogrande config: winogrande_xl split: validation args: num_few_shot: 5 metrics:
    • type: acc value: 81.37 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16 name: Open LLM Leaderboard
  • task: type: text-generation name: Text Generation dataset: name: GSM8k (5-shot) type: gsm8k config: main split: test args: num_few_shot: 5 metrics:
    • type: acc value: 55.12 name: accuracy source: url: https://huggingface.co/spaces/HuggingFaceH4/open_llm_leaderboard?query=fblgit/una-cybertron-7b-v1-fp16 name: Open LLM Leaderboard

Model Card for una-cybertron-7b-v1 (UNA: Uniform Neural Alignment)

Model Details

Model Description

• Developed by: juanako.ai
• Author: Xavier M.
• Model type: MistralAI 7B
• Funded by Cybertron's H100's

Prompt

<|im_start|>system
- You are a helpful assistant chatbot trained by MosaicML.
- You answer questions.
- You are excited to be able to help the user, but will refuse to do anything that could be considered harmful to the user.
- You are more than just an information source, you are also able to write poetry, short stories, and make jokes.<|im_end|>
<|im_start|>user
Explain QKV<|im_end|>
<|im_start|>assistant

### Assistant: I am StableVicuna, a large language model created by CarperAI. I am here to chat!

### Human: Explain QKV
### Assistant:

[Round <|round|>]
问：Explain QKV
答：

[Round <|round|>]
Question：Explain QKV
Answer：

Question：Explain QKV
Answer：

Evaluation

|    Tasks     |Version|Shots | Metric |Value |   |Stderr|
|--------------|-------|------|--------|-----:|---|-----:|
|arc_challenge |       | 25   |acc_norm|0.6817|±  |0.0136|
|truthfulqa_mc2|       | 0    |acc     |0.6398|±  |0.0151|
|hellaswag     |       | 10   |acc_norm|0.8492|±  |0.0036|
|winogrande    |       | 0    |acc     |0.809 |±  |0.011 |
|gsm8k         |       | 5    |acc     |0.2733|±  |0.0137|
|mmlu          |       | 5    |acc     |0.6207|±  |0.1230|
|              |average|      |acc     |0.6456|   |      |

|      Groups      |Version|Filter|n-shot|Metric|Value |   |Stderr|
|------------------|-------|------|-----:|------|-----:|---|-----:|
|mmlu              |N/A    |none  |     0|acc   |0.6207|_  |0.1230|
| - humanities     |N/A    |none  |     5|acc   |0.5675|_  |0.1125|
| - other          |N/A    |none  |     5|acc   |0.6933|_  |0.1108|
| - social_sciences|N/A    |none  |     5|acc   |0.7270|_  |0.0666|
| - stem           |N/A    |none  |     5|acc   |0.5249|_  |0.1311|

Framework versions

• Transformers 4.35.0-UNA
• Pytorch 2.1.0
• Datasets 2.14.6
• Tokenizers 0.14.1

Citations

@misc{unacybertron7a,
  title={Cybertron: Uniform Neural Alignment}, 
  author={Xavier Murias},
  year={2023},
  publisher = {HuggingFace},
  journal = {HuggingFace repository},
  howpublished = {\url{https://huggingface.co/fblgit/una-cybertron-7b-v1}},
}

Open LLM Leaderboard Evaluation Results