Views
No views yet
| Metric | Value |
|---|---|
| MMLU (5-shot) | 70.48 |
| ARC (25-shot) | 71.84 |
| HellaSwag (10-shot) | 87.94 |
| TruthfulQA (0-shot) | 62.26 |
| Avg. | 73.13 |
### Instruction:
<prompt> (without the <>)
### Response:garage-bAInd/Platypus2-70B trained using STEM and logic based dataset garage-bAInd/Open-Platypus.garage-bAInd/Platypus2-70B was instruction fine-tuned using LoRA on 8 A100 80GB. For training details and inference instructions please see the Platypus GitHub repo.# clone repository
git clone https://github.com/EleutherAI/lm-evaluation-harness.git
# check out the correct commit
git checkout b281b0921b636bc36ad05c0b0b0763bd6dd43463
# change to repo directory
cd lm-evaluation-harness
# install
pip install -e .python main.py --model hf-causal-experimental --model_args pretrained=garage-bAInd/Platypus2-70B --tasks arc_challenge --batch_size 1 --no_cache --write_out --output_path results/Platypus2-70B/arc_challenge_25shot.json --device cuda --num_fewshot 25python main.py --model hf-causal-experimental --model_args pretrained=garage-bAInd/Platypus2-70B --tasks hellaswag --batch_size 1 --no_cache --write_out --output_path results/Platypus2-70B/hellaswag_10shot.json --device cuda --num_fewshot 10python main.py --model hf-causal-experimental --model_args pretrained=garage-bAInd/Platypus2-70B --tasks hendrycksTest-* --batch_size 1 --no_cache --write_out --output_path results/Platypus2-70B/mmlu_5shot.json --device cuda --num_fewshot 5python main.py --model hf-causal-experimental --model_args pretrained=garage-bAInd/Platypus2-70B --tasks truthfulqa_mc --batch_size 1 --no_cache --write_out --output_path results/Platypus2-70B/truthfulqa_0shot.json --device cuda1@article{platypus2023,
2 title={Platypus: Quick, Cheap, and Powerful Refinement of LLMs},
3 author={Ariel N. Lee and Cole J. Hunter and Nataniel Ruiz},
4 booktitle={arXiv preprint arxiv:2308.07317},
5 year={2023}
6}1@misc{touvron2023llama,
2 title={Llama 2: Open Foundation and Fine-Tuned Chat Models},
3 author={Hugo Touvron and Louis Martin and Kevin Stone and Peter Albert and Amjad Almahairi and Yasmine Babaei and Nikolay Bashlykov year={2023},
4 eprint={2307.09288},
5 archivePrefix={arXiv},
6}1@inproceedings{
2 hu2022lora,
3 title={Lo{RA}: Low-Rank Adaptation of Large Language Models},
4 author={Edward J Hu and Yelong Shen and Phillip Wallis and Zeyuan Allen-Zhu and Yuanzhi Li and Shean Wang and Lu Wang and Weizhu Chen},
5 booktitle={International Conference on Learning Representations},
6 year={2022},
7 url={https://openreview.net/forum?id=nZeVKeeFYf9}
8}