This dataset was used to finetune Llama-2-7B-32K-Instruct.
We follow the distillation paradigm that is used by Alpaca, Vicuna, WizardLM, Orca
— producing instructions by querying a powerful LLM, which in our case, is the Llama-2-70B-Chat model released by Meta.
To build Llama-2-7B-32K-Instruct, we collect instructions from 19K human inputs extracted from ShareGPT-90K (only using human inputs, not ChatGPT outputs).
The actual script handles multi-turn conversations… See the full description on the dataset page:
https://huggingface.co/datasets/togethercomputer/llama-instruct.
