Is this your dataset? Claim it with the Hugging Face account that owns it.

IFM/
K2Datasets

Views

No views yet

K2 Dataset Card

The following data mix was used to train K2 and achieve results in line with Llama 2 70B.

Dataset Details

K2 was trained on 1.4T tokens across two stages. The data sources and data mix for each stage are listed below.

	Dataset Description: Stage 1

Dataset Starting Tokens Multiplier Total Tokens % of Total

dm-math 4.33B 3x 13B 1%

pubmed-abstracts (from the Pile) 4.77B 3x 14.3B 1.1%

uspto (from the Pile) 4.77B 3x… See the full description on the dataset page: https://huggingface.co/datasets/IFM/K2Datasets.