Unfortunately, the original ClimbMix (
https://huggingface.co/datasets/nvidia/ClimbMix) has four main inconveniences:
It is in GPT2 tokens, meaning you have to detokenize it to inspect it or use it with another tokenizer.
It contains all of the 20 clusters in order together (in the same "subset"), so you have to load the whole dataset in memory (~1TB) and shuffle it… See the full description on the dataset page:
https://huggingface.co/datasets/gvlassis/ClimbMix.
