A 1M token version of the RULER dataset based on the Llama-3 chat template.
It is automatically generated based on the scripts available in the RULER repository:
https://github.com/NVIDIA/RULER. It is designed for evaluating the performance of Long Language Models (LLMs) on various tasks with varying sequence lengths.
LENGTH_IN_STRING = ['4k', '8k', '16k', '32k', '64k', '128k', '256k', '512k', '1M']
TASKS =… See the full description on the dataset page:
https://huggingface.co/datasets/self-long/RULER-llama3-1M.
