This dataset splits lmsys/lmsys-chat-1m into trainval and benchmark splits. The benchmark split contains 790 samples (some repeated) of various prompt lengths. The purpose of that split is to benchmark LLM inference performance on hardware, by measuring statistics such as TTFT, throughput, etc.
Each prompt in the dataset was tokenized using TinyLlama/TinyLlama-1.1B-Chat-v1.0 tokenizer and binned into 100 equal sized bins. Within each size bin, 10 samples are randomly selected, with replacement… See the full description on the dataset page:
https://huggingface.co/datasets/NightShade9x9/lmsys-chat-1m-benchmark.
