In real-world service scenarios, each request comes with varying input token lengths.
Some requests generate only a few tokens, while others produce a significant number.
Traditional fixed-length benchmarks fail to capture this variability, making it difficult to accurately assess real-world throughput performance.
This dynamic nature of input token lengths is crucial as it directly affects key features of… See the full description on the dataset page:
https://huggingface.co/datasets/squeezebits/dynamic_sonnet_llama2.
