The SemCacheLMArena benchmark is designed to evaluate semantic caching in chatbot environments, where users may issue semantically similar prompts with different phrasing. In such settings, caches must generalize across diverse surface forms while maintaining response correctness. This benchmark captures these challenges by grouping semantically similar user inputs and testing whether caching systems can reuse responses accurately.
To construct the benchmark, we use the LM-Arena human… See the full description on the dataset page:
https://huggingface.co/datasets/vCache/SemBenchmarkLmArena.
