400 tasks. Each hands the agent a correct-but-slow reference implementation and an empty stub; the agent writes a fast GPU kernel.
reward = 0 if the submission is incorrect
reward = achieved TFLOP/s or GB/s otherwise, UNCAPPED
Correctness is the gate; speed is the reward. There is no oracle and no gold solution — the score is an absolute hardware metric, so it is hardware-portable by construction and nothing… See the full description on the dataset page:
https://huggingface.co/datasets/ZMC2019/KBench.
