This is a dataset generated by real swebenchpro agentic workload trace + codex agent.
Metric
Value
Total trials
731
Successful trials
610
Failed trials
120
No data (skipped)
1
Passed
329
Pass rate (of successful)
53.9%
Repo
Total
Success
Failed
Passed
Pass%
ansible/ansible
96
93
3
60
65%
internetarchive/openli
91
88
3
52
59%
flipt-io/flipt85
82
3
26
32%
qutebrowser/qutebrowse
79
78
1… See the full description on the dataset page:
https://huggingface.co/datasets/Inferact/codex_swebenchpro_traces.
