Views
No views yet


| Model | Easy | Medium | Hard | Impossible | Overall | Avg. Calls |
|---|---|---|---|---|---|---|
| Qwen-3-4B (baseline) | 18.31 | 9.46 | 2.82 | 10.00 | 10.57 | 7.63 |
| STT-Agent (w/o refine) | 28.17 | 16.92 | 11.86 | 47.01 | 23.10 | 32.70 |
| {model_name} (with refine) | 26.76 | 17.41 | 13.56 | 61.11 | 25.11 | 15.30 |
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_name = "{model_name}"
4tokenizer = AutoTokenizer.from_pretrained(model_name)
5model = AutoModelForCausalLM.from_pretrained(model_name)
6
7# Example tool-use prompt
8prompt = "User: Book the cheapest flight from PVG to CDG.\n"
9inputs = tokenizer(prompt, return_tensors="pt")
10outputs = model.generate(**inputs)
11print(tokenizer.decode(outputs[0]))1@misc{hui2026sttarenarealisticenvironmenttoolusing,
2 title={STT-Arena: A More Realistic Environment for Tool-Using with Spatio-Temporal Dynamics},
3 author={Tingfeng Hui and Hao Xu and Pengyu Zhu and Hongsheng Xin and Kun Zhan and Sen Su and Chunxiao Liu and Ning Miao},
4 year={2026},
5 eprint={2605.18548},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL},
8 url={https://arxiv.org/abs/2605.18548},
9}