The
Thinking Reward Model (TRM) evaluates the quality of
reasoning traces rather than just final answers. Introduced in the paper
Characterizing, Evaluating, and Optimizing Complex Reasoning, the model characterizes reasoning quality along four dimensions (the ME² principle):
The model can be used to score reasoning traces. Below is an example of how to use the model via a hosted server (e.g., using SGLang as suggested in the official repository):
1import requests
2import json
3
4# Example prompt and response
5prompt = "Your question here"
6response = "<think> Thinking process... </think> Final Answer"
7
8# Score the reasoning trace (before the termination marker).
9reasoning = response.split("</think>", 1)[0]
10input_text = f"{prompt}
11{reasoning}"
12
13payload = {"model": "RewardModel", "input": input_text}
14# Replace <TRM_HOST> and <TRM_PORT> with your server details
15resp = requests.post("http://<TRM_HOST>:<TRM_PORT>/v1/embeddings", json=payload, timeout=60)
16resp.raise_for_status()
17score = resp.json()["data"][0]["embedding"][0]
18print("TRM score:", score)
1@article{zhang2026characterizing,
2 title={Characterizing, Evaluating, and Optimizing Complex Reasoning},
3 author={Zhang, Haoran and Li, Yafu and Wang, Zhi and Wang, Zhilin and Zhang, Shunkai and Qu, Xiaoye and Cheng, Yu},
4 journal={arXiv preprint arXiv:2602.08498},
5 year={2026}
6}
