GTO is a framework designed to bridge the "draft policy misalignment" between training (which often focuses on single-token greedy paths) and inference (which uses tree-based re-ranking and verification). It introduces a Draft Tree Reward objective and a Group-based Draft Policy Training scheme to optimize acceptance lengths and inference speed.
For implementation details, training scripts, and inference code, please visit the official repository:
https://github.com/hsj576/GTOThe official implementation provides a web interface for inference. To use this draft model with a base model, you can run the following command from the GTO repository:
1python -m application.webui --ea-model-path [path of GTO weight]\
2 --base-model-path [path of the original model]\
3 --model-type [vicuna\llama3\qwen]\
4 --total-token [int]
1@article{hu2025bridging,
2 title={Bridging Draft Policy Misalignment: Group Tree Optimization for Speculative Decoding},
3 author={Hu, Shijing and Li, Jingyang and Lu, Zhihui and Zhou, Pan},
4 journal={arXiv preprint arXiv:2509.22134},
5 year={2025}
6}
The implementation is based on the open-source repository of
EAGLE and has been influenced by projects in the LLM community such as
HASS and
GRIFFIN.
