Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models
Metis-8B-ColdStart is the SFT (Supervised Fine-Tuning) checkpoint of the Metis framework, fine-tuned from Qwen3-VL-8B-Instruct on the curated Metis-ColdStart dataset. This checkpoint serves as the starting point for HDPO reinforcement learning, which produces the final Metis-8B-RL model.
The SFT corpus is curated from publicly available tool-augmented multimodal trajectories (DeepEyesV2, V-Interaction, Thyme, OpenMMReasoner) through a rigorous three-stage pipeline:
Eradicating hallucinated environmental dynamics — Execute all code in a sandbox environment; discard trajectories with execution failures.
Isolating genuine tool necessity — Filter out samples where the base model achieves pass@8 = 1 without any tools, ensuring only genuinely tool-dependent samples remain.
Multidimensional meta-cognitive filtering — An LLM judge evaluates visual relevance, reasoning coherence, and tool-use rationale to ensure high quality.
1@article{yan2026metis,
2 title={Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models},
3 author={Yan, Shilin and Tong, Jintao and Xue, Hongwei and Tang, Xiaojun and Wang, Yangyang and Shi, Kunyu and Zhang, Guannan and Li, Ruixuan and Zou, Yixiong},
4 journal={arXiv preprint arXiv:2604.08545},
5 year={2026}
6}
