qwen-tool
This model is a fine-tuned version of Qwen/Qwen3-4B, optimized for complex functional calling and multi-step tool use via the Model Context Protocol (MCP).
The model was aligned using Proximal Policy Optimization (PPO) in a closed-loop agentic environment. It leverages execution-based feedback from an MCP server to drastically reduce tool hallucinations, adhere to strict JSON formatting, and self-correct based on execution error states.
Model Details
- Developed by: Igriscodes
- Base Model:
Qwen/Qwen3-4B
- License: Mozilla Public License 2.0 (MPL 2.0)
- Training Framework: Hugging Face
trl & peft (LoRA)
- Alignment Method: PPO (Proximal Policy Optimization) with Execution-Based Reward Guidance
Intended Uses & Limitations
Intended Use Cases
- Structured Tool Calling: Interfacing natively with Model Context Protocol (MCP) servers.
- Multi-step Agentic Tasks: Iterative problem-solving across math, web searching, database queries, and data processing.
- Error-Resilient Agents: Handling tool-execution errors gracefully by rewriting payload schemas based on environment exceptions.
Training Architecture & Alignment Loop
The model was trained as the Policy (Actor) within a custom gymnasium environment (MCPGymEnv). The environment tracks an execution loop between the model's textual outputs and a backend mock MCP server.
Reward Specification Matrix
The PPO agent was optimized against a dense, feedback-driven execution reward model:
| Trigger Status | Reward | Evaluation Logic |
|---|
| Success | +10.0 | Tool executed cleanly; returned data matches the expected task state. |
| Tool Execution | 0.0 | Tool ran successfully, but the overarching objective is incomplete. |
| Tool Error | -0.5 | Target tool was hit, but threw a runtime exception (e.g., bad arguments). |
| Invalid JSON | -0.8 | Failed to output a syntactically valid JSON tool-call schema. |
| Structural Fail | -1.0 | Severe divergence from agentic system instructions or tool hallucination. |
Hyperparameters & Efficiency Stack
- Quantization: 4-bit NormalFloat (NF4) via
bitsandbytes (for base model loading).
- PEFT Adaptation: LoRA targeted all linear layers (
q_proj, v_proj, k_proj, o_proj, etc.).
- Memory Optimization: 8-bit Paged AdamW optimizer, gradient checkpointing, and parallel rollout sampling to balance the Actor-Critic-Reference model triplet footprint.
Acknowledgements
We express our gratitude to the following organizations, communities, and tools that made this project possible:
- Qwen (Alibaba Cloud) - For providing the foundational Qwen3 model weights and architecture.
- Hugging Face - For the incredible ecosystem and libraries used to load, manage, and train the model.
- PyTorch - For the robust, deep learning framework that powered the underlying tensor computations and GPU acceleration during fine-tuning.
- Google Gemini 3 - For providing assistance in optimizing, and debugging the fine-tuning code scripts.
License
