UPS Tool-Use Qwen2.5 7B is a QLoRA fine-tune of Qwen2.5-7B-Instruct for generating structured Hermes function-calling payloads for a UPS MCP server. It maps natural-language shipping and logistics requests to 18 UPS MCP tools covering tracking, rating, shipment creation, pickup workflows, address validation, paperless documents, landed cost, and location lookup.
This is the v5a release, trained on top of the v3 champion with replay + targeted augmentation data. It achieves 100% held-out accuracy and improves all stress-test axes over the prior release.
This repository publishes the PEFT LoRA adapter at the repo root and a Q4_K_M GGUF export for local inference with llama.cpp-compatible runtimes such as Ollama.
This model is not affiliated with or endorsed by UPS.
Model Details
Developed by: Matthew Hans
Model type: Qwen2.5-7B-Instruct causal language model with LoRA adapter fine-tuning
Base model:Qwen/Qwen2.5-7B-Instruct
Training base:unsloth/Qwen2.5-7B-Instruct-bnb-4bit
Fine-tuning method: QLoRA with rank-stabilized LoRA (rsLoRA)
Language: English
Output format: Hermes FC v1 tool call blocks
Release: v5a (May 2026)
Intended Use
The model is intended to sit behind a UPS MCP server and produce one structured tool call for each user request. A downstream orchestrator should parse the <tool_call_> block, validate arguments against the MCP tool schema, and then decide whether to call the UPS API.
Paraphrase fragility. Value-grounded accuracy drops to 68.6% on paraphrased prompts (terse SMS-style is worst at ~57%). Free-form text interfaces will see higher error rates.
Sibling-tool confusion. On adversarial prompts with similar tools, shape accuracy is 88.3%. The most confused pairs are get_political_divisions vs find_locations and rate_shipment vs create_shipment.
Incomplete prompts. The model fabricates missing fields instead of asking for clarification. Production orchestrators must validate arguments against the MCP schema before execution.
rate_shipment cross-field combinations. The model can pair UPS service codes with incompatible packaging types. Validate service↔packaging compatibility before submitting.
Multi-step workflow cascading. Step accuracy is 96.3% but full-workflow accuracy is 80% — early errors cascade. Validate each turn in multi-turn flows.
Specialization to 18 UPS MCP tools. Do not expect generalization to unrelated APIs without additional evaluation.
Out-of-Scope Use
Do not use this model as an autonomous authority for shipping purchases, billing decisions, customs declarations, legal compliance, or cancellation actions. It should generate candidate MCP tool calls only; production systems should validate, log, and gate execution.
License
Apache 2.0. The base Qwen2.5 model is also released under Apache 2.0.
