These files were generated from the latest evaluated merged checkpoint. The previous warning that this repository had not been regenerated is no longer applicable.
The export pipeline checked each output's GGUF magic, format version, size, and successful completion by llama-quantize. Full task evaluation was performed on the source merged model, not independently on every quantization.
Provide the available tool definitions in the training-compatible prompt format.
Decode deterministically for tool selection.
Stop after the first complete </function>.
Validate the function name and arguments against the supplied schema.
Execute the tool outside the model and provide the result in a new turn.
llama.cpp
Use a recent llama.cpp build:
bash
1llama-cli \2 -m MiniCPM5-1B-Agentic-Tooluse-Nemotron-DPO.Q4_K_M.gguf \3 -p '<user>Fix the calculator bug.</user>
4<tools>{"name":"run_tests","description":"Run the test suite."}</tools>
5<calls>'\6 -n 96\7 --temp 0
The exact chat template and EOS handling matter. If a frontend supplies its own template, verify that it matches the source model's chat_template.jinja.
vLLM
As of July 2026, vLLM documents GGUF support as highly experimental and under-optimized. GGUF support also requires the out-of-tree plugin:
The separate tokenizer/config arguments follow vLLM's recommendation to avoid unstable GGUF tokenizer conversion. The merged repository is used because it contains the matching tokenizer, configuration, and chat template.
For stable high-throughput vLLM deployment, prefer the merged safetensors repository when memory permits.
Source-Model Evaluation
External evaluation used 300 examples derived from the untouched
Team-ACE/ToolACE dataset. The same cases and deterministic first-call evaluator were used for the base and fine-tuned source model. This is not an official ToolACE leaderboard submission.
Metric
Base MiniCPM5-1B
Nemotron repaired source
Delta
Parseable tool call
0.0133
0.9933
+0.9800
Valid available-tool name
0.0133
0.9700
+0.9567
Expected tool name
0.0133
0.9267
+0.9133
Exact arguments
0.1500
0.6533
+0.5033
Argument-key overlap
0.0033
0.7517
+0.7484
No schema copying
1.0000
1.0000
+0.0000
No repetition
0.9967
1.0000
+0.0033
Clean stop
0.0000
0.1500
+0.1500
These scores establish the behavior of the merged source checkpoint. Quantization can alter individual outputs, so evaluate the exact GGUF and runtime configuration used in production.
Source-Model Improvements and GGUF Scope
These GGUF files were converted from the repaired merged checkpoint, whose reported results improved substantially over base MiniCPM5-1B:
Parseable calls: 1.33% -> 99.33%
Valid available-tool names: 1.33% -> 97.00%
Expected-tool selection: 1.33% -> 92.67%
Exact arguments: 15.00% -> 65.33%
Argument-key overlap: 0.33% -> 75.17%
No repetition: 99.67% -> 100.00%
Natural clean termination: 0.00% -> 15.00%
These are source FP16 metrics. The remaining gap to 100% is residual error after improvement, not evidence that the fine-tune made the base model worse.
Deployment Notes
Q4_K_M and Q8_0 have not yet received an independent 300-case ToolACE run, so their exact scores should not be assumed identical to FP16.
Schema validation and permission checks are universal tool-runtime safety requirements, not model-specific regressions.
Valid-name and exact-argument behavior improved strongly in the source model; rare errors can still occur on unseen tools.
MiniCPM5's parser-based serving path extracts the first completed XML call instead of relying solely on natural EOS.
vLLM GGUF support is experimental; merged safetensors remains the more reliable vLLM format.
llama.cpp behavior depends on build version and the frontend's chat-template handling.
Version History
The current Nemotron-DPO files are the July 2026 repaired exports. Legacy pre-Nemotron files are stored under legacy/ for reproducibility, and earlier repository states remain available from Hugging Face commit history.
MiniCPM5 Tool-Call Runtime Contract
MiniCPM5-1B has native XML-style tool calling. OpenBMB recommends SGLang for tool calling with its built-in minicpm5 parser:
The parser converts a completed <function ...>...</function> block into an OpenAI-compatible tool_calls response. A production agent should treat the first completed function block as the action boundary, validate it, execute the tool externally, and send the tool result in a new turn.
The reported stopped_cleanly_rate=0.15 is a strict natural-termination metric: only 15% of evaluated generations emitted a completed call and then naturally terminated at that boundary without runtime intervention. It does not mean that only 15% produced usable tool calls. On the same 300 cases, the repaired source model achieved:
parseable_rate=0.9933
valid_name_rate=0.9700
expected_name_rate=0.9267
no_repetition_rate=1.0000
Therefore, natural EOS-at-boundary remains weak, while first-call extraction and selection are much stronger. Use the official parser where supported and configure bounded generation or parser-side stopping after the first complete </function>.
Base model architecture
MiniCPM5-1B uses a standard LlamaForCausalLM architecture:
MiniCPM5-1B has a built-in <think>...</think> chat template. The same checkpoint can act as a fast assistant or a deliberate chain-of-thought reasoner — controlled by a single flag:
python
1# Fast mode — recommended for tool calling (thinking OFF)2prompt = tokenizer.apply_chat_template(3 messages, tools=tools, add_generation_prompt=True,4 enable_thinking=False,5 tokenize=False,6)78# Reasoning mode (thinking ON — NOT recommended for tool calling)9prompt = tokenizer.apply_chat_template(10 messages, tools=tools, add_generation_prompt=True,11 enable_thinking=True,12 tokenize=False,13)
Important: always use enable_thinking=False for tool/function calling. With thinking ON the model spends its token budget inside <think>...</think> and may not reach a completed function call. All benchmark numbers in this card use thinking OFF.
Citation
If you use this model, please cite the base model paper:
bibtex
1@article{minicpm4,
2 title = {MiniCPM4: Ultra-Efficient LLMs on End Devices},
3 author = {MiniCPM Team},
4 journal = {arXiv preprint arXiv:2506.07900},
5 year = {2025}
6}
And the ToolACE dataset used for fine-tuning:
bibtex
1@article{toolace,
2 title = {ToolACE: Winning the Points of LLM Function Calling},
3 author = {Liu, Ying and others},
4 journal = {arXiv preprint arXiv:2409.00920},
5 year = {2024}
6}
ModelScope
The base model is also available on ModelScope (for users in China and East Asia):
