[2025.06.06] MiniCPM4 series are released! This model achieves ultimate efficiency improvements while maintaining optimal performance at the same scale! It can achieve over 5x generation acceleration on typical end-side chips! You can find technical report here.🔥🔥🔥
MiniCPM4 Series
MiniCPM4 series are highly efficient large language models (LLMs) designed explicitly for end-side devices, which achieves this efficiency through systematic innovation in four key dimensions: model architecture, training data, training algorithms, and inference systems.
MiniCPM4-8B: The flagship of MiniCPM4, with 8B parameters, trained on 8T tokens.
MiniCPM4-0.5B: The small version of MiniCPM4, with 0.5B parameters, trained on 1T tokens.
MiniCPM4-8B-Eagle-FRSpec: Eagle head for FRSpec, accelerating speculative inference for MiniCPM4-8B. (<-- you are here)
MiniCPM4-8B-Eagle-FRSpec-QAT-cpmcu: Eagle head trained with QAT for FRSpec, efficiently integrate speculation and quantization to achieve ultra acceleration for MiniCPM4-8B.
MiniCPM4-8B-Eagle-vLLM: Eagle head in vLLM format, accelerating speculative inference for MiniCPM4-8B.
MiniCPM4-8B-marlin-Eagle-vLLM: Quantized Eagle head for vLLM format, accelerating speculative inference for MiniCPM4-8B.
BitCPM4-0.5B: Extreme ternary quantization applied to MiniCPM4-0.5B compresses model parameters into ternary values, achieving a 90% reduction in bit width.
BitCPM4-1B: Extreme ternary quantization applied to MiniCPM3-1B compresses model parameters into ternary values, achieving a 90% reduction in bit width.
MiniCPM4-Survey: Based on MiniCPM4-8B, accepts users' quiries as input and autonomously generate trustworthy, long-form survey papers.
MiniCPM4-MCP: Based on MiniCPM4-8B, accepts users' queries and available MCP tools as input and autonomously calls relevant MCP tools to satisfy users' requirements.
Introduction
MiniCPM4-8B-Eagle-FRSpec is a Eagle model trained with MiniCPM4-8B. It clould be apply on our inference framework cpm.cu with FRSpec, accelerating the generation speed by 7 times compared to Qwen3-8B.
Usage
Inference with cpm.cu
# case 1: verify model is fp16 or bf16
cd cpm.cu/tests
python3 test_generate.py \
--no-apply-quant \
--no-apply-eagle-quant
# case 2: verify model is quanted with Marlin (W4A16, group size = 128)
cd cpm.cu/tests
python3 test_generate.py \
--apply-quant \
--no-apply-eagle-quant
Evaluation
Tested on two representative edge devices, the Jetson AGX Orin and RTX 4090, MiniCPM4 with MiniCPM4-8B-Eagle-FRSpec demonstrates significantly superior processing speed over models of comparable size for long-text processing tasks. Its performance advantage becomes increasingly pronounced as the text length increases. On the Jetson AGX Orin platform, MiniCPM4 achieves approximately a 7x improvement in generation speed compared to Qwen3-8B.
speed test
Statement
As a language model, MiniCPM generates content by learning from a vast amount of text.
However, it does not possess the ability to comprehend or express personal opinions or value judgments.
Any content generated by MiniCPM does not represent the viewpoints or positions of the model developers.
Therefore, when using content generated by MiniCPM, users should take full responsibility for evaluating and verifying it on their own.
LICENSE
This repository and MiniCPM models are released under the Apache-2.0 License.
Citation
Please cite our paper if you find our work valuable.
bibtex
1@article{minicpm4,
2 title={{MiniCPM4}: Ultra-Efficient LLMs on End Devices},
3 author={MiniCPM Team},
4 year={2025}
5}
