You can deploy this model and start earning money today!

litert-community/
MiniCPM5-1B

Views

No views yet

MiniCPM5-1B (LiteRT-LM)

This repository hosts the LiteRT-LM (LiteRT formerly known as TensorFlow Lite) version of MiniCPM5-1B, optimized for fully on-device inference on mobile and edge hardware.

Available Models

minicpm_wi4b32_wi8_afp32.litertlm: This model features mixed INT4-block32(linear)/INT8(embed and lmhead) quantization (wi4b32_wi8) with FP32 activations (afp32).
minicpm_wi4b32_wi8_afp32_gpu_opt.litertlm: This model uses same quantization recipe with minicpm_wi4b32_wi8_afp32 but optimized for GPU execution.
MiniCPM5-1B_dynamic_wi8_afp32.litertlm: This model features dynamic weight-only INT8 quantization (wi8) with FP32 activations (afp32). It is configured with static prefill memory allocation.

Quantization Benchmark

The table below compares the accuracy of the FP baseline and the LiteRT-LM quantized model on the same evaluation benchmark. Scores are reported as percentages.

Category	FP (bf16)	W4
Overall	66.16	60.91
STEM	58.07	53.36
Social Science	77.91	70.87
Humanities	67.65	64.35
Other	68.71	62.14
Hard	47.08	44.23

FP (bf16) is the full-precision baseline. W4 uses mixed INT4-block32 (linear) / INT8 (embed and lmhead) quantization (wi4b32_wi8) with FP32 activations (afp32). Thinking mode has been turned off to reduce context usage. All evaluations use a temperature of 0.6, top-p of 0.95, and a random seed of 0.

What is MiniCPM?

MiniCPM5-1B is the first model in the MiniCPM5 series from OpenBMB. It is a dense 1B-parameter Transformer built specifically for on-device, local, and resource-constrained deployment, while reaching 1B-class open-source SOTA in its size class.

Highlights

🏆 1B-class open-source SOTA — strongest in tool use, code generation, and difficult reasoning among comparable open models.
🧠 Hybrid Reasoning — a single checkpoint serves as both a fast assistant and a deliberate reasoner via a built-in <think> template (enable_thinking).
📏 Long context — native 131,072-token context length.
📱 Built for the edge — compact footprint designed for local assistants, coding agents, and tool-use workflows.

Model Information

Item	Value
Type	Causal Language Model
Architecture	Standard `LlamaForCausalLM`
Parameters	1,080,632,832 (~1B)
Non-Embedding Parameters	679,552,512
Layers	24
Attention Heads (GQA)	16 (Q) / 2 (KV)
Context Length	131,072

Use the model

Edge Gallery App (Android)

Get the App: Install the app from Google Play or download the latest APK from the GitHub releases page.
Importing the Model: Navigate to the Model manager within the app and click the "+" (plus) icon in the bottom-right corner. Two options will appear:
- Import from HF (Recommended): Select this option, and a dialog box will appear showing an example Hugging Face model URL. Enter the HF link for the desired .litertlm model and click submit. The model will then appear in your list, and you can proceed to download it (a Hugging Face account login is required).
- From local model file: First, download the .litertlm model directly to your Android device, OR download it to your computer and push it via ADB (e.g., adb push MiniCPM5-1B_dynamic_wi8_afp32.litertlm /sdcard/Download/). Then, select this option, choose the downloaded file from your storage, configure your preferred parameters, and tap "Import".

For full details on importing models and other features, see the Edge Gallery App Wiki.

To build the demo app from source, please follow the instructions from the GitHub repository.

Try It (Desktop/CLI)

Install uv and run the model directly from the LiteRT-LM command line:

bash

uv tool install litert-lm
uvx litert-lm run --from-huggingface-repo=litert-community/MiniCPM5-1B MiniCPM5-1B_dynamic_wi8_afp32.litertlm --prompt="What is the capital of France?"

License

Released under the Apache-2.0 License, consistent with the upstream openbmb/MiniCPM5-1B.

Citation

bibtex

@article{minicpm4,
  title={MiniCPM4: Ultra-efficient LLMs on end devices},
  author={MiniCPM, Team},
  journal={arXiv preprint arXiv:2506.07900},
  year={2025}
}

litert-community/MiniCPM5-1B