You can deploy this model and start earning money today!

tonythethompson/
Phi-4-Mini-Instruct-CPU-INT4-ONNX

Name: Phi-4-Mini-Instruct-CPU-INT4-ONNX
Availability: InStock

Views

No views yet

Phi-4 Mini Instruct — CPU INT4 (ONNX Runtime GenAI)

This repository repackages the cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4 configuration from Microsoft's official ONNX release microsoft/Phi-4-mini-instruct-onnx. It is not a newly trained model. This is an unofficial community repackage; Microsoft is the original author and license holder.

Source

Field	Value
Upstream model	microsoft/Phi-4-mini-instruct-onnx
Upstream source revision	`fc04c8f93df696602fd9f300a30d1bf2e3081347`
Export tool/script	Microsoft ONNX Runtime GenAI model builder (upstream Phi ONNX bundle)
Quantization recipe	ONNX Runtime GenAI RTN INT4 (`cpu-int4-rtn-block-32-acc-level-4`)

Files

All files are under cpu_and_mobile/cpu-int4-rtn-block-32-acc-level-4/:

File	Size	Description
`model.onnx`	~50 MB	INT4 ONNX graph
`model.onnx.data`	~5 GB	External weights
`genai_config.json`	~1 KB	ONNX Runtime GenAI session config
`config.json`	~2 KB	Model config
`configuration_phi3.py`	~11 KB	Phi-3/4 config class
`tokenizer.json` + `tokenizer_config.json`	~15 MB	Tokenizer
`vocab.json` + `merges.txt`	~6 MB	Vocabulary / merges
`added_tokens.json` + `special_tokens_map.json`	<1 KB	Special tokens

Intended Use

A 4-bit, CPU-targeted build of Phi-4-Mini-Instruct for local text generation and translation on CPU-only or low-VRAM hardware via ONNX Runtime GenAI. A GPU variant is in tonythethompson/phi-4-mini-instruct-gpu-int4-onnx.

Runtime Notes

Designed for ONNX Runtime GenAI compatible runtimes.
Execution provider: CPU (onnxruntime-genai).
Context length: 128K tokens (inherited from Phi-4-Mini-Instruct).
Microsoft's published throughput examples: ~16-24 tokens/sec on server-class Xeon CPUs; ~3-5 tokens/sec on laptop CPUs. Validate on your hardware before production use.

Precision and Packaging

Export tooling, precision, and quantization are recorded in the Source table above. This packaging mirror does not publish independent parity benchmarks; validate on your target execution provider before production use.

Limitations

CPU inference is substantially slower than GPU inference; not suitable for real-time use on most hardware.
INT4 quantization introduces approximation error relative to the base FP16/FP32 model.
No repository-specific translation quality evaluation is documented here.
Validate on your target language pair before production use.

License

MIT — inherited from microsoft/Phi-4-mini-instruct-onnx. This packaging repo adds no new license terms.

tonythethompson/Phi-4-Mini-Instruct-CPU-INT4-ONNX