Computationally efficient CNN architecture designed specifically for mobile devices with very limited computing power.
Description
ShuffleNet is a deep convolutional network for image classification. ShuffleNetV2 is an improved architecture that is the state-of-the-art in terms of speed and accuracy tradeoff used for image classification.
Compared with the fp32 ShuffleNet-v2, int8 ShuffleNet-v2's Top-1 error rising ratio is 0.59%, Top-5 error rising ratio is 1.71% and performance improvement is 1.62x.
Note the performance depends on the test hardware.
Performance data here is collected with Intel® Xeon® Platinum 8280 Processor, 1s 4c per instance, CentOS Linux 8.3, data batch size is 1.
Inference
This script converts the ShuffleNetv2 model from PyTorch to ONNX and uses ONNX Runtime for inference.
Input to model
Input to the model are 3-channel RGB images of shape (3 x H x W), where H and W are expected to be at least 224.
data_0: float[1, 3, 224, 224]
Preprocessing steps
All pre-trained models expect input images normalized in the same way. The images have to be loaded in to a range of [0, 1] and then normalized using mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225].
Output of this model is tensor of shape 1000, with confidence scores over ImageNet's 1000 classes.
softmax_1: float[1, 1000]
Dataset (Train and Validation)
Models are pretrained on ImageNet.
For training we use train+valset in COCO except for 5000 images from minivalset, and use the minivalset to test.
Details of performance on COCO object detection are provided in this paper
Quantization
ShuffleNet-v2-int8 and ShuffleNet-v2-int8 are obtained by quantizing ShuffleNet-v2-fp32 model. We use Intel® Neural Compressor with onnxruntime backend to perform quantization. View the instructions to understand how to use Intel® Neural Compressor for quantization.
Make sure to specify the appropriate dataset path in the configuration file.
bash
1bash run_tuning.sh --input_model=path/to/model \# model path as *.onnx2--config=shufflenetv2.yaml \3--output_model=path/to/save
Model inference
We use onnxruntime to perform ShuffleNetv2_fp32 and ShuffleNetv2_int8 inference. View the notebook onnxrt_inference to understand how to use these 2 models for doing inference as well as which preprocess and postprocess we use.
