The linear modules vit.encoder.layer.5.output.dense, vit.encoder.layer.9.attention.attention.query.module, fall back to fp32 for less than 1% relative accuracy loss.

Evaluation result

	INT8	FP32
Accuracy (eval-acc)	80.576	81.326
Model size (MB)	94	331

Load with Intel® Neural Compressor:

python

from neural_compressor.utils.load_huggingface import OptimizedModel
int8_model = OptimizedModel.from_pretrained(
    'Intel/vit-base-patch16-224-int8-static',
)

INC4AI/vit-base-patch16-224-int8-static-inc

The INT8 model based on vit-base-patch16-224 which finetuned on imagenet-1k

Post-training static quantization

Evaluation result

Load with Intel® Neural Compressor:

INC4AI/
vit-base-patch16-224-int8-static-inc