t5-small-xsum-int8-dynamic-inc – AI Model by Intel | AlphaNeural AI

You can deploy this model and start earning money today!

Intel/
t5-small-xsum-int8-dynamic-inc

like

0

text2text-generation

Intel® Neural Compressor

neural-compressor

PostTrainingDynamic

text-generation-inference

endpoints_compatible

Views

No views yet

INT8 T5 small finetuned on XSum

Post-training dynamic quantization

This is an INT8 PyTorch model quantized with huggingface/optimum-intel through the usage of Intel® Neural Compressor.

The original fp32 model comes from the fine-tuned model adasnew/t5-small-xsum.

The linear modules lm.head, fall back to fp32 for less than 1% relative accuracy loss.

Evaluation result

	INT8	FP32
Accuracy (eval-rouge1)	29.9008	29.9592
Model size	154M	242M

Load with optimum:

python

from optimum.intel import INCModelForSeq2SeqLM

model_id = "Intel/t5-small-xsum-int8-dynamic-inc"
int8_model = INCModelForSeq2SeqLM.from_pretrained(model_id)