You can deploy this model and start earning money today!

ENOT-AutoDL/
gpt2-tensorrt

Views

No views yet

GPT2

This repository contains GPT2 onnx models compatible with TensorRT:

Quantization of models was performed by the ENOT-AutoDL framework. Code for building of TensorRT engines and examples published on github.

	TensorRT INT8+FP32	torch FP16
Lambada Acc	72.11%	71.43%

Input sequance length	Number of generated tokens	TensorRT INT8+FP32 ms	torch FP16 ms	Acceleration
64	64	462	1190	2.58
64	128	920	2360	2.54
64	256	1890	4710	2.54

Example of inference and accuracy test published on github:

git clone https://github.com/ENOT-AutoDL/ENOT-transformers