ESM2 Quantized is an adapted version of the ESM2 architectures. It uses local attention instead of global attention, allowing for models with longer input sizes. ESM2 Quantized models have a context size of 2,050, double that of the standard ESM2 model. This kind of model was trained with int4 quantization. Several ESM2 Quantized models are available:
