This model is trained to compress embeddings generated by the roberta_zinc_480m model from the native size of 768 to compressed sizes 512, 256, 128, 64, and 32.

The compressed embeddings are trained to maintain cosine similarity computed using the native embeddings.