This is a smaller version of the intfloat/multilingual-e5-base with only some Russian (Cyrillic in general) and English (fever) tokens (and embeddings) left.
The CulturaX dataset was used to search for the required tokens. As a result, out of 250k tokens of the original model, only 69,382 required were left.
Was the model trained in any way?
No. The tokenizer has been modified, and all changes to token identifiers have been corrected by moving embeddings in the model word_embeddings module to their new places, so the quality of this model on Cyrilic (and English) is exactly the same as the original one.
Why do we need this?
This allows you to use significantly less memory during training and also greatly reduces the weight of the model.
