Sentence Transformers: Multilingual Sentence, Paragraph, and Image Embeddings using BERT & Co.
This framework provides an easy method to compute dense vector representations for sentences, paragraphs, and images. The models are based on transformer networks like BERT / RoBERTa / XLM-RoBERTa etc. and achieve state-of-the-art performance in various task. Text is embedding in vector space such that similar text is close and can efficiently be found using cosine similarity.
Alternatively, you can also clone the latest version from the repository and install it directly from the source code:
pip install -e .
PyTorch with CUDA
If you want to use a GPU / CUDA, you must install PyTorch with the matching CUDA Version. Follow
PyTorch - Get Started for further details how to install PyTorch.
1sentences =['This framework generates embeddings for each input sentence',2'Sentences are passed as a list of string.',3'The quick brown fox jumps over the lazy dog.']4sentence_embeddings = model.encode(sentences)
And that's it already. We now have a list of numpy arrays with the embeddings.
We provide a large list of Pretrained Models for more than 100 languages. Some models are general purpose models, while others produce embeddings for specific use cases. Pre-trained models can be loaded by just passing the model name: SentenceTransformer('model_name').
This framework allows you to fine-tune your own sentence embedding methods, so that you get task-specific sentence embeddings. You have various options to choose from in order to get perfect sentence embeddings for your specific task.
See Training Overview for an introduction how to train your own embedding models. We provide various examples how to train models on various datasets.
Some highlights are:
Support of various transformer networks including BERT, RoBERTa, XLM-R, DistilBERT, Electra, BART, ...
Multi-Lingual and multi-task learning
Evaluation during training to find optimal model
10+ loss-functions allowing to tune models specifically for semantic search, paraphrase mining, semantic similarity comparison, clustering, triplet loss, contrastive loss.
Performance
Our models are evaluated extensively on 15+ datasets including challening domains like Tweets, Reddit, emails. They achieve by far the best performance from all available sentence embedding methods. Further, we provide several smaller models that are optimized for speed.
1@inproceedings{reimers-2019-sentence-bert,
2 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
3 author = "Reimers, Nils and Gurevych, Iryna",
4 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
5 month = "11",
6 year = "2019",
7 publisher = "Association for Computational Linguistics",
8 url = "https://arxiv.org/abs/1908.10084",
9}
1@inproceedings{reimers-2020-multilingual-sentence-bert,
2 title = "Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation",
3 author = "Reimers, Nils and Gurevych, Iryna",
4 booktitle = "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing",
5 month = "11",
6 year = "2020",
7 publisher = "Association for Computational Linguistics",
8 url = "https://arxiv.org/abs/2004.09813",
9}
Please have a look at Publications for our different publications that are integrated into SentenceTransformers.
Don't hesitate to send us an e-mail or report an issue, if something is broken (and it shouldn't be) or if you have further questions.
This repository contains experimental software and is published for the sole purpose of giving additional background details on the respective publication.
