1from sentence_transformers import SentenceTransformer
23# Download from the 🤗 Hub4model = SentenceTransformer("tomaarsen/static-similarity-mrl-multilingual-v1")5# Run inference6sentences =[7'It is known for its dry red chili powder .',8'It is popular for dry red chili powder .',9'These monsters will move in large groups .',10]11embeddings = model.encode(sentences)12print(embeddings.shape)13# [3, 1024]1415# Get the similarity scores for the embeddings16similarities = model.similarity(embeddings, embeddings)17print(similarities.shape)18# [3, 3]
This model was trained with Matryoshka loss, allowing this model to be used with lower dimensionalities with minimal performance loss.
Notably, a lower dimensionality allows for much faster downstream tasks, such as clustering or classification. You can specify a lower dimensionality with the truncate_dim argument when initializing the Sentence Transformer model:
python
1from sentence_transformers import SentenceTransformer
23model = SentenceTransformer("tomaarsen/static-similarity-mrl-multilingual-v1", truncate_dim=256)4embeddings = model.encode([5"I used to hate him.",6"Раньше я ненавидел его."7])8print(embeddings.shape)9# => (2, 256)
Evaluation
We've evaluated the model on 5 languages which have a lot of benchmarks across various tasks on MTEB.
We want to reiterate that this model is not intended for retrieval use cases. Instead, we evaluate on Semantic Textual Similarity (STS), Classification, and Pair Classification. We compare against the excellent and small multilingual-e5-small model.
To make up for this performance reduction, static-similarity-mrl-multilingual-v1 is approximately ~125x faster on CPU and ~10x faster on GPU devices than multilingual-e5-small. Due to the super-linear nature of attention models, versus the linear nature of static embedding models, the speedup will only grow larger as the number of tokens to encode increases.
Matryoshka Evaluation
Lastly, we experimented with the impacts on English STS on MTEB performance when we did Matryoshka-style dimensionality reduction by truncating the output embeddings to a lower dimensionality.
English STS MTEB performance vs Matryoshka dimensionality reduction
As you can see, you can easily reduce the dimensionality by 2x or 4x with minor (0.15% or 0.56%) performance hits. If the speed of your downstream task or your storage costs are a bottleneck, this should allow you to alleviate some of those concerns.
Approximate statistics based on the first 1000 samples:
english
non_english
type
string
string
details
min: 0 characters
mean: 147.77 characters
max: 668 characters
min: 0 characters
mean: 153.13 characters
max: 971 characters
Samples:
english
non_english
(SK) I would like to stress three key points in relation to this issue.
(SK) Chtěla bych zdůraznit tři klíčové body, které jsou s tímto tématem spojeny.
Women have a higher recorded rate of unemployment, especially long term unemployment.
Blandt kvinder registreres større arbejdsløshed, især blandt langtidsarbejdsløse.
You will recall that we have occasionally had disagreements over how to interpret Rule 166 of our Rules of Procedure and that certain Members thought that the Presidency was not applying it properly, since it was not giving the floor for points of order that did not refer to the issue that was being debated at that moment.
De husker nok, at vi til tider har været uenige om fortolkningen af artikel 166 i vores forretningsorden, og at nogle af medlemmerne mente, at formanden ikke anvendte den korrekt, eftersom han ikke gav ordet til indlæg til forretningsordenen, når det ikke drejede sig om det spørgsmål, der blev drøftet på det pågældende tidspunkt.
Approximate statistics based on the first 1000 samples:
post1
post2
type
string
string
details
min: 77 characters
mean: 669.56 characters
max: 3982 characters
min: 81 characters
mean: 641.44 characters
max: 4053 characters
Samples:
post1
post2
New user question about passwords Just got a refurbished computer with Ubuntu as the OS. Have never even heard of the OS and now I'm trying to learn. When I boot the system, it starts up great. But, if I try to navigate around, it requires a password. Is there a trick to finding the initial password? Please advise.
How do I reset a lost administrative password? I'm working on a Ubuntu system, and my client has completely forgotten his administrative password. He doesn't even remember entering one; however it is there. I've tried the suggestions on the website, and I have been unsuccessful in deleting the password so that I can download applets required for running some files. Is there a solution?
Reorder a list of string randomly but constant in a period of time I need to reorder a list in a random way but I want to have the same result on a short period of time ... So I have: var list = new String[] { "Angie", "David", "Emily", "James" } var shuffled = list.OrderBy(v => "4a78926c")).ToList(); But I always get the same order ... I could use Guid.NewGuid() but then I would have a different result in a short period of time. How can I do this?
Randomize a List What is the best way to randomize the order of a generic list in C#? I've got a finite set of 75 numbers in a list I would like to assign a random order to, in order to draw them for a lottery type application.
Made a mistake on check need help to fix I wrote a check and put the amount in the pay to order spot. Can I just mark it out, put the name in the spot and finish writing the check?
How to correct a mistake made when writing a check? I think I know the answer to this, but I'm not sure, and it's a good question, so I'll ask: What is the accepted/proper way to correct a mistake made on a check? For instance, I imagine that in any given January, some people accidentally date a check in the previous year. Is there a way to correct such a mistake, or must a check be voided (and wasted)? Pointers to definitive information (U.S., Canada, and elsewhere) are helpful.
Approximate statistics based on the first 1000 samples:
text
simplified
type
string
string
details
min: 18 characters
mean: 149.3 characters
max: 573 characters
min: 16 characters
mean: 123.58 characters
max: 576 characters
Samples:
text
simplified
The next morning , it had a small CDO and well-defined bands , and the system , either a weak tropical storm or a strong tropical depression , likely reached its peak .
The next morning , it had a small amounts of convection near the center and well-defined bands , and the system , either a weak tropical storm or a strong tropical depression , likely reached its peak .
The region of measurable parameter space that corresponds to a regime is very often loosely defined . Examples include [object Object] the steady state regime '' or `` the femtosecond regime '' .
This is common if a regime is threatened by another regime .
The Lamborghini Diablo is a high-performance mid-engined sports car that was built by Italian automaker Lamborghini between 1990 and 2001 .
The Lamborghini Diablo is a sport car that was built by Lamborghini from 1990 to 2001 .
Approximate statistics based on the first 1000 samples:
text
simplified
type
string
string
details
min: 13 characters
mean: 131.03 characters
max: 492 characters
min: 13 characters
mean: 112.41 characters
max: 492 characters
Samples:
text
simplified
Reinforcement and punishment are the core tools of operant conditioning .
Principles of operant conditioning :
The Japanese Ministry of Health , Labour and Welfare defines `` hikikomori '' as people who refuse to leave their house and , thus , isolate themselves from society in their homes for a period exceeding six months .
The Japanese Ministry of Health , Labour and Welfare defines hikikomori as people who refuse to leave their house for over six months .
It has six rows of black spines and has a pair of long , clubbed spines on the head .
It has a pair of long , clubbed spines on the head .
Approximate statistics based on the first 1000 samples:
anchor
positive
negative
type
string
string
string
details
min: 11 characters
mean: 87.69 characters
max: 483 characters
min: 7 characters
mean: 43.85 characters
max: 244 characters
min: 7 characters
mean: 43.87 characters
max: 172 characters
Samples:
anchor
positive
negative
A white horse and a rider wearing a ale blue shirt, white pants, and a black helmet are jumping a hurdle.
An equestrian is having a horse jump a hurdle.
A competition is taking place in a kitchen.
A group of people in a dome like building.
A gathering inside a building.
Cats are having a party.
Home to thousands of sheep and a few scattered farming families, the area is characterized by the stark beauty of bare peaks, rugged fells, and the most remote lakes, combined with challenging, narrow roads.
There are no wide and easy roads going through the area.
Approximate statistics based on the first 1000 samples:
english
non_english
type
string
string
details
min: 0 characters
mean: 148.52 characters
max: 1215 characters
min: 0 characters
mean: 154.44 characters
max: 1316 characters
Samples:
english
non_english
Mr Schmidt, Mr Trichet, I absolutely cannot go along with these proposals.
Pane Schmidte, pane Trichete, s těmito návrhy nemohu vůbec souhlasit.
The Council and Parliament recently adopted the regulation on the Single European Sky, one of the provisions of which was Community membership of Eurocontrol, so that Parliament has already indirectly expressed its views on this matter.
Der Rat und das Parlament haben kürzlich die Verordnung über die Schaffung eines einheitlichen europäischen Luftraums verabschiedet, in der unter anderem die Mitgliedschaft der Gemeinschaft bei Eurocontrol festgelegt ist, so dass das Parlament seine Auffassungen hierzu indirekt bereits dargelegt hat.
It was held over from the January part-session until this part-session.
Ihre Behandlung wurde von der Januar-Sitzung auf die jetzige vertagt.
Approximate statistics based on the first 1000 samples:
english
non_english
type
string
string
details
min: 3 characters
mean: 115.61 characters
max: 629 characters
min: 3 characters
mean: 121.61 characters
max: 664 characters
Samples:
english
non_english
Haiti: Security vs. Relief? · Global Voices
Haïti : Zones rouges, zones vertes - sécurité contre aide humanitaire ?
In order to prevent weapon smuggling through tunnels, his forces would have fought and killed Palestinians over a sustained period of time.
Con el fin de impedir el contrabando de armas a través de túneles, sus fuerzas habrían combatido y muerto palestinos durante un largo período de tiempo.
Tombstone of Vitalis, an ancient Roman cavalry officer, displayed in front of the Skopje City Museum.
Lápida de Vitalis, un antiguo oficial romano de caballería, exhibida frente al Museo de la Ciudad de Skopje.
Approximate statistics based on the first 1000 samples:
post1
post2
type
string
string
details
min: 64 characters
mean: 669.92 characters
max: 4103 characters
min: 62 characters
mean: 644.68 characters
max: 4121 characters
Samples:
post1
post2
Find the particular solution for this linear ODE $y' '-2y'+5y=e^x \cos2x$. Find the particular solution for this linear ODE :$y' '-2y'+5y=e^x \cos2x$. How can I use Undetermined coefficients method ?
Particular solution of $y''-4y'+5y = 4e^{2x} (\sin x)$ How do I find the particular solution of this second order inhomogenous differential equation? (Using undetermined coefficients). $y''-4y'+5y = 4e^{2x} (\sin x)$ I can find the generel homogenous solutions but I need help for the particular.
Unbounded sequence has an divergent subsequence Show that if $(x_n)$ is unbounded, then there exists a subsequence $(x_{n_k})$ such that $\lim 1/(x_{n_k}) =0.$ I was thinking that $(x_n)$ is a subsequence of itself. WLOG, suppose $(x_n)$ does not have an upper bound. By Algebraic Limit Theorem, $\lim 1/(x_{n_k}) =0.$ Is there any flaws in my proof?
Given the sequence $(x_n)$ is unbounded, show that there exist a subsequence $(x_{n_k})$ such that $\lim(1/x_n)=0$. Given the sequence $(x_n)$ is unbounded, show that there exist a subsequence $(x_{n_k})$ such that $\lim(1/x_{n_k})=0$. I guess I have to prove that $(x_{n_k})$ diverge, but I don't know how to carry on. Thanks.
"The problem is who can we get to replace her" vs. "The problem is who we can get to replace her" "The problem is who can we get to replace her" vs. "The problem is who we can get to replace her" Which one is correct and why?
Changing subject and verb positions in statements and questions We always change subject and verb positions in whenever we want to ask a question such as "What is your name?". But when it comes to statements like the following, which form is correct? I don't understand what are you talking about. I don't understand what you are talking about. Another example Do you know what time is it? Do you know what time it is? Another example Do you care how do I feel about this? Do you care how I feel about this?
Approximate statistics based on the first 1000 samples:
anchor
positive
negative
type
string
string
string
details
min: 15 characters
mean: 52.48 characters
max: 164 characters
min: 12 characters
mean: 52.86 characters
max: 162 characters
min: 12 characters
mean: 56.18 characters
max: 298 characters
Samples:
anchor
positive
negative
Is pornography an art?
Can pornography be art?
Does pornography involve the objectification of women?
How can I improve my speaking in public?
How can I improve my public speaking ability?
How do I improve my vocabulary and English speaking skills? I am a 22 year old software engineer and come from a Telugu medium background. I am able to write well, but my speaking skills are poor.
Approximate statistics based on the first 1000 samples:
anchor
positive
negative
type
string
string
string
details
min: 15 characters
mean: 72.82 characters
max: 300 characters
min: 12 characters
mean: 34.11 characters
max: 126 characters
min: 11 characters
mean: 36.38 characters
max: 121 characters
Samples:
anchor
positive
negative
Two women are embracing while holding to go packages.
Two woman are holding packages.
The men are fighting outside a deli.
Two young children in blue jerseys, one with the number 9 and one with the number 2 are standing on wooden steps in a bathroom and washing their hands in a sink.
Two kids in numbered jerseys wash their hands.
Two kids in jackets walk to school.
A man selling donuts to a customer during a world exhibition event held in the city of Angeles
Approximate statistics based on the first 1000 samples:
text
simplified
type
string
string
details
min: 24 characters
mean: 147.36 characters
max: 599 characters
min: 19 characters
mean: 124.94 characters
max: 540 characters
Samples:
text
simplified
It marks the southernmost point of the Bahà a de Banderas , upon which the port and resort city of Puerto Vallarta stands .
It is the most southern point of the Bahà a de Banderas .
The interiors of the stations resemble that of the former western Soviet nations , with chandeliers hanging from the corridors .
Its interior resembles that of western former Soviet nations with chandeliers hanging from the corridors .
The Senegal national football team , nicknamed the Lions of Teranga , is the national team of Senegal and is controlled by the Fà dà ration Sà nà galaise de Football .
Senegal national football team is the national football team of Senegal .
1@inproceedings{reimers-2019-sentence-bert,
2 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
3 author = "Reimers, Nils and Gurevych, Iryna",
4 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
5 month = "11",
6 year = "2019",
7 publisher = "Association for Computational Linguistics",
8 url = "https://arxiv.org/abs/1908.10084",
9}
MatryoshkaLoss
bibtex
1@misc{kusupati2024matryoshka,
2 title={Matryoshka Representation Learning},
3 author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
4 year={2024},
5 eprint={2205.13147},
6 archivePrefix={arXiv},
7 primaryClass={cs.LG}
8}
MultipleNegativesRankingLoss
bibtex
1@misc{henderson2017efficient,
2 title={Efficient Natural Language Response Suggestion for Smart Reply},
3 author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
4 year={2017},
5 eprint={1705.00652},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL}
8}
