SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Triplet

• Dataset: gooqa-dev
• Evaluated with TripletEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 44,286 training samples
• Columns: question, context, and negative
• Approximate statistics based on the first 1000 samples:

  	question	context	negative
  type	string	string	string
  details	min: 7 tokens mean: 14.5 tokens max: 43 tokens	min: 33 tokens mean: 145.82 tokens max: 256 tokens	min: 29 tokens mean: 150.3 tokens max: 256 tokens

• Samples:

  question	context	negative
  When did more stringent testing determine that humans preferred a 24 hour day?	Early research into circadian rhythms suggested that most people preferred a day closer to 25 hours when isolated from external stimuli like daylight and timekeeping. However, this research was faulty because it failed to shield the participants from artificial light. Although subjects were shielded from time cues (like clocks) and daylight, the researchers were not aware of the phase-delaying effects of indoor electric lights.[dubious – discuss] The subjects were allowed to turn on light when they were awake and to turn it off when they wanted to sleep. Electric light in the evening delayed their circadian phase.[citation needed] A more stringent study conducted in 1999 by Harvard University estimated the natural human rhythm to be closer to 24 hours, 11 minutes: much closer to the solar day but still not perfectly in sync.	Population testing is still being done. Some Native American groups that have been sampled may not have shared the pattern of markers being searched for. Geneticists acknowledge that DNA testing cannot yet distinguish among members of differing cultural Native American nations. There is genetic evidence for three major migrations into North America, but not for more recent historic differentiation. In addition, not all Native Americans have been tested, so scientists do not know for sure that Native Americans have only the genetic markers they have identified.
  Southampton's range of industries includes the manufacture of cars and what other transport?	During the latter half of the 20th century, a more diverse range of industry also came to the city, including aircraft and car manufacture, cables, electrical engineering products, and petrochemicals. These now exist alongside the city's older industries of the docks, grain milling, and tobacco processing.	According to Hampshire Constabulary figures, Southampton is currently safer than it has ever been before, with dramatic reductions in violent crime year on year for the last three years. Data from the Southampton Safer City Partnership shows there has been a reduction in all crimes in recent years and an increase in crime detection rates. According to government figures Southampton has a higher crime rate than the national average. There is some controversy regarding comparative crime statisitics due to inconsistencies between different police forces recording methodologies. For example, in Hampshire all reported incidents are recorded and all records then retained. However, in neighbouring Dorset crimes reports withdrawn or shown to be false are not recorded, reducing apparent crime figures. In the violence against the person category, the national average is 16.7 per 1000 population while Southampton is 42.4 per 1000 population. In the theft from a vehicle category, the national aver...
  What models imply that changes in diversity are guided by a first-order positive feedback?	On the other hand, changes through the Phanerozoic correlate much better with the hyperbolic model (widely used in population biology, demography and macrosociology, as well as fossil biodiversity) than with exponential and logistic models. The latter models imply that changes in diversity are guided by a first-order positive feedback (more ancestors, more descendants) and/or a negative feedback arising from resource limitation. Hyperbolic model implies a second-order positive feedback. The hyperbolic pattern of the world population growth arises from a second-order positive feedback between the population size and the rate of technological growth. The hyperbolic character of biodiversity growth can be similarly accounted for by a feedback between diversity and community structure complexity. The similarity between the curves of biodiversity and human population probably comes from the fact that both are derived from the interference of the hyperbolic trend with cyclical and stochastic...	This still left open the question of whether the opposite of approach in the prefrontal cortex is better described as moving away (Direction Model), as unmoving but with strength and resistance (Movement Model), or as unmoving with passive yielding (Action Tendency Model). Support for the Action Tendency Model (passivity related to right prefrontal activity) comes from research on shyness and research on behavioral inhibition. Research that tested the competing hypotheses generated by all four models also supported the Action Tendency Model.

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Evaluation Dataset

Unnamed Dataset

• Size: 5,000 evaluation samples
• Columns: question, context, and negative_1
• Approximate statistics based on the first 1000 samples:

  	question	context	negative_1
  type	string	string	string
  details	min: 7 tokens mean: 14.58 tokens max: 34 tokens	min: 28 tokens mean: 149.66 tokens max: 256 tokens	min: 28 tokens mean: 146.44 tokens max: 256 tokens

• Samples:

  question	context	negative_1
  The Tethys sea developed during what period of time?	The formation of the Alps (the Alpine orogeny) was an episodic process that began about 300 million years ago. In the Paleozoic Era the Pangaean supercontinent consisted of a single tectonic plate; it broke into separate plates during the Mesozoic Era and the Tethys sea developed between Laurasia and Gondwana during the Jurassic Period. The Tethys was later squeezed between colliding plates causing the formation of mountain ranges called the Alpide belt, from Gibraltar through the Himalayas to Indonesia—a process that began at the end of the Mesozoic and continues into the present. The formation of the Alps was a segment of this orogenic process, caused by the collision between the African and the Eurasian plates that began in the late Cretaceous Period.	Sea levels began to rise during the Jurassic, which was probably caused by an increase in seafloor spreading. The formation of new crust beneath the surface displaced ocean waters by as much as 200 m (656 ft) more than today, which flooded coastal areas. Furthermore, Pangaea began to rift into smaller divisions, bringing more land area in contact with the ocean by forming the Tethys Sea. Temperatures continued to increase and began to stabilize. Humidity also increased with the proximity of water, and deserts retreated.
  What was the name of Charanjit Singh's 1982 album?	The electronic instrumentation and minimal arrangement of Charanjit Singh's Synthesizing: Ten Ragas to a Disco Beat (1982), an album of Indian ragas performed in a disco style, anticipated the sounds of acid house music, but it is not known to have had any influence on the genre prior to the album's rediscovery in the 21st century.	The electronic instrumentation and minimal arrangement of Charanjit Singh's Synthesizing: Ten Ragas to a Disco Beat (1982), an album of Indian ragas performed in a disco style, anticipated the sounds of acid house music, but it is not known to have had any influence on the genre prior to the album's rediscovery in the 21st century.
  Who did Deshin Shekpa persuade the Yongle Emperor to give the title to?	Throughout the following month, the Yongle Emperor and his court showered the Karmapa with presents. At Linggu Temple in Nanjing, he presided over the religious ceremonies for the Yongle Emperor's deceased parents, while twenty-two days of his stay were marked by religious miracles that were recorded in five languages on a gigantic scroll that bore the Emperor's seal. During his stay in Nanjing, Deshin Shekpa was bestowed the title "Great Treasure Prince of Dharma" by the Yongle Emperor. Elliot Sperling asserts that the Yongle Emperor, in bestowing Deshin Shekpa with the title of "King" and praising his mystical abilities and miracles, was trying to build an alliance with the Karmapa as the Mongols had with the Sakya lamas, but Deshin Shekpa rejected the Yongle Emperor's offer. In fact, this was the same title that Kublai Khan had offered the Sakya Phagpa lama, but Deshin Shekpa persuaded the Yongle Emperor to grant the title to religious leaders of other Tibetan Buddhist sects.	Throughout the following month, the Yongle Emperor and his court showered the Karmapa with presents. At Linggu Temple in Nanjing, he presided over the religious ceremonies for the Yongle Emperor's deceased parents, while twenty-two days of his stay were marked by religious miracles that were recorded in five languages on a gigantic scroll that bore the Emperor's seal. During his stay in Nanjing, Deshin Shekpa was bestowed the title "Great Treasure Prince of Dharma" by the Yongle Emperor. Elliot Sperling asserts that the Yongle Emperor, in bestowing Deshin Shekpa with the title of "King" and praising his mystical abilities and miracles, was trying to build an alliance with the Karmapa as the Mongols had with the Sakya lamas, but Deshin Shekpa rejected the Yongle Emperor's offer. In fact, this was the same title that Kublai Khan had offered the Sakya Phagpa lama, but Deshin Shekpa persuaded the Yongle Emperor to grant the title to religious leaders of other Tibetan Buddhist sects.

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 256
• num_train_epochs: 10
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: no_duplicates

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.11.0
• Sentence Transformers: 4.0.1
• Transformers: 4.50.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.2
• Datasets: 3.5.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss