SentenceTransformer based on sentence-transformers/all-MiniLM-L6-v2

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L6-v2
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 57,126 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 3 tokens mean: 58.68 tokens max: 133 tokens	min: 11 tokens mean: 143.97 tokens max: 256 tokens

• Samples:

  anchor	positive
  advanced march lecture : lps weights ola svensson1 this lecture do the : we ( actually hedge method. solve lps. fast very solving lps approximately. version 11 of topics in tcs, ” were simon rodriguez the by that used in the last lecture. recall last the lecture, saw how fairly follow the of recall that game t and n experts was : for : i ∈ n gives up or ) based the expert, up 3. with the expert advice ’ decides the up / down	advanced algorithms march 22, 2022 lecture 9 : solving lps using multiplicative weights notes by ola svensson1 in this lecture we do the following : • we describe the multiplicative weight update ( actually hedge ) method. • we then use this method to solve covering lps. • this is a very fast and simple ( i. e., very attractive ) method for solving these lps approximately. these lecture notes are partly based on an updated version of “ lecture 11 of topics in tcs, 2015 ” that were written by vincent eggerling and simon rodriguez and on the lecture notes by shiva kaul that we used in the last lecture. 1 recall last lecture in the previous lecture, we saw how to use the weighted majority method in order to fairly smartly follow the advice of experts. recall that the general game - setting with t days and n experts was as follows : for t = 1,..., t : 1. each expert i ∈ [ n ] gives some advice : up or down 2. aggregator ( you ) predicts, based on the advice of the expert, up or down. 3. ad...
  or down predicts, up down. adversary, of expert the the / down 4. aggregator the parameterized > 0 “ rate now as : • expert i ( 1. ( experts are in begin - at each t • predict / based weighted vote w = w t w n observing the set ( t i = w ( i · ( 1−ε i ] was ( trustworthiness experts. lecture the when / 2. the sequence of outcomes, t, and expert ∈ [ n ], of wm mistakes ≤2	up or down 2. aggregator ( you ) predicts, based on the advice of the expert, up or down. 3. adversary, with knowledge of the expert advice and the aggregator ’ s decision, decides the up / down outcome. 4. aggregator observes the outcome and [UNK] if his prediction was incorrect. the weighted majority algorithm, parameterized by [UNK] > 0 ( the “ learning rate ” ), now works as follows : • assign each expert i a weight w ( 1 ) i initialized to 1. ( all experts are equally trustworthy in the begin - ning. ) at each time t : • predict up / down based on a weighted majority vote per w ( t ) = ( w ( t ) 1,..., w ( t ) n ). • after observing the cost vector, set w ( t + 1 ) i = w ( t ) i · ( 1−ε ) for every expert i ∈ [ n ] whose prediction was wrong. ( discount the trustworthiness of erroneous experts. ) last lecture we analyzed the case when [UNK] = 1 / 2. the same proof gives the following theorem 1 for any sequence of outcomes, duration t, and expert i ∈ [ n ], # of wm mistakes ≤2
  ) last lecture analyzed [UNK] = 1 / 2. the following theorem sequence outcomes, duration t, and n wm ≤2 [UNK] ( ’ + o ( ( ). notes as for lecturer. have not inconsistent omit citations 1	##roneous experts. ) last lecture we analyzed the case when [UNK] = 1 / 2. the same proof gives the following theorem 1 for any sequence of outcomes, duration t, and expert i ∈ [ n ], # of wm mistakes ≤2 ( 1 + [UNK] ) · ( # of i ’ s mistakes ) + o ( log ( n ) / [UNK] ). 1disclaimer : these notes were written as notes for the lecturer. they have not been peer - reviewed and may contain inconsistent notation, typos, and omit citations of relevant works. 1

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Evaluation Dataset

Unnamed Dataset

• Size: 6,348 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 14 tokens mean: 97.31 tokens max: 142 tokens	min: 53 tokens mean: 238.84 tokens max: 256 tokens

• Samples:

  anchor	positive
  more at gesic doses. be predominantly however, it also the μ agonist weak or partial nist it mixed available. it used orally however, its injection miscellaneous tramadol is on blockade has been norepinephrine function. because it partially is μ agonist. the recommended mg orally times daily. association with ; drug contraindicated in history of epilepsy with that lower the serious risk the development sero toni	##phine but appears to produce more sedation at equianal - gesic doses. butorphanol is considered to be predominantly a κ agonist. however, it may also act as a partial agonist or antagonist at the μ receptor. benzomorphans pentazocine is a κ agonist with weak μ - antagonist or partial ago - nist properties. it is the oldest mixed agent available. it may be used orally or parenterally. however, because of its irritant properties, the injection of pentazocine subcutaneously is not recommended. miscellaneous tramadol is a centrally acting analgesic whose mechanism of action is predominantly based on blockade of serotonin reuptake. tramadol has also been found to inhibit norepinephrine transporter function. because it is only partially antagonized by naloxone, it is believed to be only a weak μ - receptor agonist. the recommended dosage is 50 – 100 mg orally four times daily. toxicity includes association with seizures ; the drug is relatively contraindicated in patients with a history of...
  ##ly four times daily. toxicity includes relatively in a of the serious is the of - inhibitor ( ). typically abate after several days of is no clinically respiration or tem thus far given that action of tramadol largely - serve as an adjunct pure opioid treatment of chronic is newer with modest μ significant norepinephrine - inhibiting models, its effects moderately by naloxone but reduced adrenoceptor antagonist. furthermore, norepinephrine	##ly four times daily. toxicity includes association with seizures ; the drug is relatively contraindicated in patients with a history of epilepsy and for use with other drugs that lower the seizure threshold. another serious risk is the development of sero - tonin syndrome, especially if selective serotonin reuptake inhibitor ( ssri ) antidepressants are being administered ( see chapter 16 ). other side effects include nausea and dizziness, but these symptoms typically abate after several days of therapy. it is surprising that no clinically significant effects on respiration or the cardiovascular sys - tem have thus far been reported. given the fact that the analgesic action of tramadol is largely independent of μ - receptor action, tra - madol may serve as an adjunct with pure opioid agonists in the treatment of chronic neuropathic pain. tapentadol is a newer analgesic with modest μ - opioid receptor affinity and significant norepinephrine reuptake - inhibiting action. in animal mode...
  - action. in its analgesic effects were moderately by strongly adrenoceptor antagonist. porter ( 6 was than of its the transporter ( of tapentadol 2008 been shown to as oxycodone the to gastrointesti complaints nausea. carries risk for for is how in cal to tramadol mechanism based opioid antitussives the effective drugs suppression of this is analgesia. in effect	- inhibiting action. in animal models, its analgesic effects were only moderately reduced by naloxone but strongly reduced by an α 2 - adrenoceptor antagonist. furthermore, its binding to the norepinephrine trans - porter ( net, see chapter 6 ) was stronger than that of tramadol, whereas its binding to the serotonin transporter ( sert ) was less than that of tramadol. tapentadol was approved in 2008 and has been shown to be as effective as oxycodone in the treatment of moderate to severe pain but with a reduced profile of gastrointesti - nal complaints such as nausea. tapentadol carries risk for seizures in patients with seizure disorders and for the development of sero - tonin syndrome. it is unknown how tapentadol compares in clini - cal utility to tramadol or other analgesics whose mechanism of action is not based primarily on opioid receptor pharmacology. antitussives the opioid analgesics are among the most effective drugs available for the suppression of cough. this effect is oft...

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• learning_rate: 2e-05
• num_train_epochs: 5
• warmup_ratio: 0.1
• fp16: True
• dataloader_drop_last: True
• dataloader_num_workers: 2
• load_best_model_at_end: True
• push_to_hub: True
• hub_model_id: RikoteMaster/retriever_pdf_and_books
• hub_strategy: end
• hub_private_repo: False

All Hyperparameters

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.17
• Sentence Transformers: 4.1.0
• Transformers: 4.52.3
• PyTorch: 2.7.0+cu126
• Accelerate: 1.7.0
• Datasets: 3.6.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss