SentenceTransformer based on sentence-transformers/all-MiniLM-L12-v2

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L12-v2
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Information Retrieval

• Dataset: climate-val-simpler
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 3,316 training samples
• Columns: text_a and text_b
• Approximate statistics based on the first 1000 samples:

  	text_a	text_b
  type	string	string
  details	min: 9 tokens mean: 26.15 tokens max: 82 tokens	min: 9 tokens mean: 36.84 tokens max: 199 tokens

• Samples:

  text_a	text_b
  The thermal expansion of the oceans, compounded by melting glaciers, resulted in the highest global sea level on record in 2015.	Since the last glacial maximum about 20,000 years ago, the sea level has risen by more than 125 metres (410 ft), with rates varying from less than a mm/year to 40+ mm/year, as a result of melting ice sheets over Canada and Eurasia.
  The thermal expansion of the oceans, compounded by melting glaciers, resulted in the highest global sea level on record in 2015.	This acceleration is due mostly to human-caused global warming, which is driving thermal expansion of seawater and the melting of land-based ice sheets and glaciers.
  The thermal expansion of the oceans, compounded by melting glaciers, resulted in the highest global sea level on record in 2015.	Between 1993 and 2018, thermal expansion of the oceans contributed 42% to sea level rise; the melting of temperate glaciers, 21%; Greenland, 15%; and Antarctica, 8%.

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Evaluation Dataset

Unnamed Dataset

• Size: 380 evaluation samples
• Columns: text_a and text_b
• Approximate statistics based on the first 380 samples:

  	text_a	text_b
  type	string	string
  details	min: 7 tokens mean: 28.0 tokens max: 70 tokens	min: 7 tokens mean: 39.17 tokens max: 475 tokens

• Samples:

  text_a	text_b
  Postma's model contains many simple errors; in no way does Postma undermine the existence or necessity of the greenhouse effect.	Since it is absurd to have no logical method for settling on one hypothesis amongst an infinite number of equally data-compliant hypotheses, we should choose the simplest theory: "Either science is irrational [in the way it judges theories and predictions probable] or the principle of simplicity is a fundamental synthetic a priori truth."
  Postma's model contains many simple errors; in no way does Postma undermine the existence or necessity of the greenhouse effect.	Thus, complex hypotheses must predict data much better than do simple hypotheses before researchers reject the simple hypotheses.
  Postma's model contains many simple errors; in no way does Postma undermine the existence or necessity of the greenhouse effect.	Minimum description length Minimum message length – Formal information theory restatement of Occam's Razor Newton's flaming laser sword Philosophical razor – Principle or rule of thumb that allows one to eliminate unlikely explanations for a phenomenon Philosophy of science – The philosophical study of the assumptions, foundations, and implications of science Simplicity "Ockham's razor does not say that the more simple a hypothesis, the better."

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 4
• warmup_ratio: 0.1
• load_best_model_at_end: True

All Hyperparameters

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.11.12
• Sentence Transformers: 4.1.0
• Transformers: 4.51.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.6.0
• Datasets: 2.14.4
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss