SentenceTransformer based on BAAI/bge-small-en-v1.5

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-small-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Information Retrieval

• Dataset: eval
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 1,760 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 9 tokens mean: 24.87 tokens max: 70 tokens	min: 11 tokens mean: 68.37 tokens max: 500 tokens

• Samples:

  anchor	positive
  How is a proper coloring of a graph defined in the context of vertices and edges?	"A coloring is called proper if for each edge joining two distinct vertices, the two vertices it joins have different colors."
  What is the relationship between the first excited state of the box model and the p orbitals in a hydrogen atom?	"The p orbitals are similar to the first excited state of the box, i.e. $(n_{x},n_{y},n_{z})=(2,1,1)$ is similar to a $p_{x}$ orbital, $(n_{x},n_{y},n_{z})=(1,2,1)$ is similar to a $p_{y}$ orbital and $(n_{x},n_{y},n_{z})=(1,1,2)$ is similar to a $p_{z}$ orbital."
  How can the behavior of the derivative ( f'(x) ) indicate the presence of a local maximum or minimum at a critical point ( x=a )?	"If there is a local maximum when ( x=a ), the function must be lower near ( x=a ) than it is right at ( x=a ). If the derivative exists near ( x=a ), this means ( f'(x)>0 ) when ( x ) is near ( a ) and ( x < a ), because the function must 'slope up' just to the left of ( a ). Similarly, ( f'(x) < 0 ) when ( x ) is near ( a ) and ( x>a ), because ( f ) slopes down from the local maximum as we move to the right. Using the same reasoning, if there is a local minimum at ( x=a ), the derivative of ( f ) must be negative just to the left of ( a ) and positive just to the right."

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Evaluation Dataset

Unnamed Dataset

• Size: 420 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 420 samples:

  	anchor	positive
  type	string	string
  details	min: 12 tokens mean: 24.97 tokens max: 66 tokens	min: 7 tokens mean: 68.52 tokens max: 452 tokens

• Samples:

  anchor	positive
  What are the two central classes mentioned in the FileSystem framework and what do they represent?	"The class [object Object] is the most important entry point to the framework." and "FileSystem is a powerful and elegant library to manipulate files."
  What is the significance of Turing's work in the context of PDE-based models for self-organization of complex systems?	"Turing’s monumental work on the chemical basis of morphogenesis played an important role in igniting researchers’ attention to the PDE-based continuous field models as a mathematical framework to study self-organization of complex systems."
  What are the two options for reducing accelerations as discussed in the passage?	"From the above definitions we see that there are really two options for reducing accelerations. We can reduce the amount that velocity changes, or we can increase the time over which the velocity changes (or both)."

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• weight_decay: 0.05
• num_train_epochs: 10
• warmup_ratio: 0.1
• fp16: True
• eval_on_start: True

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.12.9
• Sentence Transformers: 4.1.0
• Transformers: 4.52.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.7.0
• Datasets: 3.6.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss