SentenceTransformer based on intfloat/e5-large-v2

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/e5-large-v2
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 784,827 training samples
• Columns: query, answer, and label
• Approximate statistics based on the first 1000 samples:

  	query	answer	label
  type	string	string	int
  details	min: 60 tokens mean: 77.86 tokens max: 93 tokens	min: 3 tokens mean: 8.84 tokens max: 70 tokens	0: ~96.70% 1: ~3.30%

• Samples:

  query	answer	label
  query: The study addresses the challenge of action segmentation under weak supervision, where the available ground truth only indicates the presence of actions without providing their temporal ordering or occurrence timing in training videos. This limitation necessitates the development of a method to generate pseudo-ground truth for effective training and improve performance in action segmentation and alignment tasks.We suggest combining 'a Hidden Markov Model' and	a multilayer perceptron	1
  query: The study addresses the challenge of action segmentation under weak supervision, where the available ground truth only indicates the presence of actions without providing their temporal ordering or occurrence timing in training videos. This limitation necessitates the development of a method to generate pseudo-ground truth for effective training and improve performance in action segmentation and alignment tasks.We suggest combining 'a Hidden Markov Model' and	an optimal transport problem	0
  query: The study addresses the challenge of action segmentation under weak supervision, where the available ground truth only indicates the presence of actions without providing their temporal ordering or occurrence timing in training videos. This limitation necessitates the development of a method to generate pseudo-ground truth for effective training and improve performance in action segmentation and alignment tasks.We suggest combining 'a Hidden Markov Model' and	a context enhancement module	0

• Loss: ContrastiveLoss with these parameters:
  json

  1{ 2 "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE", 3 "margin": 0.5, 4 "size_average": true 5}

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 64
• learning_rate: 4.0560820385265185e-06
• warmup_ratio: 0.21933051020273267
• bf16: True
• prompts: {'query': 'query: '}
• batch_sampler: no_duplicates

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.11.2
• Sentence Transformers: 3.3.1
• Transformers: 4.49.0
• PyTorch: 2.5.1+cu124
• Accelerate: 1.0.1
• Datasets: 3.1.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

ContrastiveLoss

• 🌐 Project
• 📃 Paper
• 🛠️ Code