SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-mpnet-base-v2
• Maximum Sequence Length: 384 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 784,827 training samples
• Columns: query, answer, and label
• Approximate statistics based on the first 1000 samples:

  	query	answer	label
  type	string	string	int
  details	min: 60 tokens mean: 77.86 tokens max: 93 tokens	min: 3 tokens mean: 8.82 tokens max: 64 tokens	0: ~96.70% 1: ~3.30%

• Samples:

  query	answer	label
  Background: The study addresses the challenge of action segmentation under weak supervision, where the available ground truth only indicates the presence of actions without providing their temporal ordering or occurrence timing in training videos. This limitation necessitates the development of a method to generate pseudo-ground truth for effective training and improve performance in action segmentation and alignment tasks.[object Object]Contribution: Combine 'a Hidden Markov Model' and	a multilayer perceptron	1
  Background: The study addresses the challenge of action segmentation under weak supervision, where the available ground truth only indicates the presence of actions without providing their temporal ordering or occurrence timing in training videos. This limitation necessitates the development of a method to generate pseudo-ground truth for effective training and improve performance in action segmentation and alignment tasks.[object Object]Contribution: Combine 'a Hidden Markov Model' and	synthetic occlusion augmentation during training	0
  Background: The study addresses the challenge of action segmentation under weak supervision, where the available ground truth only indicates the presence of actions without providing their temporal ordering or occurrence timing in training videos. This limitation necessitates the development of a method to generate pseudo-ground truth for effective training and improve performance in action segmentation and alignment tasks.[object Object]Contribution: Combine 'a Hidden Markov Model' and	robustness of deep learning methods	0

• Loss: ContrastiveLoss with these parameters:
  json

  1{ 2 "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE", 3 "margin": 0.5, 4 "size_average": true 5}

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 64
• learning_rate: 1.9218937402834593e-05
• num_train_epochs: 2
• warmup_ratio: 0.08278167292320517
• bf16: True
• batch_sampler: no_duplicates

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.11.2
• Sentence Transformers: 3.3.1
• Transformers: 4.49.0
• PyTorch: 2.5.1+cu124
• Accelerate: 1.0.1
• Datasets: 3.1.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

ContrastiveLoss

• 🌐 Project
• 📃 Paper
• 🛠️ Code