SentenceTransformer based on sentence-transformers/LaBSE

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/LaBSE
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 768, 'bias': True, 'activation_function': 'torch.nn.modules.activation.Tanh'})
  (3): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 172,826 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 5 tokens mean: 15.2 tokens max: 80 tokens	min: 5 tokens mean: 15.47 tokens max: 50 tokens	min: 0.76 mean: 0.95 max: 1.0

• Samples:

  sentence1	sentence2	score
  تفاوت بین تحلیلگر تحقیقات بازار و تحلیلگر تجارت چیست؟	تفاوت بین تحقیقات بازاریابی و تحلیلگر تجارت چیست؟	0.982593297958374
  خوردن چه چیزی باعث دل درد میشود؟	چه چیزی باعث رفع دل درد میشود؟	0.9582258462905884
  بهترین نرم افزار ویرایش ویدیویی کدام است؟	بهترین نرم افزار برای ویرایش ویدیو چیست؟	0.9890836477279663

• Loss: CosineSimilarityLoss with these parameters:
  json

  1{ 2 "loss_fct": "torch.nn.modules.loss.MSELoss" 3}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 12
• learning_rate: 1e-05
• weight_decay: 0.01
• num_train_epochs: 5
• warmup_ratio: 0.1
• push_to_hub: True
• hub_model_id: codersan/validadted_faLabse_withCosSimb
• eval_on_start: True
• batch_sampler: no_duplicates

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.47.0
• PyTorch: 2.5.1+cu121
• Accelerate: 1.2.1
• Datasets: 3.2.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers