This is a sentence-transformers model finetuned from intfloat/multilingual-e5-small. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: intfloat/multilingual-e5-small
Maximum Sequence Length: 512 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("codersan/validadted_e5Fa_onV8d")
# Run inference
sentences = [
    'کودک جوان دارد اسکوتر سه چرخ را روبه پایین در پیاده رو می راند.',
    'کودک جوانی دارد اسکوتر سه چرخ را  روبه پایین در پیاده رو می راند.',
    'کتاب قابوس نامه اثر عنصرالمعالی کیکاووس بن اسکندر می باشد.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

Size: 21,484 training samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 4 tokens
mean: 21.18 tokens
max: 107 tokens
min: 5 tokens
mean: 21.09 tokens
max: 77 tokens

Samples:

anchor	positive
`کارگردان چگونه بر یک نمایش تئاتری تأثیر می‌گذارد؟`	`کارگردان نورپردازی و جلوه‌های صوتی را که در نمایش استفاده خواهد شد انتخاب می‌کند، که بر حال و هوا و جو اجرای نمایش تأثیر می‌گذارد.`
`پیش از پیدایش شهر اراک گویش‌های متفاوتی در منطقه وجود داشت، اما با مهاجرت گروه‌های مختلف و ساکنان آن‌ها در شهر ترکیب خاصی از لهجه‌های مختلف به وجود آمد که امروزه به نام لهجه اراکی شناخته می‌شود.`	`لهجه اراکی ترکیبی از لهجه های مختلف است`
`اهمیت تاریخی واتیکان چیست؟`	`واتیکان مرکز روحانی و اداری کلیسای کاتولیک رومی است و برای قرن‌ها یک نهاد مذهبی و سیاسی مهم بوده است.`

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
learning_rate: 5e-06
weight_decay: 0.01
num_train_epochs: 10
warmup_ratio: 0.1
push_to_hub: True
hub_model_id: codersan/validadted_e5Fa_onV8d
eval_on_start: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-06
weight_decay: 0.01
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 10
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: True
resume_from_checkpoint: None
hub_model_id: codersan/validadted_e5Fa_onV8d
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: True
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss
0	0	-
0.0372	100	0.6517
0.0745	200	0.5496
0.1117	300	0.4086
0.1489	400	0.2798
0.1862	500	0.1495
0.2234	600	0.0888
0.2606	700	0.0624
0.2978	800	0.0511
0.3351	900	0.0595
0.3723	1000	0.0521
0.4095	1100	0.0362
0.4468	1200	0.0321
0.4840	1300	0.0538
0.5212	1400	0.0328
0.5585	1500	0.0491
0.5957	1600	0.0475
0.6329	1700	0.0693
0.6701	1800	0.0369
0.7074	1900	0.0386
0.7446	2000	0.057
0.7818	2100	0.0314
0.8191	2200	0.0373
0.8563	2300	0.031
0.8935	2400	0.0352
0.9308	2500	0.0456
0.9680	2600	0.0528
1.0052	2700	0.0418
1.0424	2800	0.0398
1.0797	2900	0.0304
1.1169	3000	0.0262
1.1541	3100	0.0513
1.1914	3200	0.0421
1.2286	3300	0.029
1.2658	3400	0.0356
1.3031	3500	0.0245
1.3403	3600	0.0293
1.3775	3700	0.0245
1.4147	3800	0.0174
1.4520	3900	0.013
1.4892	4000	0.0322
1.5264	4100	0.0204
1.5637	4200	0.0292
1.6009	4300	0.0325
1.6381	4400	0.0448
1.6754	4500	0.015
1.7126	4600	0.0134
1.7498	4700	0.0264
1.7870	4800	0.0112
1.8243	4900	0.0169
1.8615	5000	0.0225
1.8987	5100	0.0251
1.9360	5200	0.0116
1.9732	5300	0.032
2.0104	5400	0.0215
2.0477	5500	0.0298
2.0849	5600	0.0195
2.1221	5700	0.0151
2.1593	5800	0.0253
2.1966	5900	0.0155
2.2338	6000	0.02
2.2710	6100	0.0173
2.3083	6200	0.0082
2.3455	6300	0.0133
2.3827	6400	0.0119
2.4200	6500	0.0084
2.4572	6600	0.0048
2.4944	6700	0.0231
2.5316	6800	0.005
2.5689	6900	0.0181
2.6061	7000	0.0222
2.6433	7100	0.0257
2.6806	7200	0.0056
2.7178	7300	0.0047
2.7550	7400	0.0143
2.7923	7500	0.0074
2.8295	7600	0.0054
2.8667	7700	0.0098
2.9039	7800	0.0134
2.9412	7900	0.0083
2.9784	8000	0.0206
3.0156	8100	0.0073
3.0529	8200	0.0132
3.0901	8300	0.0119
3.1273	8400	0.0084
3.1646	8500	0.0125
3.2018	8600	0.0093
3.2390	8700	0.0171
3.2762	8800	0.012
3.3135	8900	0.0094
3.3507	9000	0.0043
3.3879	9100	0.0066
3.4252	9200	0.0035
3.4624	9300	0.0044
3.4996	9400	0.0104
3.5369	9500	0.0058
3.5741	9600	0.008
3.6113	9700	0.0175
3.6485	9800	0.0078
3.6858	9900	0.0033
3.7230	10000	0.0042
3.7602	10100	0.0058
3.7975	10200	0.0063
3.8347	10300	0.0025
3.8719	10400	0.0067
3.9092	10500	0.0095
3.9464	10600	0.0119
3.9836	10700	0.0102
4.0208	10800	0.004
4.0581	10900	0.0069
4.0953	11000	0.0088
4.1325	11100	0.0067
4.1698	11200	0.0077
4.2070	11300	0.0057
4.2442	11400	0.0123
4.2815	11500	0.007
4.3187	11600	0.0039
4.3559	11700	0.0034
4.3931	11800	0.0046
4.4304	11900	0.002
4.4676	12000	0.0058
4.5048	12100	0.0046
4.5421	12200	0.0063
4.5793	12300	0.0148
4.6165	12400	0.0064
4.6538	12500	0.0043
4.6910	12600	0.0015
4.7282	12700	0.0033
4.7655	12800	0.0038
4.8027	12900	0.0018
4.8399	13000	0.0015
4.8771	13100	0.0038
4.9144	13200	0.0123
4.9516	13300	0.0063
4.9888	13400	0.0047
5.0261	13500	0.003
5.0633	13600	0.005
5.1005	13700	0.0037
5.1378	13800	0.0051
5.1750	13900	0.0031
5.2122	14000	0.0024
5.2494	14100	0.0089
5.2867	14200	0.005
5.3239	14300	0.002
5.3611	14400	0.0015
5.3984	14500	0.0014
5.4356	14600	0.0014
5.4728	14700	0.0034
5.5101	14800	0.0041
5.5473	14900	0.0041
5.5845	15000	0.0091
5.6217	15100	0.0073
5.6590	15200	0.0019
5.6962	15300	0.0012
5.7334	15400	0.0022
5.7707	15500	0.0026
5.8079	15600	0.0018
5.8451	15700	0.0013
5.8824	15800	0.0036
5.9196	15900	0.0114
5.9568	16000	0.0047
5.9940	16100	0.0031
6.0313	16200	0.0025
6.0685	16300	0.0023
6.1057	16400	0.0019
6.1430	16500	0.0031
6.1802	16600	0.0026
6.2174	16700	0.0014
6.2547	16800	0.0074
6.2919	16900	0.0013
6.3291	17000	0.0037
6.3663	17100	0.0013
6.4036	17200	0.0014
6.4408	17300	0.001
6.4780	17400	0.0018
6.5153	17500	0.0028
6.5525	17600	0.0044
6.5897	17700	0.0085
6.6270	17800	0.0046
6.6642	17900	0.0015
6.7014	18000	0.0012
6.7386	18100	0.0013
6.7759	18200	0.0016
6.8131	18300	0.0012
6.8503	18400	0.0017
6.8876	18500	0.0012
6.9248	18600	0.0058
6.9620	18700	0.0042
6.9993	18800	0.0019
7.0365	18900	0.003
7.0737	19000	0.0015
7.1109	19100	0.0018
7.1482	19200	0.0026
7.1854	19300	0.0015
7.2226	19400	0.0053
7.2599	19500	0.0019
7.2971	19600	0.0009
7.3343	19700	0.0021
7.3716	19800	0.0016
7.4088	19900	0.0007
7.4460	20000	0.0007
7.4832	20100	0.0016
7.5205	20200	0.001
7.5577	20300	0.0019
7.5949	20400	0.008
7.6322	20500	0.004
7.6694	20600	0.001
7.7066	20700	0.0008
7.7439	20800	0.0011
7.7811	20900	0.0014
7.8183	21000	0.0012
7.8555	21100	0.0012
7.8928	21200	0.0009
7.9300	21300	0.0059
7.9672	21400	0.0036
8.0045	21500	0.0018
8.0417	21600	0.0013
8.0789	21700	0.0025
8.1162	21800	0.0009
8.1534	21900	0.0016
8.1906	22000	0.001
8.2278	22100	0.0054
8.2651	22200	0.0012
8.3023	22300	0.001
8.3395	22400	0.0014
8.3768	22500	0.0008
8.4140	22600	0.0007
8.4512	22700	0.0008
8.4885	22800	0.0017
8.5257	22900	0.0009
8.5629	23000	0.0028
8.6001	23100	0.0075
8.6374	23200	0.0021
8.6746	23300	0.0008
8.7118	23400	0.0007
8.7491	23500	0.0011
8.7863	23600	0.0012
8.8235	23700	0.0007
8.8608	23800	0.0011
8.8980	23900	0.0041
8.9352	24000	0.0015
8.9724	24100	0.0031
9.0097	24200	0.0008
9.0469	24300	0.001
9.0841	24400	0.0017
9.1214	24500	0.0007
9.1586	24600	0.0017
9.1958	24700	0.0014
9.2331	24800	0.0039
9.2703	24900	0.0012
9.3075	25000	0.0007
9.3448	25100	0.0028
9.3820	25200	0.0011
9.4192	25300	0.0008
9.4564	25400	0.0007
9.4937	25500	0.0014
9.5309	25600	0.0006
9.5681	25700	0.0021
9.6054	25800	0.0086
9.6426	25900	0.0023
9.6798	26000	0.0006
9.7171	26100	0.0008
9.7543	26200	0.0011
9.7915	26300	0.0007
9.8287	26400	0.0009
9.8660	26500	0.001
9.9032	26600	0.0056
9.9404	26700	0.0024
9.9777	26800	0.0024

Framework Versions

Python: 3.10.12
Sentence Transformers: 3.3.1
Transformers: 4.47.0
PyTorch: 2.5.1+cu121
Accelerate: 1.2.1
Datasets: 3.2.0
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

bibtex

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

	anchor	positive
type	string	string
details	min: 4 tokens mean: 21.18 tokens max: 107 tokens	min: 5 tokens mean: 21.09 tokens max: 77 tokens