SentenceTransformer(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'XLMRobertaModel'})
(1): Pooling({'embedding_dimension': 1024, 'pooling_mode': 'cls', 'include_prompt': True})
(2): Normalize({})
)pip install -U sentence-transformers1from sentence_transformers import SentenceTransformer
2
3# Download from the 🤗 Hub
4model = SentenceTransformer("sentence_transformers_model_id")
5# Run inference
6sentences = [
7 "I've been grieving a significant loss for a long time, and while I know I need to move forward, my thoughts constantly pull me back to the past. How do I let go and find peace?",
8 'uddhared ātmanātmānaṃ nātmānam avasādayet | ātmaiva hy ātmano bandhur ātmaiva ripur ātmanaḥ ||5||',
9 'etair vimuktaḥ kaunteya tamo-dvārais tribhir naraḥ | ācaraty ātmanaḥ śreyas tato yāti parāṃ gatim ||22||',
10]
11embeddings = model.encode(sentences)
12print(embeddings.shape)
13# [3, 1024]
14
15# Get the similarity scores for the embeddings
16similarities = model.similarity(embeddings, embeddings)
17print(similarities)
18# tensor([[1.0000, 0.4964, 0.1087],
19# [0.4964, 1.0000, 0.3406],
20# [0.1087, 0.3406, 1.0000]])sentence_0, sentence_1, and sentence_2| sentence_0 | sentence_1 | sentence_2 | |
|---|---|---|---|
| type | string | string | string |
| modality | text | text | text |
| details |
|
|
|
| sentence_0 | sentence_1 | sentence_2 |
|---|---|---|
As a professional, I feel constantly burnt out, always chasing the next promotion or project. I've lost touch with why I even started, and joy seems like a distant memory. Is there a way to reconnect with my passion? | yaṃ labdhvā cāparaṃ lābhaṃ manyate nādhikaṃ tataḥ | yasmin sthito na duḥkhena guruṇāpi vicālyate ||22|| taṃ vidyād duḥkha-saṃyoga-viyogaṃ yoga-saṃjñitam | sa niścayena yoktavyo yogo 'nirviṇṇa-cetasā ||23|| | yaṃ hi na vyathayanty ete puruṣaṃ puruṣarṣabha | sama-duḥkha-sukhaṃ dhīraṃ so 'mṛtatvāya kalpate ||15|| |
My teenage son is rebelling and pushing me away. I feel like I'm losing him. What can I do? | ayaneṣu ca sarveṣu yathābhāgam avasthitāḥ | bhīṣmam evābhirakṣantu bhavantaḥ sarva eva hi ||11|| | acchedyo 'yam adāhyo 'yam akledyo 'śoṣya eva ca | nityaḥ sarva-gataḥ sthāṇur acalo 'yaṃ sanātanaḥ ||24|| |
I'm constantly worried about the future – what if my plans fail? What if things don't go my way? This anxiety paralyzes me and prevents me from acting. | yajñadānatapaḥkarma na tyājyaṃ kāryam eva tat | yajño dānaṃ tapaś caiva pāvanāni manīṣiṇām ||5|| | ahiṃsā samatā tuṣṭis tapo dānaṃ yaśo 'yaśaḥ | bhavanti bhāvā bhūtānāṃ matta eva pṛthagvidhāḥ ||5|| |
MultipleNegativesRankingLoss with these parameters:
1{
2 "scale": 20.0,
3 "similarity_fct": "cos_sim",
4 "gather_across_devices": false,
5 "directions": [
6 "query_to_doc"
7 ],
8 "partition_mode": "joint",
9 "hardness_mode": null,
10 "hardness_strength": 0.0
11}per_device_train_batch_size: 16num_train_epochs: 2per_device_eval_batch_size: 16multi_dataset_batch_sampler: round_robinper_device_train_batch_size: 16num_train_epochs: 2max_steps: -1learning_rate: 5e-05lr_scheduler_type: linearlr_scheduler_kwargs: Nonewarmup_steps: 0optim: adamw_torch_fusedoptim_args: Noneweight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08optim_target_modules: Nonegradient_accumulation_steps: 1average_tokens_across_devices: Truemax_grad_norm: 1label_smoothing_factor: 0.0bf16: Falsefp16: Falsebf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonegradient_checkpointing: Falsegradient_checkpointing_kwargs: Nonetorch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneuse_liger_kernel: Falseliger_kernel_config: Noneuse_cache: Falseneftune_noise_alpha: Nonetorch_empty_cache_steps: Noneauto_find_batch_size: Falselog_on_each_node: Truelogging_nan_inf_filter: Trueinclude_num_input_tokens_seen: nolog_level: passivelog_level_replica: warningdisable_tqdm: Falseproject: huggingfacetrackio_space_id: Nonetrackio_bucket_id: Nonetrackio_static_space_id: Noneper_device_eval_batch_size: 16prediction_loss_only: Trueeval_on_start: Falseeval_do_concat_batches: Trueeval_use_gather_object: Falseeval_accumulation_steps: Noneinclude_for_metrics: []batch_eval_metrics: Falsesave_only_model: Falsesave_on_each_node: Falseenable_jit_checkpoint: Falsepush_to_hub: Falsehub_private_repo: Nonehub_model_id: Nonehub_strategy: every_savehub_always_push: Falsehub_revision: Noneload_best_model_at_end: Falseignore_data_skip: Falserestore_callback_states_from_checkpoint: Falsefull_determinism: Falseseed: 42data_seed: Noneuse_cpu: Falseaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedataloader_drop_last: Falsedataloader_num_workers: 0dataloader_pin_memory: Truedataloader_persistent_workers: Falsedataloader_prefetch_factor: Noneremove_unused_columns: Truelabel_names: Nonetrain_sampling_strategy: randomlength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falseddp_static_graph: Noneddp_backend: Noneddp_timeout: 1800fsdp: Nonefsdp_config: Nonedeepspeed: Nonedebug: []skip_memory_metrics: Truedo_predict: Falseresume_from_checkpoint: Nonewarmup_ratio: Nonelocal_rank: -1prompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: round_robinrouter_mapping: {}learning_rate_mapping: {}| Epoch | Step | Training Loss |
|---|---|---|
| 1.6447 | 500 | 2.8599 |
1@inproceedings{reimers-2019-sentence-bert,
2 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
3 author = "Reimers, Nils and Gurevych, Iryna",
4 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
5 month = "11",
6 year = "2019",
7 publisher = "Association for Computational Linguistics",
8 url = "https://arxiv.org/abs/1908.10084",
9}1@misc{oord2019representationlearningcontrastivepredictive,
2 title={Representation Learning with Contrastive Predictive Coding},
3 author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
4 year={2019},
5 eprint={1807.03748},
6 archivePrefix={arXiv},
7 primaryClass={cs.LG},
8 url={https://arxiv.org/abs/1807.03748},
9}