You can deploy this model and start earning money today!

vab46/
nomic-embed-text-v1.5_Clinical-Trials_Matryoshka

Name: nomic-embed-text-v1.5_Clinical-Trials_Matryoshka
Rating: 2.5 (60 reviews)

Views

No views yet

nomic-embed text-v1.5 Clinical-Trials Matryoshka

This is a sentence-transformers model finetuned from nomic-ai/nomic-embed-text-v1.5 on the dataset from Clinical_trials_anchor-positive-pairs_EmbeddingModel-data. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: nomic-ai/nomic-embed-text-v1.5
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Supported Modality: Text
Training Dataset: Clinical_trials_anchor-positive-pairs_EmbeddingModel-data
Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'NomicBertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("vab46/nomic-embed-text-v1.5_Clinical-Trials_Matryoshka")
# Run inference
documents = [
    "TITLE: Exergames-acceptance and Commitment Therapy(e-ACT) for Breast Cancer Depression and Anxiety: A Randomized Controlled Trial The Efficacy of Exergames-acceptance and Commitment Therapy Program for Treatment of Depression and Other Psychological Complications in Breast Cancer: Comparison With Acceptance and Commitment Therapy Alone and Treatment-as-usual in a Randomized Controlled Trial\nSUMMARY: The goal of this three arm, double-blind, randomized controlled trial is to learn if an exergame-acceptance and commitment therapy (e-ACT) program can treat depression and anxiety in breast cancer patients, compared to acceptance and commitment therapy (ACT) alone and treatment-as-usual. The main question it aims to answer is:\nIs there a difference in the effectiveness of the e-ACT program in reducing depressive and anxiety symptoms, cancer-related fatigue, experiential avoidance, and serum IL-6 levels, while increasing BDNF and improving posttraumatic growth and quality of life among breast cancer patients, compared to ACT alone and treatment-as-usual, measured at baseline, 8 weeks (post-intervention), and 12 weeks after the intervention (follow-up)?\nResearchers will compare the e-ACT group, the ACT-alone group, and the treatment-as-usual group to see if the e-ACT program yields superior outcomes in reducing psychological distress and improving well-being.\nParticipants will:\nBe randomly assigned to one of three groups: (1) e-ACT (exergame + ACT), (2) ACT alone, or (3) treatment-as-usual (general patient education).\nAttend an 8-week program (one session per week) if in the e-ACT or ACT group; the control group continues their usual care.\nComplete questionnaires at three time points (baseline, 8 weeks, and 20 weeks) to assess depression, anxiety, quality of life, posttraumatic growth, valued living, experiential avoidance, and cancer-related fatigue.\nProvide blood samples at pre-intervention and post-intervention (week 8) for analysis of interleukin-6 (IL-6) and brain-derived neurotrophic factor (BDNF) biomarkers.\nThe INCLUSION CRITERIA include:\n1. Newly diagnosed breast cancer patients and patients with recurrent breast cancer confirmed by histopathological report, regardless of the stage of cancer.\n2. Those with HADS score of 8 or higher in both Depression and Anxiety sub-scales of the HADS.\n3. Patients who had been treated with surgery or were undergoing the standard regime of clinical anti-tumor treatment (chemotherapy, radiotherapy, immunotherapy, targeted therapy, etc.).\n4. Age 18 years old and above.\n5. Patients who were able to read and understand written Chinese.\nThe EXCLUSION CRITERIA are:\n1. Pregnant women，considered for the following reasons: Pregnancy involves significant hormonal and physical changes that could affect the participant's response to the e-ACT; the safety of e-ACT to the unborn child was of concern.\n2. Those who have current and lifetime history of engaging in any psychotherapy\n3. Those who consumed alcohol and illicit drugs .\n4. Those who has current and lifetime history of other psychiatric illnesses, such as psychotic disorders (schizophrenia, schizophreniform disorder, schizoaffective disorders, brief psychotic disorder, and delusional disorder), bipolar mood disorder, obsessive compulsive disorder, posttraumatic stress disorder, and attention deficit hyperactive disorder, and autism spectrum disorder\n5. Those who are on medications that can induce psychiatric symptoms, such as cardiovascular agents (clonidine, guanethidine, methyldopa, reserpine, beta blockers), dermatologic agents (isotretinoin), anticonvulsants (levetiracetam), antimigraine medications (triptans), hormonal agents (corticosteroids, oral contraceptives, gonadotropin-releasing hormone agonists, tamoxifen), varenicline, immunological agents (interferons), and levodopa.Or those who ccurrently using any psychotropic medication.\n6. Patient who has suicidal tendency.\n7. those who are physically unfit to answer questionnaires. (those are bed-bound or too weak to answer the questionnaire).",
]
queries = [
    'Could a patient with a history of depression and anxiety, who has been diagnosed with breast cancer, qualify for this study?',
    'I have obesity and high blood pressure, can I participate in this study and what will happen if I join?',
    'How old do you need to be to be eligible for this clinical trial?',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.7282, 0.1019, 0.2957]])

Evaluation

Metrics

Information Retrieval

Dataset: dim_768, dim_512, dim_256, dim_128, dim_64
Evaluated with InformationRetrievalEvaluator with these parameters:

Metric	dim 756	dim 512	dim 256	dim 128	dim 64
cosine_accuracy@1	0.3887	0.3787	0.3725	0.345	0.2963
cosine_accuracy@3	0.5138	0.5138	0.4988	0.47	0.43
cosine_accuracy@5	0.5625	0.5575	0.5375	0.525	0.4813
cosine_accuracy@10	0.6062	0.5988	0.585	0.5675	0.5288
cosine_precision@1	0.3887	0.3787	0.3725	0.345	0.2963
cosine_precision@3	0.1713	0.1713	0.1663	0.1567	0.1433
cosine_precision@5	0.1125	0.1115	0.1075	0.105	0.0963
cosine_precision@10	0.0606	0.0599	0.0585	0.0567	0.0529
cosine_recall@1	0.3887	0.3787	0.3725	0.345	0.2963
cosine_recall@3	0.5138	0.5138	0.4988	0.47	0.43
cosine_recall@5	0.5625	0.5575	0.5375	0.525	0.4813
cosine_recall@10	0.6062	0.5988	0.585	0.5675	0.5288
cosine_ndcg@10	**0.4972	0.4891	0.4777	0.455	0.41**
cosine_mrr@10	0.4623	0.4538	0.4434	0.419	0.3722
cosine_map@100	0.467	0.4589	0.449	0.425	0.3788

Comparison with the baseline model performance on same data

metric	dimensions	base_model_value	FT_model_value	diff	%change
accuracy@1	768	0.315	0.41375	0.09875	31.34920635
accuracy@1	512	0.30125	0.415	0.11375	37.7593361
accuracy@1	256	0.28625	0.3925	0.10625	37.11790393
accuracy@1	128	0.26125	0.35875	0.0975	37.32057416
accuracy@1	64	0.21125	0.3025	0.09125	43.19526627
accuracy@10	64	0.39875	0.51125	0.1125	28.21316614
accuracy@10	256	0.46875	0.58625	0.1175	25.06666667
accuracy@10	128	0.4375	0.57375	0.13625	31.14285714
accuracy@10	768	0.50375	0.59875	0.095	18.85856079
accuracy@10	512	0.5	0.59625	0.09625	19.25
accuracy@3	768	0.42	0.52625	0.10625	25.29761905
accuracy@3	512	0.40375	0.52	0.11625	28.79256966
accuracy@3	256	0.38875	0.50375	0.115	29.58199357
accuracy@3	128	0.365	0.475	0.11	30.1369863
accuracy@3	64	0.2975	0.42375	0.12625	42.43697479
accuracy@5	64	0.345	0.46625	0.12125	35.14492754
accuracy@5	256	0.43	0.54	0.11	25.58139535
accuracy@5	128	0.39375	0.515	0.12125	30.79365079
accuracy@5	768	0.4575	0.55	0.0925	20.21857923
accuracy@5	512	0.44875	0.55375	0.105	23.39832869
map@100	768	0.382026465	0.48036608	0.098339615	25.74157148
map@100	512	0.370630917	0.478453773	0.107822857	29.09170609
map@100	256	0.351904934	0.459528498	0.107623564	30.58313571
map@100	128	0.324395831	0.431321436	0.106925606	32.96146109
map@100	64	0.272953217	0.376392287	0.103439069	37.89626304
mrr@10	256	0.346236111	0.454637897	0.108401786	31.3086308
mrr@10	64	0.266976687	0.369970734	0.102994048	38.57791816
mrr@10	128	0.318508433	0.426855655	0.108347222	34.01706553
mrr@10	512	0.364892857	0.473862103	0.108969246	29.8633541
mrr@10	768	0.376446429	0.475852679	0.09940625	26.40647977
ndcg@10	768	0.407159105	0.505456639	0.098297535	24.14229076
ndcg@10	512	0.397313669	0.503315103	0.106001433	26.67953348
ndcg@10	256	0.37577036	0.48623639	0.110466031	29.39721772
ndcg@10	128	0.347224832	0.462075633	0.1148508	33.07678184
ndcg@10	64	0.298362231	0.404066399	0.105704168	35.42813311
precision@1	64	0.21125	0.3025	0.09125	43.19526627
precision@1	128	0.26125	0.35875	0.0975	37.32057416
precision@1	256	0.28625	0.3925	0.10625	37.11790393
precision@1	512	0.30125	0.415	0.11375	37.7593361
precision@1	768	0.315	0.41375	0.09875	31.34920635
precision@10	768	0.050375	0.059875	0.0095	18.85856079
precision@10	512	0.05	0.059625	0.009625	19.25
precision@10	256	0.046875	0.058625	0.01175	25.06666667
precision@10	128	0.04375	0.057375	0.013625	31.14285714
precision@10	64	0.039875	0.051125	0.01125	28.21316614
precision@3	64	0.099166667	0.14125	0.042083333	42.43697479
precision@3	128	0.121666667	0.158333333	0.036666667	30.1369863
precision@3	256	0.129583333	0.167916667	0.038333333	29.58199357
precision@3	512	0.134583333	0.173333333	0.03875	28.79256966
precision@3	768	0.14	0.175416667	0.035416667	25.29761905
precision@5	768	0.0915	0.11	0.0185	20.21857923
precision@5	512	0.08975	0.11075	0.021	23.39832869
precision@5	256	0.086	0.108	0.022	25.58139535
precision@5	128	0.07875	0.103	0.02425	30.79365079
precision@5	64	0.069	0.09325	0.02425	35.14492754
recall@1	64	0.21125	0.3025	0.09125	43.19526627
recall@1	128	0.26125	0.35875	0.0975	37.32057416
recall@1	512	0.30125	0.415	0.11375	37.7593361
recall@1	768	0.315	0.41375	0.09875	31.34920635
recall@1	256	0.28625	0.3925	0.10625	37.11790393
recall@10	768	0.50375	0.59875	0.095	18.85856079
recall@10	512	0.5	0.59625	0.09625	19.25
recall@10	256	0.46875	0.58625	0.1175	25.06666667
recall@10	128	0.4375	0.57375	0.13625	31.14285714
recall@10	64	0.39875	0.51125	0.1125	28.21316614
recall@3	512	0.40375	0.52	0.11625	28.79256966
recall@3	768	0.42	0.52625	0.10625	25.29761905
recall@3	256	0.38875	0.50375	0.115	29.58199357
recall@3	128	0.365	0.475	0.11	30.1369863
recall@3	64	0.2975	0.42375	0.12625	42.43697479
recall@5	256	0.43	0.54	0.11	25.58139535
recall@5	128	0.39375	0.515	0.12125	30.79365079
recall@5	768	0.4575	0.55	0.0925	20.21857923
recall@5	512	0.44875	0.55375	0.105	23.39832869
recall@5	64	0.345	0.46625	0.12125	35.14492754
AVERAGE					30.22854974
VARIANCE					43.17916576
ST-DEV					6.571085584

Training Details

Training Dataset

json

Dataset: json
Size: 7,193 training samples
Columns: positive and anchor
Approximate statistics based on the first 100 samples:
positive anchor
type string string
modality text text
details
min: 271 tokens
mean: 671.29 tokens
max: 2110 tokens
min: 10 tokens
mean: 20.11 tokens
max: 35 tokens

Samples:

positive	anchor
TITLE: Comparison of Postoperative Analgesic Efficacy of Two Different Blocks in Modified Radical Mastectomy Surgery Comparison of Postoperative Analgesic Efficacy of Superior Posterior Serratus Intercostal Plane Block (SPSIP) and Anterior Serratus Plane Block in Modified Radical Mastectomy Surgery[object Object]SUMMARY: Postoperative pain following modified radical mastectomy remains a significant clinical challenge and may delay recovery, impair patient comfort, and increase opioid consumption. Ultrasound-guided fascial plane blocks have become an important component of multimodal analgesia for breast surgery. The serratus anterior plane block (SAPB) is an established regional anesthesia technique that provides effective postoperative analgesia. The serratus posterior superior intercostal plane block (SPSIPB) is a recently described interfascial block with promising analgesic effects, but evidence comparing its efficacy with SAPB in breast surgery remains limited.[object Object]This prospective, randomized, dou...	`Is this trial open to male patients?`
TITLE: Masticatory Efficiency, Bite Force, and Patient-Reported Outcomes in Patients Rehabilitated With Complete and Partial Dentures Supported by Teeth, Mini-Implants, or Conventional Implants Masticatory Efficiency, Bite Force, and Patient-Reported Outcomes in Patients Rehabilitated With Conventional Complete Dentures, Conventional Removable Partial Dentures, Mini-Implant-Retained Overdentures, Conventional Implant-Retained Overdentures, and Implant-Assisted Removable Partial Dentures: A Cross-Sectional and Longitudinal Clinical Study[object Object]SUMMARY: This study aims to evaluate masticatory efficiency, bite force, and patient-reported outcomes in edentulous and partially edentulous patients rehabilitated with different removable prosthodontic treatment modalities. The study includes both cross-sectional and longitudinal components. The cross-sectional component will compare patients wearing conventional complete dentures, conventional removable partial dentures, mini-implant-retained overden...	`Could I qualify for this trial if I have a partial denture?`
TITLE: ctDNA-driven Adaptive Proton Craniospinal Irradiation in Non-Small Cell Lung Cancer With Leptomeningeal Metastasis After Resistance to Third-Generation TKIs in the Consolidation Phase Dynamic Adaptive Radiotherapy for Non-Small Cell Lung Cancer With Leptomeningeal Metastasis After Resistance to Third-Generation TKIs in the Consolidation Phase: A Multicenter Randomized Controlled Trial Comparing Outcomes Between Proton Craniospinal Irradiation and Intrathecal Pemetrexed (DART-LM)[object Object]SUMMARY: Brief Summary (English)[object Object]The goal of this clinical trial is to learn if a risk-adaptive consolidation therapy, guided by cerebrospinal fluid (CSF) circulating tumor DNA (ctDNA) clearance kinetics after induction intrathecal pemetrexed, can improve intracranial progression-free survival (iPFS) compared to standard intrathecal pemetrexed consolidation in patients with leptomeningeal metastasis (LM) from EGFR-mutant non-small cell lung cancer (NSCLC) that has progressed on third-generation E...	`I have lung cancer that has spread to my brain, can I take part in a trial that uses a special radiation treatment?`

Loss: MatryoshkaLoss with these parameters:
json
1{ 2 "loss": "MultipleNegativesRankingLoss", 3 "matryoshka_dims": [ 4 768, 5 512, 6 256, 7 128, 8 64 9 ], 10 "matryoshka_weights": [ 11 1, 12 1, 13 1, 14 1, 15 1 16 ], 17 "n_dims_per_step": -1 18}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
num_train_epochs: 4
learning_rate: 2e-05
lr_scheduler_type: cosine
warmup_steps: 0.1
gradient_accumulation_steps: 8
fp16: True
per_device_eval_batch_size: 32
load_best_model_at_end: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

per_device_train_batch_size: 4
num_train_epochs: 4
max_steps: -1
learning_rate: 2e-05
lr_scheduler_type: cosine
lr_scheduler_kwargs: None
warmup_steps: 0.1
optim: adamw_torch_fused
optim_args: None
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
optim_target_modules: None
gradient_accumulation_steps: 8
average_tokens_across_devices: True
max_grad_norm: 1.0
label_smoothing_factor: 0.0
bf16: False
fp16: True
bf16_full_eval: False
fp16_full_eval: False
tf32: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
use_liger_kernel: False
liger_kernel_config: None
use_cache: False
neftune_noise_alpha: None
torch_empty_cache_steps: None
auto_find_batch_size: False
log_on_each_node: True
logging_nan_inf_filter: True
include_num_input_tokens_seen: no
log_level: passive
log_level_replica: warning
disable_tqdm: False
project: huggingface
trackio_space_id: None
trackio_bucket_id: None
trackio_static_space_id: None
per_device_eval_batch_size: 32
prediction_loss_only: True
eval_on_start: False
eval_do_concat_batches: True
eval_use_gather_object: False
eval_accumulation_steps: None
include_for_metrics: []
batch_eval_metrics: False
save_only_model: False
save_on_each_node: False
enable_jit_checkpoint: False
push_to_hub: False
hub_private_repo: None
hub_model_id: None
hub_strategy: every_save
hub_always_push: False
hub_revision: None
load_best_model_at_end: True
ignore_data_skip: False
restore_callback_states_from_checkpoint: False
full_determinism: False
seed: 42
data_seed: None
use_cpu: False
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_pin_memory: True
dataloader_persistent_workers: False
dataloader_prefetch_factor: None
dataloader_multiprocessing_context: None
dataloader_in_order: True
remove_unused_columns: True
label_names: None
train_sampling_strategy: random
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
ddp_static_graph: None
ddp_backend: None
ddp_timeout: 1800
fsdp: None
fsdp_config: None
deepspeed: None
debug: []
skip_memory_metrics: True
do_predict: False
resume_from_checkpoint: None
local_rank: -1
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}
warmup_ratio: None

Training Logs

Epoch	Step	Training Loss	dim_768_cosine_ndcg@10	dim_512_cosine_ndcg@10	dim_256_cosine_ndcg@10	dim_128_cosine_ndcg@10	dim_64_cosine_ndcg@10
0.0445	10	4.1210	-	-	-	-	-
0.0889	20	3.0847	-	-	-	-	-
0.1334	30	2.5468	-	-	-	-	-
0.1779	40	2.4293	-	-	-	-	-
0.2223	50	2.0351	-	-	-	-	-
0.2668	60	1.9562	-	-	-	-	-
0.3113	70	1.8560	-	-	-	-	-
0.3558	80	1.7936	-	-	-	-	-
0.4002	90	1.4392	-	-	-	-	-
0.4447	100	1.8119	0.5052	0.5024	0.4855	0.4616	0.4034
0.4892	110	1.9161	-	-	-	-	-
0.5336	120	1.5390	-	-	-	-	-
0.5781	130	1.2008	-	-	-	-	-
0.6226	140	1.5791	-	-	-	-	-
0.6670	150	1.7636	-	-	-	-	-
0.7115	160	1.8731	-	-	-	-	-
0.7560	170	1.6094	-	-	-	-	-
0.8004	180	1.4552	-	-	-	-	-
0.8449	190	1.3247	-	-	-	-	-
0.8894	200	1.6684	0.4950	0.4951	0.4729	0.4431	0.3967
0.9339	210	1.1478	-	-	-	-	-
0.9783	220	1.5489	-	-	-	-	-
1.0222	230	1.1574	-	-	-	-	-
1.0667	240	1.1790	-	-	-	-	-
1.1112	250	1.0418	-	-	-	-	-
1.1556	260	0.9808	-	-	-	-	-
1.2001	270	1.3452	-	-	-	-	-
1.2446	280	1.4245	-	-	-	-	-
1.2890	290	1.1960	-	-	-	-	-
1.3335	300	1.2613	0.4888	0.4840	0.4702	0.4500	0.4001
1.3780	310	1.2790	-	-	-	-	-
1.4225	320	1.0427	-	-	-	-	-
1.4669	330	1.4931	-	-	-	-	-
1.5114	340	0.9111	-	-	-	-	-
1.5559	350	1.2460	-	-	-	-	-
1.6003	360	1.2518	-	-	-	-	-
1.6448	370	1.5406	-	-	-	-	-
1.6893	380	1.1824	-	-	-	-	-
1.7337	390	0.8772	-	-	-	-	-
1.7782	400	1.3137	0.4884	0.4824	0.4687	0.4401	0.3959
1.8227	410	1.4667	-	-	-	-	-
1.8671	420	1.2337	-	-	-	-	-
1.9116	430	1.3271	-	-	-	-	-
1.9561	440	1.1768	-	-	-	-	-
2.0	450	1.2846	-	-	-	-	-
2.0445	460	0.9551	-	-	-	-	-
2.0889	470	0.8428	-	-	-	-	-
2.1334	480	0.8362	-	-	-	-	-
2.1779	490	0.6628	-	-	-	-	-
2.2223	500	0.7811	0.4884	0.4798	0.4622	0.4487	0.3936
2.2668	510	1.1356	-	-	-	-	-
2.3113	520	0.7177	-	-	-	-	-
2.3558	530	0.9575	-	-	-	-	-
2.4002	540	0.7495	-	-	-	-	-
2.4447	550	0.9872	-	-	-	-	-
2.4892	560	0.9362	-	-	-	-	-
2.5336	570	0.7413	-	-	-	-	-
2.5781	580	0.9984	-	-	-	-	-
2.6226	590	0.7781	-	-	-	-	-
2.6670	600	0.9545	0.4887	0.4860	0.4685	0.4507	0.4027
2.7115	610	0.7938	-	-	-	-	-
2.7560	620	0.7651	-	-	-	-	-
2.8004	630	1.1165	-	-	-	-	-
2.8449	640	0.9484	-	-	-	-	-
2.8894	650	0.9166	-	-	-	-	-
2.9339	660	0.8746	-	-	-	-	-
2.9783	670	1.1498	-	-	-	-	-
3.0222	680	0.7173	-	-	-	-	-
3.0667	690	1.0119	-	-	-	-	-
3.1112	700	0.7311	0.4970	0.4910	0.4771	0.4534	0.4073
3.1556	710	0.7264	-	-	-	-	-
3.2001	720	0.6853	-	-	-	-	-
3.2446	730	0.6763	-	-	-	-	-
3.2890	740	0.8079	-	-	-	-	-
3.3335	750	0.6731	-	-	-	-	-
3.3780	760	0.9650	-	-	-	-	-
3.4225	770	0.7704	-	-	-	-	-
3.4669	780	1.0366	-	-	-	-	-
3.5114	790	0.9004	-	-	-	-	-
3.5559	800	0.5977	0.4963	0.4885	0.4771	0.4544	0.4090
3.6003	810	0.6151	-	-	-	-	-
3.6448	820	0.8600	-	-	-	-	-
3.6893	830	0.5171	-	-	-	-	-
3.7337	840	0.8487	-	-	-	-	-
3.7782	850	0.5347	-	-	-	-	-
3.8227	860	0.7020	-	-	-	-	-
3.8671	870	0.4907	-	-	-	-	-
3.9116	880	0.5780	-	-	-	-	-
3.9561	890	0.7386	-	-	-	-	-
4.0	900	0.8223	0.4972	0.4891	0.4777	0.4550	0.4100

The bold row denotes the saved checkpoint.

Training Time

Training: 1.6 hours

Framework Versions

Python: 3.12.13
Sentence Transformers: 5.7.0
Transformers: 5.15.0
PyTorch: 2.11.0+cu128
Accelerate: 1.14.0
Datasets: 4.0.0
Tokenizers: 0.22.2

Additional Resources

Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

bibtex

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

bibtex

@misc{oord2019representationlearningcontrastivepredictive,
      title={Representation Learning with Contrastive Predictive Coding},
      author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
      year={2019},
      eprint={1807.03748},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/1807.03748},
}

	positive	anchor
type	string	string
modality	text	text
details	min: 271 tokens mean: 671.29 tokens max: 2110 tokens	min: 10 tokens mean: 20.11 tokens max: 35 tokens