This is a sentence-transformers model finetuned from sentence-transformers/multi-qa-mpnet-base-dot-v1. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/multi-qa-mpnet-base-dot-v1
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 tokens
Similarity Function: Dot Product

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'ear malformations, nipple abnormalities, dental anomalies',
    'A number sign (#) is used with this entry because scalp-ear-nipple syndrome (SENS) is caused by heterozygous mutation in the KCTD1 gene (613420) on chromosome 18q11.\n\nDescription\n\nScalp-ear-nipple syndrome is characterized by aplasia cutis congenita of the scalp, breast anomalies that range from hypothelia or athelia to amastia, and minor anomalies of the external ears. Less frequent clinical characteristics include nail dystrophy, dental anomalies, cutaneous syndactyly of the digits, and renal malformations. Penetrance appears to be high, although there is substantial variable expressivity within families (Marneros et al., 2013).\n\nClinical Features',
    'This article is an orphan, as no other articles link to it. Please introduce links to this page from related articles; try the Find link tool for suggestions. (July 2016)  \n  \nInguinal lymphadenopathy  \nInguinal lymphadenopathy  \n  \nInguinal lymphadenopathy causes swollen lymph nodes in the groin area. It can be a symptom of infective or neoplastic processes. Infective aetiologies include Tuberculosis, HIV, non-specific or reactive lymphadenopathy to recent lower limb infection or groin infections. Another notable infectious cause is Lymphogranuloma venereum, which is a sexually transmitted infection of the lymphatic system. Neoplastic aetiologies include lymphoma, leukaemia and metastatic disease from primary tumours in the lower limb, external genitalia or perianal region and melanoma.\n\n## References[edit]\n\n  * Ferrer R (October 1998). "Lymphadenopathy: differential diagnosis and evaluation". Am Fam Physician. 58 (6): 1313–20. PMID 9803196.\n\n## Further reading[edit]',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.1807
cosine_accuracy@3	0.5427
cosine_accuracy@5	0.7381
cosine_accuracy@10	0.8161
cosine_precision@1	0.1807
cosine_precision@3	0.1809
cosine_precision@5	0.1476
cosine_precision@10	0.0816
cosine_recall@1	0.1807
cosine_recall@3	0.5427
cosine_recall@5	0.7381
cosine_recall@10	0.8161
cosine_ndcg@10	0.4947
cosine_mrr@10	0.3907
cosine_map@100	0.3953
dot_accuracy@1	0.1827
dot_accuracy@3	0.5413
dot_accuracy@5	0.743
dot_accuracy@10	0.8167
dot_precision@1	0.1827
dot_precision@3	0.1804
dot_precision@5	0.1486
dot_precision@10	0.0817
dot_recall@1	0.1827
dot_recall@3	0.5413
dot_recall@5	0.743
dot_recall@10	0.8167
dot_ndcg@10	0.4957
dot_mrr@10	0.3918
dot_map@100	0.3963

Training Details

Training Dataset

Unnamed Dataset

Size: 98,928 training samples
Columns: queries and chunks
Approximate statistics based on the first 1000 samples:
queries chunks
type string string
details
min: 7 tokens
mean: 17.4 tokens
max: 76 tokens
min: 5 tokens
mean: 159.93 tokens
max: 334 tokens

Samples:

queries	chunks
`fever, malaise, headaches, lymphadenopathy`	A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days.
`rash, papulovesicular, generalized, constitutional symptoms`	A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days.
`myalgia, diaphoresis, nausea, vomiting`	A rare, acquired, self-limiting, infectious disease due to the mite-borne bacteria Rickettsia akari characterized by an asymptomatic, 0.5 to 2 cm in diameter papulovesicle that typically ulcerates and forms an eschar, followed by a generalized papulovesicular rash associating variable constitutional symptoms, such as localized lymphadenopathy, fever, malaise, and headaches. Additonal symptoms may include diaphoresis, myalgia and, less frequently, rhinorrhea, pharyngitis, nausea, vomiting, splenomegaly, conjunctival hyperemia, and abdominal pain. Systemic symtoms resolve within 6-10 days.

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 1, 3 "similarity_fct": "dot_score" 4}

Evaluation Dataset

Unnamed Dataset

Size: 9,308 evaluation samples
Columns: queries and chunks
Approximate statistics based on the first 1000 samples:
queries chunks
type string string
details
min: 7 tokens
mean: 17.8 tokens
max: 48 tokens
min: 4 tokens
mean: 166.19 tokens
max: 299 tokens

Samples:

queries	chunks
`facial features, overgrowth, learning disabilities, delayed development`	Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited).
`long face, high forehead, flushed cheeks, small chin, down-slanting palpebral fissures`	Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited).
`intellectual disability, behavioral problems, speech and language difficulties, hypotonia`	Sotos syndrome is a condition characterized mainly by distinctive facial features; overgrowth in childhood; and learning disabilities or delayed development. Facial features may include a long, narrow face; a high forehead; flushed (reddened) cheeks; a small, pointed chin; and down-slanting palpebral fissures. Affected infants and children tend to grow quickly; they are significantly taller than their siblings and peers and have a large head. Other signs and symptoms may include intellectual disability; behavioral problems; problems with speech and language; and/or weak muscle tone (hypotonia). Sotos syndrome is usually caused by a mutation in the NSD1 gene and is inherited in an autosomal dominant manner. About 95% of cases are due to a new mutation in the affected person and occur sporadically (are not inherited).

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 1, 3 "similarity_fct": "dot_score" 4}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
learning_rate: 2e-05
num_train_epochs: 25
warmup_ratio: 0.1
fp16: True
load_best_model_at_end: True
eval_on_start: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 25
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: True
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: True
eval_use_gather_object: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss	loss	dot_map@100
0	0	-	1.8701	0.2095
0.1295	100	1.5494	-	-
0.2591	200	0.9993	-	-
0.3886	300	0.7225	-	-
0.5181	400	0.6533	-	-
0.6477	500	0.6618	0.5939	0.3722
0.7772	600	0.6454	-	-
0.9067	700	0.5568	-	-
1.0363	800	0.5435	-	-
1.1658	900	0.499	-	-
1.2953	1000	0.5386	0.4768	0.3842
1.4249	1100	0.5077	-	-
1.5544	1200	0.4929	-	-
1.6839	1300	0.5194	-	-
1.8135	1400	0.5157	-	-
1.9430	1500	0.4337	0.4455	0.3894
2.0725	1600	0.4373	-	-
2.2021	1700	0.4569	-	-
2.3316	1800	0.4084	-	-
2.4611	1900	0.42	-	-
2.5907	2000	0.4112	0.4578	0.3886
2.7202	2100	0.4498	-	-
2.8497	2200	0.415	-	-
2.9793	2300	0.3734	-	-
3.1088	2400	0.3359	-	-
3.2383	2500	0.3923	0.4339	0.3929
3.3679	2600	0.3345	-	-
3.4974	2700	0.3324	-	-
3.6269	2800	0.3574	-	-
3.7565	2900	0.4078	-	-
3.8860	3000	0.3221	0.4293	0.3904
4.0155	3100	0.2895	-	-
4.1451	3200	0.2821	-	-
4.2746	3300	0.3192	-	-
4.4041	3400	0.28	-	-
4.5337	3500	0.2716	0.4486	0.3885
4.6632	3600	0.3147	-	-
4.7927	3700	0.3565	-	-
4.9223	3800	0.2465	-	-
5.0518	3900	0.2436	-	-
5.1813	4000	0.2297	0.4486	0.3917
5.3109	4100	0.2538	-	-
5.4404	4200	0.2448	-	-
5.5699	4300	0.2433	-	-
5.6995	4400	0.3017	-	-
5.8290	4500	0.2958	0.4737	0.3934
5.9585	4600	0.2142	-	-
6.0881	4700	0.1939	-	-
6.2176	4800	0.2449	-	-
6.3472	4900	0.2026	-	-
6.4767	5000	0.2006	0.4901	0.3895
6.6062	5100	0.2118	-	-
6.7358	5200	0.3064	-	-
6.8653	5300	0.2276	-	-
6.9948	5400	0.1809	-	-
7.1244	5500	0.1782	0.4992	0.3915
7.2539	5600	0.2211	-	-
7.3834	5700	0.1728	-	-
7.5130	5800	0.1651	-	-
7.6425	5900	0.2158	-	-
7.7720	6000	0.2864	0.5113	0.3892
7.9016	6100	0.179	-	-
8.0311	6200	0.1677	-	-
8.1606	6300	0.1517	-	-
8.2902	6400	0.1851	-	-
8.4197	6500	0.1646	0.5030	0.3933
8.5492	6600	0.1608	-	-
8.6788	6700	0.217	-	-
8.8083	6800	0.2357	-	-
8.9378	6900	0.1404	-	-
9.0674	7000	0.1465	0.5153	0.3877
9.1969	7100	0.1791	-	-
9.3264	7200	0.1261	-	-
9.4560	7300	0.1406	-	-
9.5855	7400	0.1626	-	-
9.7150	7500	0.223	0.5326	0.3939
9.8446	7600	0.1806	-	-
9.9741	7700	0.1289	-	-
10.1036	7800	0.1269	-	-
10.2332	7900	0.1609	-	-
10.3627	8000	0.1279	0.5113	0.3933
10.4922	8100	0.1264	-	-
10.6218	8200	0.1453	-	-
10.7513	8300	0.2227	-	-
10.8808	8400	0.1314	-	-
11.0104	8500	0.1192	0.5444	0.3925
11.1399	8600	0.1164	-	-
11.2694	8700	0.1418	-	-
11.3990	8800	0.1202	-	-
11.5285	8900	0.1152	-	-
11.658	9000	0.1454	0.529	0.3963
11.7876	9100	0.1952	-	-
11.9171	9200	0.1079	-	-
12.0466	9300	0.1139	-	-
12.1762	9400	0.1067	-	-
12.3057	9500	0.1219	0.5257	0.3938
12.4352	9600	0.119	-	-
12.5648	9700	0.1195	-	-
12.6943	9800	0.158	-	-
12.8238	9900	0.156	-	-
12.9534	10000	0.0974	0.5434	0.3934
13.0829	10100	0.0928	-	-
13.2124	10200	0.1266	-	-
13.3420	10300	0.0964	-	-
13.4715	10400	0.1007	-	-
13.6010	10500	0.112	0.5789	0.3893
13.7306	10600	0.1699	-	-
13.8601	10700	0.1084	-	-
13.9896	10800	0.0967	-	-
14.1192	10900	0.0856	-	-
14.2487	11000	0.1142	0.5252	0.3933
14.3782	11100	0.0891	-	-
14.5078	11200	0.0911	-	-
14.6373	11300	0.1128	-	-
14.7668	11400	0.1686	-	-
14.8964	11500	0.0874	0.5874	0.3945
15.0259	11600	0.0909	-	-
15.1554	11700	0.0778	-	-
15.2850	11800	0.1055	-	-
15.4145	11900	0.0872	-	-
15.5440	12000	0.0884	0.5894	0.3934
15.6736	12100	0.1101	-	-
15.8031	12200	0.1354	-	-
15.9326	12300	0.0762	-	-
16.0622	12400	0.0782	-	-
16.1917	12500	0.0936	0.5589	0.3919
16.3212	12600	0.072	-	-
16.4508	12700	0.0806	-	-
16.5803	12800	0.0929	-	-
16.7098	12900	0.1215	-	-
16.8394	13000	0.1039	0.6025	0.3926
16.9689	13100	0.0738	-	-
17.0984	13200	0.0651	-	-
17.2280	13300	0.0943	-	-
17.3575	13400	0.0678	-	-
17.4870	13500	0.077	0.6002	0.3941
17.6166	13600	0.0839	-	-
17.7461	13700	0.1268	-	-
17.8756	13800	0.0764	-	-
18.0052	13900	0.0686	-	-
18.1347	14000	0.0697	0.5898	0.3913
18.2642	14100	0.0871	-	-
18.3938	14200	0.0699	-	-
18.5233	14300	0.0611	-	-
18.6528	14400	0.0872	-	-
18.7824	14500	0.1281	0.6087	0.3927
18.9119	14600	0.0583	-	-
19.0415	14700	0.0658	-	-
19.1710	14800	0.0595	-	-
19.3005	14900	0.0816	-	-
19.4301	15000	0.0699	0.6078	0.3965
19.5596	15100	0.0729	-	-
19.6891	15200	0.0908	-	-
19.8187	15300	0.0978	-	-
19.9482	15400	0.0585	-	-
20.0777	15500	0.0557	0.5861	0.3925
20.2073	15600	0.0787	-	-
20.3368	15700	0.061	-	-
20.4663	15800	0.0638	-	-
20.5959	15900	0.0656	-	-
20.7254	16000	0.1003	0.6032	0.3923
20.8549	16100	0.0718	-	-
20.9845	16200	0.0625	-	-
21.1140	16300	0.0532	-	-
21.2435	16400	0.0739	-	-
21.3731	16500	0.0552	0.6080	0.3942
21.5026	16600	0.0588	-	-
21.6321	16700	0.0716	-	-
21.7617	16800	0.1078	-	-
21.8912	16900	0.0559	-	-
22.0207	17000	0.0596	0.6044	0.3922
22.1503	17100	0.0512	-	-
22.2798	17200	0.0716	-	-
22.4093	17300	0.0574	-	-
22.5389	17400	0.058	-	-
22.6684	17500	0.07	0.6117	0.3942
22.7979	17600	0.0965	-	-
22.9275	17700	0.0507	-	-
23.0570	17800	0.0498	-	-
23.1865	17900	0.0524	-	-
23.3161	18000	0.0656	0.5936	0.3936
23.4456	18100	0.057	-	-
23.5751	18200	0.0619	-	-
23.7047	18300	0.0785	-	-
23.8342	18400	0.0729	-	-
23.9637	18500	0.0541	0.6174	0.3979
24.0933	18600	0.0456	-	-
24.2228	18700	0.0696	-	-
24.3523	18800	0.048	-	-
24.4819	18900	0.0547	-	-
24.6114	19000	0.0553	0.6146	0.3962
24.7409	19100	0.0936	-	-
24.8705	19200	0.0579	-	-
25.0	19300	0.0498	0.5290	0.3963

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.11.9
Sentence Transformers: 3.0.1
Transformers: 4.43.3
PyTorch: 2.3.1+cu121
Accelerate: 0.30.1
Datasets: 2.19.2
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

bibtex

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply}, 
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

	queries	chunks
type	string	string
details	min: 7 tokens mean: 17.4 tokens max: 76 tokens	min: 5 tokens mean: 159.93 tokens max: 334 tokens

antonkirk/retrieval-mpnet-dot-finetuned-llama3-synthetic-dataset

SentenceTransformer based on sentence-transformers/multi-qa-mpnet-base-dot-v1

Model Details

Model Description

Model Sources

Full Model Architecture

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Information Retrieval

Training Details

Training Dataset

Unnamed Dataset

Evaluation Dataset

Unnamed Dataset

Training Hyperparameters

Non-Default Hyperparameters

All Hyperparameters

Training Logs

Framework Versions

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss

antonkirk/
retrieval-mpnet-dot-finetuned-llama3-synthetic-dataset