This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/all-mpnet-base-v2
Maximum Sequence Length: 384 tokens
Output Dimensionality: 768 tokens
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False}) with Transformer model: MPNetModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'moor brown gb 2014 audi a5s5 cabriolet south africa market body middle armrest front pr6e3gb if period 0911 gb automotive',
    'moor brown gb 2013 audi a4 allroad quattro canada market body middle armrest front pr6e3gb if period 1111 gb automotive',
    'smart racing products smartstrings 4wheel alignment set 1996 bmw 318i base convertible suspension tools page 1 automotive',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

Size: 269,761 training samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 15 tokens
mean: 69.38 tokens
max: 384 tokens
min: 11 tokens
mean: 70.41 tokens
max: 384 tokens

Samples:

anchor	positive
2008 dodge ram 2500 laramie used white for sale in euless tx dodge automotive mileage 17383 exterior color arctic white transmission 7speed automatic wod engine intercooled turbo diesel i4 21 l131 stock 4gp307369 wdzpe7dd4gp307369 mercedesbenz used sprinter passenger vans mileage 148438 exterior color white transmission 4speed hd automatic wod 4l65e engine gas v8 60l364 stock x4h100607 5grgn23ux4h100607 hummer used h2 mileage 34383 exterior color white satin pearl transmission 8speed automatic wmanual shift engine gas v8 50l307 stock 4du072090 kmhgh4jh4du072090 hyundai used equus mileage 107520 exterior color bright white clearcoat transmission 6speed automatic engine gas i4 24l144 stock xcn207909 1c3ccbbbxcn207909 chrysler used 200 mileage 59582 exterior color bright white clearcoat transmission 6speed automatic engine gas i4 24l144 stock 3dn554452 1c3ccbbb3dn554452 chrysler used 200 mileage 76681 exterior color taffeta white transmission 5speed automatic engine gas i4 24l144 stock 5ag705280 3czre3h35ag705280 honda used crv mileage 99896 exterior color premium white pearl transmission 5speed automatic wod engine gas i4 24l144 stock 6xac03652 jh4cu2f6xac036527 acura used tsx mileage 125522 exterior color crystal white transmission 1speed ecvt engine gaselectric v6 33l202 stock x82851820 jtjgw31ux82851820 lexus used rx 400h mileage 36797 exterior color clear white transmission 6speed automatic wod engine regular unleaded i4 24 l144 stock 0h7070128 kndpm3ac0h7070128 kia used sportage mileage 77058 exterior color bright white clearcoat transmission 5speed automatic engine gas v6 36l220 stock 0dh649637 2c3cdyag0dh649637 dodge used challenger mileage 39319 exterior color arctic white transmission 7speed automatic engine gas v6 30l183 stock 8br178298 wddgf8bb8br178298 mercedesbenz used cclass mileage 64038 exterior color white gold clearcoat transmission 5speed automatic wod engine gas v8 57l345 stock 7bs615073 1d7rb1ct7bs615073 ram used 1500 mileage 69807 exterior color bright white clearcoat transmission 4speed automatic engine gas v8 57l345 stock 9cs228465 1c6rd6ft9cs228465 ram used 1500 mileage 14918 exterior color pearl white transmission 1speed cvt wod engine regular unleaded i4 25 l152 stock xgp686433 knmat2mtxgp686433 nissan used rogue mileage 39730 exterior color fuji white transmission 6speed automatic engine gas i6 32l195 stock xch305122 salfr2bnxch305122 land rover used lr2 mileage 34618 exterior color snow white pearl transmission 6speed automatic wod engine regular unleaded v6 33 l204 stock 6fg577726 5xykw4a76fg577726 kia used sorento mileage 169861 exterior color oxford white transmission 6speed automatic engine turbocharged diesel v8 67l406 stock 1bea21901 1ft7w2bt1bea21901 ford used super duty f250 srw mileage 21309 exterior color pure red transmission 8speed cvt wod engine intercooled turbo premium unleaded h4 20 l122 stock 3g8807653 jf1va1e63g8807653 subaru used wrx mileage 61401 exterior color white gold clearcoat transmission automatic engine gas v8 57l345 stock 5bg609448 3d7tt2ct5bg609448 ram used 2500 mileage 28253 exterior color bright white clearcoat transmission 8speed automatic wod engine regular unleaded v6 36 l220 stock 2ec478489 1c4rjfbg2ec478489 jeep used grand cherokee mileage 33929 exterior color bright white clearcoat transmission 6speed automatic engine gas v8 57l345 stock xdc653013 1c4sdjctxdc653013 dodge used durango mileage 178293 exterior color bright white clearcoat transmission automatic engine diesel i6 67l409 stock 3d3ks29a dodge used ram 2500 mileage 107825 exterior color diamond white metallic transmission 7speed automatic wmanual shift engine turbocharged gas v8 46l285 stock 9da114638 4jgdf7ce9da114638 mercedesbenz used glclass mileage 17831 exterior color cirrus white transmission 7speed autoshift	2008 dodge ram 2500 laramie used white for sale in euless tx dodge automotive mileage 17383 exterior color arctic white transmission 7speed automatic wod engine intercooled turbo diesel i4 21 l131 stock 4gp307369 wdzpe7dd4gp307369 mercedesbenz used sprinter passenger vans mileage 148438 exterior color white transmission 4speed hd automatic wod 4l65e engine gas v8 60l364 stock x4h100607 5grgn23ux4h100607 hummer used h2 mileage 34383 exterior color white satin pearl transmission 8speed automatic wmanual shift engine gas v8 50l307 stock 4du072090 kmhgh4jh4du072090 hyundai used equus mileage 107520 exterior color bright white clearcoat transmission 6speed automatic engine gas i4 24l144 stock xcn207909 1c3ccbbbxcn207909 chrysler used 200 mileage 59582 exterior color bright white clearcoat transmission 6speed automatic engine gas i4 24l144 stock 3dn554452 1c3ccbbb3dn554452 chrysler used 200 mileage 76681 exterior color taffeta white transmission 5speed automatic engine gas i4 24l144 stock 5ag705280 3czre3h35ag705280 honda used crv mileage 99896 exterior color premium white pearl transmission 5speed automatic wod engine gas i4 24l144 stock 6xac03652 jh4cu2f6xac036527 acura used tsx mileage 125522 exterior color crystal white transmission 1speed ecvt engine gaselectric v6 33l202 stock x82851820 jtjgw31ux82851820 lexus used rx 400h mileage 36797 exterior color clear white transmission 6speed automatic wod engine regular unleaded i4 24 l144 stock 0h7070128 kndpm3ac0h7070128 kia used sportage mileage 77058 exterior color bright white clearcoat transmission 5speed automatic engine gas v6 36l220 stock 0dh649637 2c3cdyag0dh649637 dodge used challenger mileage 39319 exterior color arctic white transmission 7speed automatic engine gas v6 30l183 stock 8br178298 wddgf8bb8br178298 mercedesbenz used cclass mileage 64038 exterior color white gold clearcoat transmission 5speed automatic wod engine gas v8 57l345 stock 7bs615073 1d7rb1ct7bs615073 ram used 1500 mileage 69807 exterior color bright white clearcoat transmission 4speed automatic engine gas v8 57l345 stock 9cs228465 1c6rd6ft9cs228465 ram used 1500 mileage 14918 exterior color pearl white transmission 1speed cvt wod engine regular unleaded i4 25 l152 stock xgp686433 knmat2mtxgp686433 nissan used rogue mileage 39730 exterior color fuji white transmission 6speed automatic engine gas i6 32l195 stock xch305122 salfr2bnxch305122 land rover used lr2 mileage 34618 exterior color snow white pearl transmission 6speed automatic wod engine regular unleaded v6 33 l204 stock 6fg577726 5xykw4a76fg577726 kia used sorento mileage 169861 exterior color oxford white transmission 6speed automatic engine turbocharged diesel v8 67l406 stock 1bea21901 1ft7w2bt1bea21901 ford used super duty f250 srw mileage 21309 exterior color pure red transmission 8speed cvt wod engine intercooled turbo premium unleaded h4 20 l122 stock 3g8807653 jf1va1e63g8807653 subaru used wrx mileage 61401 exterior color white gold clearcoat transmission automatic engine gas v8 57l345 stock 5bg609448 3d7tt2ct5bg609448 ram used 2500 mileage 28253 exterior color bright white clearcoat transmission 8speed automatic wod engine regular unleaded v6 36 l220 stock 2ec478489 1c4rjfbg2ec478489 jeep used grand cherokee mileage 33929 exterior color bright white clearcoat transmission 6speed automatic engine gas v8 57l345 stock xdc653013 1c4sdjctxdc653013 dodge used durango mileage 178293 exterior color bright white clearcoat transmission automatic engine diesel i6 67l409 stock 3d3ks29a dodge used ram 2500 mileage 107825 exterior color diamond white metallic transmission 7speed automatic wmanual shift engine turbocharged gas v8 46l285 stock 9da114638 4jgdf7ce9da114638 mercedesbenz used glclass mileage 17831 exterior color cirrus white transmission 7speed autoshift
`pacon fade resistant neon poster board us pacon officesupplycomus officeproducts`	`pacon fade resistant neon poster board us pacon officesupplycomus officeproducts`
`2015 bmw 535i xdrive sedan texas 76132 2015 535 lease special promotion on a for 87681 per month fort worth automotive`	2015 bmw 535i xdrive sedan florida 34787 2015 535 xdriv lease special promotion on sedan a for 76289 per month winter garden automotive listing number car bmw 535i xdrive sedan condition outstanding transmission automatic vin wba5b3c59fd540941 down payment 000 down engine 6 cylinder fuel type gasoline doors four door drivetrain four wheel drive rear brake width 09 emergency braking preparation emergency braking preparation brake drying brake drying front brake width 14 rear brake type ventilated disc electronic brakeforce distribution electronic brakeforce distribution front brake diameter 137 front brake type ventilated disc antilock braking system 4wheel abs braking assist braking assist rear brake diameter 130 overall width without mirrors 732 wheelbase 1169 overall length 1931 minimum ground clearance 56 overall height 576 rear track 641 front track 630 ege highway mpg 29 epa combined mpg 23 manufacturer 0 100km acceleration time seconds 58 aerodynamic drag cd 030 epa city mpg 19 turning diameter 392 curb weight 4233 tco curb weight 4233 epa highway mpg 29 fuel capacity 185 ege city mpg 19 height adjustable driver seat height adjustable driver seat adjustable lumbar power adjustable lumbar support driver seat whiplash protection whiplash protection system driver seat easy entry power steering wheel number of driver seat power adjustments 10 number of memorized driver seat settings 2 adjustable driver seat headrest power adjustable headrests 2nd row leg room 361 2nd row shoulder room 562 2nd row head room 383 1st row leg room 414 1st row head room 405 1st row shoulder room 583 epa interior volume 116 in dash cd single cd player audio system memory feature includes audio system digital audio input auxiliary audio input and ipodiphone integration total number of speakers 10 mp3 player cd mp3 playback antenna type diversity audio security system audio security system usb connection usb connection radio data system radio data system radio amfm hd radio watts 180 subwoofer 2 rear tire width 245 run flat tires run flat tire type all season rear tire diameter 18 rear tire profile 45 rear tire speed rating v rear tire width 245 run flat tires run flat tire type all season rear tire diameter 19 rear tire profile 40 in dash cd single cd player digital audio input auxiliary audio input and ipodiphone integration total number of speakers 16 mp3 player cd mp3 playback speed sensitive volume control speed sensitive volume control radio data system radio data system dvd audio dvdaudio radio amfm hd radio satellite radio sirius watts 600 surround audio 51 months of provided satellite radio service 12 rear tire width 245 run flat tires run flat tire type all season rear tire diameter 19 rear tire profile 40 warranty maximum mileage 50000 warranty miles limitedunlimited l warranty years limitedunlimited l warranty maximum years 4 warranty type basic warranty maximum mileage 50000 warranty miles limitedunlimited l warranty years limitedunlimited l warranty maximum years 4 warranty type drivetrain warranty maximum mileage 50000 warranty comments free full maintenance for 4 years or 50000 miles warranty miles limitedunlimited l warranty years limitedunlimited l warranty maximum years 4 warranty

Loss: CachedMultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Evaluation Dataset

Unnamed Dataset

Size: 67,441 evaluation samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 14 tokens
mean: 68.57 tokens
max: 384 tokens
min: 15 tokens
mean: 69.25 tokens
max: 384 tokens

Samples:

anchor	positive
`barracuda essentials for office 365 complete protection and compliance li barracuda li email security cdwgcom five year license renewal for barracuda essentials for office 365 complete protection for up to 249 users otherelectronics`	`barracuda essentials for office 365 complete protection and compliance li barracuda li email security cdwcom five year license renewal for barracuda essentials for office 365 complete protection for up to 249 users otherelectronics`
`cool carbon performance brake pad set rear 2003 bmw 325i base sedan upgrades page 2 automotive`	`cool carbon performance brake pad set rear 2002 bmw 325i base wagon upgrades page 2 automotive`
decoart americana acrylic paint williamsburg blue 2 ounces blue 2 oz cover art projects with affordable quality our 2ounce bottle of americana acrylic paint in williamsburg blue boasts a bright pigment that can be used on almost any surface once dry this paint will create a durable matte finish that can be varnished to create a variety of sheens blue acrylic paint permanent waterbased nontoxic lightfast pigment one coat coverage with most colors intermixable blends easily consistent viscosity made in usa toolsandhomeimprovement	`decoart americana acrylic paint williamsburg blue 2 oz wholesale darice decoart americana acrylic paint williamsburg blue 2 oz decoart toolsandhomeimprovement`

Loss: CachedMultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
learning_rate: 1e-05
num_train_epochs: 2
warmup_ratio: 0.1
fp16: True
auto_find_batch_size: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 1e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 2
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: False
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: True
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
eval_use_gather_object: False
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Epoch	Step	Training Loss	loss
0.2076	7000	0.015	0.0053
0.4152	14000	0.0059	0.0042
0.6228	21000	0.0041	0.0034
0.8303	28000	0.0038	0.0030
1.0379	35000	0.0028	0.0026
1.2455	42000	0.0023	0.0023
1.4531	49000	0.0011	0.0023
1.6607	56000	0.0008	0.0020
1.8683	63000	0.0007	0.0020

Framework Versions

Python: 3.10.13
Sentence Transformers: 3.0.1
Transformers: 4.44.0
PyTorch: 2.2.1
Accelerate: 0.33.0
Datasets: 2.21.0
Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CachedMultipleNegativesRankingLoss

bibtex

@misc{gao2021scaling,
    title={Scaling Deep Contrastive Learning Batch Size under Memory Limited Setup}, 
    author={Luyu Gao and Yunyi Zhang and Jiawei Han and Jamie Callan},
    year={2021},
    eprint={2101.06983},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

	anchor	positive
type	string	string
details	min: 15 tokens mean: 69.38 tokens max: 384 tokens	min: 11 tokens mean: 70.41 tokens max: 384 tokens

spl4shedEdu/mpnet_ESB

SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

Model Details

Model Description

Model Sources

Full Model Architecture

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

Evaluation Dataset

Unnamed Dataset

Training Hyperparameters

Non-Default Hyperparameters

All Hyperparameters

Training Logs

Framework Versions

Citation

BibTeX

Sentence Transformers

CachedMultipleNegativesRankingLoss

spl4shedEdu/
mpnet_ESB