You can deploy this model and start earning money today!

Views

No views yet

bge-m3

Name: bge-m3-new2
Availability: InStock

This is a sentence-transformers model finetuned from BAAI/bge-m3. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-m3
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity

Language: en
License: apache-2.0

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Who should communicate a personal data breach to the data subject?',
    'The controller should communicate to the data subject a personal data breach, without undue delay, where that personal data breach is likely to result in a high risk to the rights and freedoms of the natural person in order to allow him or her to take the necessary precautions. The communication should describe the nature of the personal data breach as well as recommendations for the natural person concerned to mitigate potential adverse effects. Such communications to data subjects should be made as soon as reasonably feasible and in close cooperation with the supervisory authority, respecting guidance provided by it or by other relevant authorities such as law-enforcement authorities. For example, the need to mitigate an immediate risk of damage would call for prompt communication with data subjects whereas the need to implement appropriate measures against continuing or similar personal data breaches may justify more time for communication.',
    "1.In the absence of an adequacy decision pursuant to Article 45(3), or of appropriate safeguards pursuant to Article 46, including binding corporate rules, a transfer or a set of transfers of personal data to a third country or an international organisation shall take place only on one of the following conditions: (a)  the data subject has explicitly consented to the proposed transfer, after having been informed of the possible risks of such transfers for the data subject due to the absence of an adequacy decision and appropriate safeguards; (b)  the transfer is necessary for the performance of a contract between the data subject and the controller or the implementation of pre-contractual measures taken at the data subject's request; (c)  the transfer is necessary for the conclusion or performance of a contract concluded in the interest of the data subject between the controller and another natural or legal person; (d)  the transfer is necessary for important reasons of public interest; (e)  the transfer is necessary for the establishment, exercise or defence of legal claims; (f)  the transfer is necessary in order to protect the vital interests of the data subject or of other persons, where the data subject is physically or legally incapable of giving consent; (g)  the transfer is made from a register which according to Union or Member State law is intended to provide information to the public and which is open to consultation either by the public in general or by any person who can demonstrate a legitimate interest, but only to the extent that the conditions laid down by Union or Member State law for consultation are fulfilled in the particular case. Where a transfer could not be based on a provision in Article 45 or 46, including the provisions on binding corporate rules, and none of the derogations for a specific situation referred to in the first subparagraph of this paragraph is applicable, a transfer to a third country or an international organisation may take place only if the transfer is not repetitive, concerns only a limited number of data subjects, is necessary for the purposes of compelling legitimate interests pursued by the controller which are not overridden by the interests or rights and freedoms of the data subject, and the controller has assessed all the circumstances surrounding the data transfer and has on the basis of that assessment provided suitable safeguards with regard to the protection of personal data. The controller shall inform the supervisory authority of the transfer. The controller shall, in addition to providing the information referred to in Articles 13 and 14, inform the data subject of the transfer and on the compelling legitimate interests pursued.\n2.A transfer pursuant to point (g) of the first subparagraph of paragraph 1 shall not involve the entirety of the personal data or entire categories of the personal data contained in the register. Where the register is intended for consultation by persons having a legitimate interest, the transfer shall be made only at the request of those persons or if they are to be the recipients. 4.5.2016 L 119/64  \n3.Points (a), (b) and (c) of the first subparagraph of paragraph 1 and the second subparagraph thereof shall not apply to activities carried out by public authorities in the exercise of their public powers.\n4.The public interest referred to in point (d) of the first subparagraph of paragraph 1 shall be recognised in Union law or in the law of the Member State to which the controller is subject.\n5.In the absence of an adequacy decision, Union or Member State law may, for important reasons of public interest, expressly set limits to the transfer of specific categories of personal data to a third country or an international organisation. Member States shall notify such provisions to the Commission.\n6.The controller or processor shall document the assessment as well as the suitable safeguards referred to in the second subparagraph of paragraph 1 of this Article in the records referred to in Article 30.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.7041, 0.2945],
#         [0.7041, 1.0000, 0.3575],
#         [0.2945, 0.3575, 1.0000]])

Evaluation

Metrics

Information Retrieval

Dataset: dim_768
Evaluated with InformationRetrievalEvaluator with these parameters:
json
1{ 2 "truncate_dim": 768 3}

Metric	Value
cosine_accuracy@1	0.5051
cosine_accuracy@3	0.5379
cosine_accuracy@5	0.5783
cosine_accuracy@10	0.6237
cosine_precision@1	0.5051
cosine_precision@3	0.4832
cosine_precision@5	0.4495
cosine_precision@10	0.3889
cosine_recall@1	0.1083
cosine_recall@3	0.2625
cosine_recall@5	0.3474
cosine_recall@10	0.4711
cosine_ndcg@10	0.5624
cosine_mrr@10	0.5318
cosine_map@100	0.6073

Information Retrieval

Dataset: dim_512
Evaluated with InformationRetrievalEvaluator with these parameters:
json
1{ 2 "truncate_dim": 512 3}

Metric	Value
cosine_accuracy@1	0.4924
cosine_accuracy@3	0.5328
cosine_accuracy@5	0.5707
cosine_accuracy@10	0.6212
cosine_precision@1	0.4924
cosine_precision@3	0.4747
cosine_precision@5	0.4439
cosine_precision@10	0.3828
cosine_recall@1	0.105
cosine_recall@3	0.257
cosine_recall@5	0.3459
cosine_recall@10	0.472
cosine_ndcg@10	0.5541
cosine_mrr@10	0.5217
cosine_map@100	0.6007

Information Retrieval

Dataset: dim_256
Evaluated with InformationRetrievalEvaluator with these parameters:
json
1{ 2 "truncate_dim": 256 3}

Metric	Value
cosine_accuracy@1	0.4621
cosine_accuracy@3	0.5
cosine_accuracy@5	0.548
cosine_accuracy@10	0.5783
cosine_precision@1	0.4621
cosine_precision@3	0.4461
cosine_precision@5	0.4227
cosine_precision@10	0.3644
cosine_recall@1	0.0973
cosine_recall@3	0.238
cosine_recall@5	0.3246
cosine_recall@10	0.4473
cosine_ndcg@10	0.5236
cosine_mrr@10	0.4904
cosine_map@100	0.5706

Information Retrieval

Dataset: dim_128
Evaluated with InformationRetrievalEvaluator with these parameters:
json
1{ 2 "truncate_dim": 128 3}

Metric	Value
cosine_accuracy@1	0.4596
cosine_accuracy@3	0.4823
cosine_accuracy@5	0.5202
cosine_accuracy@10	0.5758
cosine_precision@1	0.4596
cosine_precision@3	0.4377
cosine_precision@5	0.4071
cosine_precision@10	0.3545
cosine_recall@1	0.097
cosine_recall@3	0.2362
cosine_recall@5	0.314
cosine_recall@10	0.4391
cosine_ndcg@10	0.5128
cosine_mrr@10	0.4834
cosine_map@100	0.5606

Information Retrieval

Dataset: dim_64
Evaluated with InformationRetrievalEvaluator with these parameters:
json
1{ 2 "truncate_dim": 64 3}

Metric	Value
cosine_accuracy@1	0.3662
cosine_accuracy@3	0.4015
cosine_accuracy@5	0.4571
cosine_accuracy@10	0.5126
cosine_precision@1	0.3662
cosine_precision@3	0.3544
cosine_precision@5	0.3369
cosine_precision@10	0.3048
cosine_recall@1	0.0789
cosine_recall@3	0.1968
cosine_recall@5	0.2664
cosine_recall@10	0.3808
cosine_ndcg@10	0.4353
cosine_mrr@10	0.3988
cosine_map@100	0.4852

Training Details

Training Dataset

Unnamed Dataset

Size: 1,580 training samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 8 tokens
mean: 17.23 tokens
max: 37 tokens
min: 27 tokens
mean: 739.0 tokens
max: 2824 tokens

Samples:

anchor	positive
`How has technology affected the collection and sharing of personal data?`	Rapid technological developments and globalisation have brought new challenges for the protection of personal data. The scale of the collection and sharing of personal data has increased significantly. Technology allows both private companies and public authorities to make use of personal data on an unprecedented scale in order to pursue their activities. Natural persons increasingly make personal information available publicly and globally. Technology has transformed both the economy and social life, and should further facilitate the free flow of personal data within the Union and the transfer to third countries and international organisations, while ensuring a high level of the protection of personal data.
`When was the Official Journal of the European Union published?`	In order to create incentives to apply pseudonymisation when processing personal data, measures of pseudonymisation should, whilst allowing general analysis, be possible within the same controller when that controller has taken technical and organisational measures necessary to ensure, for the processing concerned, that this Regulation is implemented, and that additional information for attributing the personal data to a specific data subject is kept separately. The controller processing the personal data should indicate the authorised persons within the same controller. 4.5.2016 L 119/5 Official Journal of the European Union EN
`What is the meaning of 'processing' in the context of personal data?`	1) 'personal data' means any information relating to an identified or identifiable natural person ('data subject'); an identifiable natural person is one who can be identified, directly or indirectly, in particular by reference to an identifier such as a name, an identification number, location data, an online identifier or to one or more factors specific to the physical, physiological, genetic, mental, economic, cultural or social identity of that natural person;[object Object](2) ‘processing’ means any operation or set of operations which is performed on personal data or on sets of personal data, whether or not by automated means, such as collection, recording, organisation, structuring, storage, adaptation or alteration, retrieval, consultation, use, disclosure by transmission, dissemination or otherwise making available, alignment or combination, restriction, erasure or destruction;[object Object](3) ‘restriction of processing’ means the marking of stored personal data with the aim of limiting their processin...

Loss: MatryoshkaLoss with these parameters:
json
1{ 2 "loss": "MultipleNegativesRankingLoss", 3 "matryoshka_dims": [ 4 768, 5 512, 6 256, 7 128, 8 64 9 ], 10 "matryoshka_weights": [ 11 1, 12 1, 13 1, 14 1, 15 1 16 ], 17 "n_dims_per_step": -1 18}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
gradient_accumulation_steps: 4
learning_rate: 3e-05
num_train_epochs: 20
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
load_best_model_at_end: True
optim: adamw_torch_fused
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 4
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 3e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 20
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
tp_size: 0
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	dim_768_cosine_ndcg@10	dim_512_cosine_ndcg@10	dim_256_cosine_ndcg@10	dim_128_cosine_ndcg@10	dim_64_cosine_ndcg@10
-1	-1	-	0.4571	0.4375	0.4008	0.3170	0.2563
0.2020	10	21.3767	-	-	-	-	-
0.4040	20	19.2573	-	-	-	-	-
0.6061	30	18.6623	-	-	-	-	-
0.8081	40	18.2772	-	-	-	-	-
1.0	50	16.0375	0.4596	0.4511	0.4154	0.3519	0.2872
1.2020	60	17.5302	-	-	-	-	-
1.4040	70	17.0313	-	-	-	-	-
1.6061	80	14.3965	-	-	-	-	-
1.8081	90	14.3407	-	-	-	-	-
2.0	100	12.6464	0.4835	0.4805	0.4585	0.4101	0.3288
2.2020	110	12.1612	-	-	-	-	-
2.4040	120	12.1747	-	-	-	-	-
2.6061	130	10.8968	-	-	-	-	-
2.8081	140	11.3075	-	-	-	-	-
3.0	150	9.3359	0.5314	0.5141	0.4915	0.4568	0.3777
3.2020	160	8.9723	-	-	-	-	-
3.4040	170	9.4735	-	-	-	-	-
3.6061	180	9.3438	-	-	-	-	-
3.8081	190	9.643	-	-	-	-	-
4.0	200	8.3103	0.5374	0.5426	0.5113	0.4848	0.4180
4.2020	210	7.3441	-	-	-	-	-
4.4040	220	7.8091	-	-	-	-	-
4.6061	230	7.1368	-	-	-	-	-
4.8081	240	7.7728	-	-	-	-	-
5.0	250	8.0139	0.5495	0.5443	0.5049	0.4923	0.4108
5.2020	260	5.6836	-	-	-	-	-
5.4040	270	6.0556	-	-	-	-	-
5.6061	280	7.1601	-	-	-	-	-
5.8081	290	7.8578	-	-	-	-	-
6.0	300	5.7557	0.5700	0.5564	0.5235	0.5106	0.4292
6.2020	310	5.7718	-	-	-	-	-
6.4040	320	5.1293	-	-	-	-	-
6.6061	330	6.7643	-	-	-	-	-
6.8081	340	5.1791	-	-	-	-	-
7.0	350	5.4592	0.5624	0.5541	0.5236	0.5128	0.4353
7.2020	360	5.0135	-	-	-	-	-
7.4040	370	4.9931	-	-	-	-	-
7.6061	380	5.1182	-	-	-	-	-
7.8081	390	4.5808	-	-	-	-	-
8.0	400	4.6313	0.5716	0.5549	0.5362	0.4893	0.4189
8.2020	410	5.0518	-	-	-	-	-
8.4040	420	4.7227	-	-	-	-	-
8.6061	430	3.5472	-	-	-	-	-
8.8081	440	5.4843	-	-	-	-	-
9.0	450	4.6137	0.5738	0.5589	0.5320	0.4849	0.4286
-1	-1	-	0.5624	0.5541	0.5236	0.5128	0.4353

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.12.11
Sentence Transformers: 5.1.0
Transformers: 4.51.3
PyTorch: 2.8.0+cu126
Accelerate: 1.10.1
Datasets: 4.0.0
Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

bibtex

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

bibtex

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

	anchor	positive
type	string	string
details	min: 8 tokens mean: 17.23 tokens max: 37 tokens	min: 27 tokens mean: 739.0 tokens max: 2824 tokens

IoannisKat1/bge-m3-new2

bge-m3

Model Details

Model Description

Model Sources

Full Model Architecture

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Information Retrieval

Information Retrieval

Information Retrieval

Information Retrieval

Information Retrieval

Training Details

Training Dataset

Unnamed Dataset

Training Hyperparameters

Non-Default Hyperparameters

All Hyperparameters

Training Logs

Framework Versions

Citation

BibTeX

Sentence Transformers

MatryoshkaLoss

MultipleNegativesRankingLoss

IoannisKat1/
bge-m3-new2