This is a sentence-transformers model finetuned from BAAI/bge-m3. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-m3
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'Umweltgutachten',
    'Zentralblatt der Bauverwaltung Nachrichten d. Reichs- u. Staatsbehörden',
    'Materialfluss Materialfluss <Landsberg> / Portrait einer Branche',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.3120, 0.2367],
#         [0.3120, 1.0000, 0.2667],
#         [0.2367, 0.2667, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 5,139,569 training samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 3 tokens
mean: 20.6 tokens
max: 101 tokens
min: 3 tokens
mean: 20.99 tokens
max: 100 tokens

Samples:

anchor	positive
`The art of Star Wars - das Erwachen der Macht`	`Harry Potter: magische Orte aus den Filmen`
`Fachdidaktik Kunst und Design Lehren und Lernen mit Portfolios`	`Mit Kindern moderne Kunst entdecken kreative Ideen auch für Fachfremde, 2. - 4. Schuljahr ; [mit Farbabbildungen und Kopiervorlagen auf CD-ROM]`
`Der Ditz alles was man über Niederländer wissen sollte`	`Umgangsformen Protokoll und Etikette, privat und im Beruf`

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Evaluation Dataset

Unnamed Dataset

Size: 3,736 evaluation samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 3 tokens
mean: 19.09 tokens
max: 96 tokens
min: 3 tokens
mean: 18.28 tokens
max: 96 tokens

Samples:

anchor	positive
`Bibliotheksstatistik staatliche Allgemeinbibliotheken und Gewerkschaftsbibliotheken der Deutschen Demokratischen Republik in Zahlen ; Bericht und Tabellen zu den Gesamtergebnissen in der Republik und in den Bezirken einschließlich der Wissenschaftlichen Allgemeinbibliotheken der Bezirke`	`Fortschrittsbericht Bohrtechnik, Erdöl und Erdgasgewinnung und verwandte Gebiete`
`Bibliotheksstatistik staatliche Allgemeinbibliotheken und Gewerkschaftsbibliotheken der Deutschen Demokratischen Republik in Zahlen ; Bericht und Tabellen zu den Gesamtergebnissen in der Republik und in den Bezirken einschließlich der Wissenschaftlichen Allgemeinbibliotheken der Bezirke`	`Zentralkatalog der DDR ZKZ ; Zeitschriften u. Serien d. Auslandes ZKZ`
`Zentralkatalog der DDR ZKZ ; Zeitschriften u. Serien d. Auslandes ZKZ`	`Fortschrittsbericht Bohrtechnik, Erdöl und Erdgasgewinnung und verwandte Gebiete`

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 64
per_device_eval_batch_size: 64
learning_rate: 1e-05
num_train_epochs: 2

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 64
per_device_eval_batch_size: 64
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 1e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 2
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
tp_size: 0
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss
0.0062	500	2.1697	-
0.0125	1000	2.0869	2.2847
0.0187	1500	2.0643	-
0.0249	2000	2.0358	2.2490
0.0311	2500	2.0275	-
0.0374	3000	2.0191	2.2716
0.0436	3500	2.0037	-
0.0498	4000	1.9998	2.2407
0.0560	4500	1.976	-
0.0623	5000	1.9776	2.2088
0.0685	5500	1.9705	-
0.0747	6000	1.9796	2.2011
0.0809	6500	1.952	-
0.0872	7000	1.9462	2.2119
0.0934	7500	1.9466	-
0.0996	8000	1.9162	2.1985
0.1058	8500	1.9171	-
0.1121	9000	1.9207	2.1956
0.1183	9500	1.9041	-
0.1245	10000	1.908	2.2035
0.1307	10500	1.9105	-
0.1370	11000	1.8817	2.1951
0.1432	11500	1.9109	-
0.1494	12000	1.9033	2.1413
0.1557	12500	1.8991	-
0.1619	13000	1.8875	2.1770
0.1681	13500	1.8777	-
0.1743	14000	1.8822	2.1892
0.1806	14500	1.8696	-
0.1868	15000	1.8628	2.1856
0.1930	15500	1.8828	-
0.1992	16000	1.8597	2.1755
0.2055	16500	1.8762	-
0.2117	17000	1.8724	2.1548
0.2179	17500	1.8685	-
0.2241	18000	1.8733	2.1681
0.2304	18500	1.852	-
0.2366	19000	1.8412	2.1868
0.2428	19500	1.859	-
0.2490	20000	1.8433	2.1439
0.2553	20500	1.852	-
0.2615	21000	1.8537	2.1446
0.2677	21500	1.803	-
0.2740	22000	1.8233	2.1722
0.2802	22500	1.8294	-
0.2864	23000	1.8297	2.1857
0.2926	23500	1.8306	-
0.2989	24000	1.8336	2.1519
0.3051	24500	1.8203	-
0.3113	25000	1.8247	2.1728
0.3175	25500	1.8077	-
0.3238	26000	1.7915	2.1985
0.3300	26500	1.8087	-
0.3362	27000	1.8128	2.1808
0.3424	27500	1.8139	-
0.3487	28000	1.8118	2.1518
0.3549	28500	1.8113	-
0.3611	29000	1.8077	2.1721
0.3673	29500	1.8193	-
0.3736	30000	1.7986	2.1783
0.3798	30500	1.8002	-
0.3860	31000	1.807	2.1628
0.3922	31500	1.7917	-
0.3985	32000	1.7757	2.1827
0.4047	32500	1.8038	-
0.4109	33000	1.7838	2.1703
0.4172	33500	1.7748	-
0.4234	34000	1.8039	2.1684
0.4296	34500	1.7674	-
0.4358	35000	1.7597	2.1902
0.4421	35500	1.7828	-
0.4483	36000	1.7747	2.1980
0.4545	36500	1.776	-
0.4607	37000	1.8078	2.1805
0.4670	37500	1.7814	-
0.4732	38000	1.7741	2.1616
0.4794	38500	1.7755	-
0.4856	39000	1.7576	2.1644
0.4919	39500	1.7562	-
0.4981	40000	1.7602	2.1709
0.5043	40500	1.757	-
0.5105	41000	1.7693	2.1938
0.5168	41500	1.7655	-
0.5230	42000	1.7617	2.1970
0.5292	42500	1.7687	-
0.5355	43000	1.7565	2.1630
0.5417	43500	1.7492	-
0.5479	44000	1.7579	2.1694
0.5541	44500	1.7629	-
0.5604	45000	1.7364	2.1836
0.5666	45500	1.7444	-
0.5728	46000	1.7308	2.1785
0.5790	46500	1.7351	-
0.5853	47000	1.7588	2.1523
0.5915	47500	1.7235	-
0.5977	48000	1.7317	2.1694
0.6039	48500	1.7415	-
0.6102	49000	1.7588	2.1867
0.6164	49500	1.7103	-
0.6226	50000	1.7269	2.1812
0.6288	50500	1.7254	-
0.6351	51000	1.7398	2.1663
0.6413	51500	1.7178	-
0.6475	52000	1.7454	2.1856
0.6537	52500	1.7188	-
0.6600	53000	1.7204	2.1688
0.6662	53500	1.72	-
0.6724	54000	1.7356	2.1861
0.6787	54500	1.7338	-
0.6849	55000	1.7254	2.1700
0.6911	55500	1.7243	-
0.6973	56000	1.7308	2.1781
0.7036	56500	1.7215	-
0.7098	57000	1.7115	2.1784
0.7160	57500	1.7071	-
0.7222	58000	1.7155	2.1884
0.7285	58500	1.7236	-
0.7347	59000	1.7078	2.1825
0.7409	59500	1.7056	-
0.7471	60000	1.724	2.1695
0.7534	60500	1.7077	-
0.7596	61000	1.6948	2.1562
0.7658	61500	1.6858	-
0.7720	62000	1.7207	2.1835
0.7783	62500	1.7086	-
0.7845	63000	1.7173	2.1645
0.7907	63500	1.717	-
0.7970	64000	1.7032	2.1732
0.8032	64500	1.6992	-
0.8094	65000	1.7061	2.1841
0.8156	65500	1.6926	-
0.8219	66000	1.6881	2.1970
0.8281	66500	1.6905	-
0.8343	67000	1.6924	2.1774
0.8405	67500	1.6926	-
0.8468	68000	1.6888	2.1826
0.8530	68500	1.6975	-
0.8592	69000	1.6884	2.1740
0.8654	69500	1.6843	-
0.8717	70000	1.6766	2.1717
0.8779	70500	1.6887	-
0.8841	71000	1.6827	2.1788
0.8903	71500	1.6922	-
0.8966	72000	1.6796	2.2068
0.9028	72500	1.6682	-
0.9090	73000	1.6731	2.1866
0.9152	73500	1.6747	-
0.9215	74000	1.6751	2.1737
0.9277	74500	1.6545	-
0.9339	75000	1.6828	2.1894
0.9402	75500	1.6649	-
0.9464	76000	1.6692	2.1638
0.9526	76500	1.6596	-
0.9588	77000	1.6649	2.1874
0.9651	77500	1.6731	-
0.9713	78000	1.6845	2.1805
0.9775	78500	1.69	-
0.9837	79000	1.6622	2.1604
0.9900	79500	1.6581	-
0.9962	80000	1.6763	2.1873
1.0024	80500	1.6351	-
1.0086	81000	1.5969	2.2085
1.0149	81500	1.598	-
1.0211	82000	1.5877	2.2237
1.0273	82500	1.6039	-
1.0335	83000	1.6065	2.2369
1.0398	83500	1.5922	-
1.0460	84000	1.5896	2.2240
1.0522	84500	1.5905	-
1.0585	85000	1.5725	2.2057
1.0647	85500	1.5865	-
1.0709	86000	1.5912	2.2256
1.0771	86500	1.5947	-
1.0834	87000	1.5734	2.2342
1.0896	87500	1.5942	-
1.0958	88000	1.5872	2.2254
1.1020	88500	1.5838	-
1.1083	89000	1.5681	2.2428
1.1145	89500	1.5838	-
1.1207	90000	1.5668	2.2431
1.1269	90500	1.5891	-
1.1332	91000	1.6029	2.2416
1.1394	91500	1.579	-
1.1456	92000	1.5797	2.2316
1.1518	92500	1.5944	-
1.1581	93000	1.581	2.2542
1.1643	93500	1.5601	-
1.1705	94000	1.5653	2.2278
1.1767	94500	1.562	-
1.1830	95000	1.5985	2.2356
1.1892	95500	1.5599	-
1.1954	96000	1.5738	2.2325
1.2017	96500	1.5809	-
1.2079	97000	1.5816	2.2242
1.2141	97500	1.5852	-
1.2203	98000	1.5611	2.2319
1.2266	98500	1.567	-
1.2328	99000	1.5615	2.2454
1.2390	99500	1.5696	-
1.2452	100000	1.5812	2.2469
1.2515	100500	1.566	-
1.2577	101000	1.5751	2.2255
1.2639	101500	1.5864	-
1.2701	102000	1.5796	2.2340
1.2764	102500	1.5646	-
1.2826	103000	1.573	2.2555
1.2888	103500	1.5824	-
1.2950	104000	1.5531	2.2520
1.3013	104500	1.5672	-
1.3075	105000	1.5619	2.2355
1.3137	105500	1.576	-
1.3200	106000	1.5767	2.2266
1.3262	106500	1.563	-
1.3324	107000	1.5627	2.2367
1.3386	107500	1.5589	-
1.3449	108000	1.5512	2.2533
1.3511	108500	1.5725	-
1.3573	109000	1.5454	2.2460
1.3635	109500	1.5609	-
1.3698	110000	1.549	2.2515
1.3760	110500	1.5587	-
1.3822	111000	1.5839	2.2551
1.3884	111500	1.5547	-
1.3947	112000	1.5433	2.2389
1.4009	112500	1.558	-
1.4071	113000	1.5542	2.2350
1.4133	113500	1.5641	-
1.4196	114000	1.5567	2.2444
1.4258	114500	1.5537	-
1.4320	115000	1.5537	2.2395
1.4382	115500	1.5501	-
1.4445	116000	1.5488	2.2517
1.4507	116500	1.5518	-
1.4569	117000	1.5631	2.2351
1.4632	117500	1.5626	-
1.4694	118000	1.5568	2.2289
1.4756	118500	1.5591	-
1.4818	119000	1.5448	2.2264
1.4881	119500	1.5463	-
1.4943	120000	1.5345	2.2319
1.5005	120500	1.5645	-
1.5067	121000	1.5457	2.2289
1.5130	121500	1.5509	-
1.5192	122000	1.5562	2.2302
1.5254	122500	1.5469	-
1.5316	123000	1.5514	2.2322
1.5379	123500	1.5686	-
1.5441	124000	1.5437	2.2453
1.5503	124500	1.5304	-
1.5565	125000	1.5609	2.2427
1.5628	125500	1.5416	-
1.5690	126000	1.5418	2.2385
1.5752	126500	1.5458	-
1.5815	127000	1.5735	2.2241
1.5877	127500	1.5601	-
1.5939	128000	1.546	2.2267
1.6001	128500	1.5419	-
1.6064	129000	1.5579	2.2396
1.6126	129500	1.5383	-
1.6188	130000	1.5451	2.2371
1.6250	130500	1.5505	-
1.6313	131000	1.5374	2.2264
1.6375	131500	1.5357	-
1.6437	132000	1.5223	2.2416
1.6499	132500	1.5312	-
1.6562	133000	1.5438	2.2300
1.6624	133500	1.5366	-
1.6686	134000	1.5354	2.2329
1.6748	134500	1.5316	-
1.6811	135000	1.5452	2.2388
1.6873	135500	1.5548	-
1.6935	136000	1.5448	2.2342
1.6997	136500	1.5281	-
1.7060	137000	1.529	2.2372
1.7122	137500	1.5254	-
1.7184	138000	1.5163	2.2371
1.7247	138500	1.537	-
1.7309	139000	1.5531	2.2455
1.7371	139500	1.5269	-
1.7433	140000	1.5299	2.2372
1.7496	140500	1.5331	-
1.7558	141000	1.5494	2.2293
1.7620	141500	1.5337	-
1.7682	142000	1.5268	2.2365
1.7745	142500	1.5331	-
1.7807	143000	1.5314	2.2318
1.7869	143500	1.5375	-
1.7931	144000	1.5375	2.2257
1.7994	144500	1.5386	-
1.8056	145000	1.5314	2.2387
1.8118	145500	1.5446	-
1.8180	146000	1.5257	2.2421
1.8243	146500	1.5275	-
1.8305	147000	1.5329	2.2389
1.8367	147500	1.5362	-
1.8430	148000	1.5556	2.2322
1.8492	148500	1.5214	-
1.8554	149000	1.5178	2.2337
1.8616	149500	1.5156	-
1.8679	150000	1.5244	2.2318
1.8741	150500	1.5283	-
1.8803	151000	1.5386	2.2400
1.8865	151500	1.5326	-
1.8928	152000	1.5339	2.2408
1.8990	152500	1.5201	-
1.9052	153000	1.5297	2.2402
1.9114	153500	1.5325	-
1.9177	154000	1.5503	2.2428
1.9239	154500	1.5382	-
1.9301	155000	1.534	2.2397
1.9363	155500	1.5245	-
1.9426	156000	1.5198	2.2380
1.9488	156500	1.5301	-
1.9550	157000	1.5324	2.2399
1.9612	157500	1.5171	-
1.9675	158000	1.5298	2.2399
1.9737	158500	1.5309	-
1.9799	159000	1.5053	2.2406
1.9862	159500	1.5237	-
1.9924	160000	1.5228	2.2414
1.9986	160500	1.539	-

Framework Versions

Python: 3.9.18
Sentence Transformers: 5.1.0
Transformers: 4.51.3
PyTorch: 2.6.0+cu124
Accelerate: 1.10.1
Datasets: 4.0.0
Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

bibtex

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

	anchor	positive
type	string	string
details	min: 3 tokens mean: 20.6 tokens max: 101 tokens	min: 3 tokens mean: 20.99 tokens max: 100 tokens

KatjaK/GND-few-shot-retriever

SentenceTransformer based on BAAI/bge-m3

Model Details

Model Description

Model Sources

Full Model Architecture

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

Evaluation Dataset

Unnamed Dataset

Training Hyperparameters

Non-Default Hyperparameters

All Hyperparameters

Training Logs

Framework Versions

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss

KatjaK/
GND-few-shot-retriever