This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-m-v2.0 on the train dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: Snowflake/snowflake-arctic-embed-m-v2.0
Maximum Sequence Length: 8192 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- train

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 8192, 'do_lower_case': False, 'architecture': 'GteModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("BjarneNPO/finetune_21_08_2025_16_36_24")
# Run inference
queries = [
    "fragt wie der Stand zu dem aktuellen Problem ist",
]
documents = [
    'In Klärung mit der Kollegin - Das Problem liegt leider an deren Betreiber. Die sind aber informiert und arbeiten bereits daran',
    'findet diese in der Übersicht der Gruppen.',
    'Userin muss sich an die Bistums IT wenden.',
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 768] [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[ 0.2860, -0.0155,  0.0361]])

Evaluation

Metrics

Information Retrieval

Dataset: Snowflake/snowflake-arctic-embed-m-v2.0
Evaluated with scripts.InformationRetrievalEvaluatorCustom.InformationRetrievalEvaluatorCustom with these parameters:
json
1{ 2 "query_prompt_name": "query", 3 "corpus_prompt_name": "document" 4}

Metric	Value
cosine_accuracy@1	0.2263
cosine_accuracy@3	0.5182
cosine_accuracy@5	0.5912
cosine_accuracy@10	0.6715
cosine_precision@1	0.2263
cosine_precision@3	0.2311
cosine_precision@5	0.235
cosine_precision@10	0.2328
cosine_recall@1	0.0069
cosine_recall@3	0.0323
cosine_recall@5	0.0579
cosine_recall@10	0.1006
cosine_ndcg@10	0.2429
cosine_mrr@10	0.388
cosine_map@100	0.1079

Training Details

Training Dataset

train

Dataset: train
Size: 19,964 training samples
Columns: query and answer
Approximate statistics based on the first 1000 samples:
query answer
type string string
details
min: 4 tokens
mean: 27.77 tokens
max: 615 tokens
min: 3 tokens
mean: 22.87 tokens
max: 151 tokens

Samples:

query	answer
`Wie kann man die Jahresurlaubsübersicht exportieren?`	`über das 3 Punkte Menü rechts oben. Mitarbeiter auswählen und exportieren`
`1. Vertragsabschlüsse werden nicht übertragen`
2. Kinder kommen nicht von nach
3. Absage kann bei Portalstatus nicht erstellt werden.	`Ticket`
Userin gebeten sich an den Support zu wenden, da der Fehler liegt.
`Wird im Anmeldeportal nicht gefunden.`	`Die Schnittstelle war noch nicht aktiviert und Profil ebenfalls nicht.`

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
gradient_accumulation_steps: 8
learning_rate: 2e-05
num_train_epochs: 30
lr_scheduler_type: cosine
warmup_ratio: 0.1
bf16: True
tf32: False
load_best_model_at_end: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 8
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 30
max_steps: -1
lr_scheduler_type: cosine
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: True
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: False
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Snowflake/snowflake-arctic-embed-m-v2.0_cosine_ndcg@10
0.1282	10	2.8405	-
0.2564	20	2.7926	-
0.3846	30	2.813	-
0.5128	40	2.6249	-
0.6410	50	2.5239	-
0.7692	60	2.4235	-
0.8974	70	2.3373	-
1.0	78	-	0.2362
1.0256	80	2.2133	-
1.1538	90	2.1829	-
1.2821	100	2.0595	-
1.4103	110	2.0093	-
1.5385	120	1.9229	-
1.6667	130	1.8385	-
1.7949	140	1.7497	-
1.9231	150	1.7266	-
2.0	156	-	0.2843
2.0513	160	1.5565	-
2.1795	170	1.603	-
2.3077	180	1.5571	-
2.4359	190	1.5195	-
2.5641	200	1.507	-
2.6923	210	1.3831	-
2.8205	220	1.4236	-
2.9487	230	1.3074	-
3.0	234	-	0.2604
3.0769	240	1.2417	-
3.2051	250	1.2665	-
3.3333	260	1.2308	-
3.4615	270	1.2236	-
3.5897	280	1.1529	-
3.7179	290	1.1073	-
3.8462	300	1.0869	-
3.9744	310	1.0643	-
4.0	312	-	0.2556
4.1026	320	0.9498	-
4.2308	330	1.0315	-
4.3590	340	1.02	-
4.4872	350	0.9776	-
4.6154	360	0.9402	-
4.7436	370	0.8848	-
4.8718	380	0.9032	-
5.0	390	0.8495	0.2446
5.1282	400	0.8177	-
5.2564	410	0.8437	-
5.3846	420	0.8357	-
5.5128	430	0.812	-
5.6410	440	0.7483	-
5.7692	450	0.7256	-
5.8974	460	0.7517	-
6.0	468	-	0.2438
6.0256	470	0.6514	-
6.1538	480	0.6968	-
6.2821	490	0.6787	-
6.4103	500	0.6681	-
6.5385	510	0.6641	-
6.6667	520	0.6134	-
6.7949	530	0.6142	-
6.9231	540	0.6174	-
7.0	546	-	0.2497
7.0513	550	0.5282	-
7.1795	560	0.5788	-
7.3077	570	0.5827	-
7.4359	580	0.5651	-
7.5641	590	0.5564	-
7.6923	600	0.4742	-
7.8205	610	0.5336	-
7.9487	620	0.4734	-
8.0	624	-	0.2346
8.0769	630	0.4536	-
8.2051	640	0.4689	-
8.3333	650	0.4689	-
8.4615	660	0.466	-
8.5897	670	0.4224	-
8.7179	680	0.4031	-
8.8462	690	0.4304	-
8.9744	700	0.4021	-
9.0	702	-	0.2496
9.1026	710	0.3539	-
9.2308	720	0.4073	-
9.3590	730	0.4026	-
9.4872	740	0.3799	-
9.6154	750	0.3551	-
9.7436	760	0.341	-
9.8718	770	0.3426	-
10.0	780	0.3236	0.2439
10.1282	790	0.3102	-
10.2564	800	0.3441	-
10.3846	810	0.3224	-
10.5128	820	0.3253	-
10.6410	830	0.2874	-
10.7692	840	0.2949	-
10.8974	850	0.2972	-
11.0	858	-	0.2403
11.0256	860	0.2458	-
11.1538	870	0.2849	-
11.2821	880	0.275	-
11.4103	890	0.2721	-
11.5385	900	0.2627	-
11.6667	910	0.2441	-
11.7949	920	0.2491	-
11.9231	930	0.2418	-
12.0	936	-	0.2515
12.0513	940	0.2018	-
12.1795	950	0.2316	-
12.3077	960	0.2389	-
12.4359	970	0.2291	-
12.5641	980	0.2254	-
12.6923	990	0.1941	-
12.8205	1000	0.2183	-
12.9487	1010	0.1923	-
13.0	1014	-	0.2463
13.0769	1020	0.1881	-
13.2051	1030	0.1917	-
13.3333	1040	0.2072	-
13.4615	1050	0.2046	-
13.5897	1060	0.1828	-
13.7179	1070	0.1741	-
13.8462	1080	0.1816	-
13.9744	1090	0.1701	-
14.0	1092	-	0.2536
14.1026	1100	0.1542	-
14.2308	1110	0.1769	-
14.3590	1120	0.1718	-
14.4872	1130	0.1782	-
14.6154	1140	0.1613	-
14.7436	1150	0.1565	-
14.8718	1160	0.1561	-
15.0	1170	0.1455	0.2524
15.1282	1180	0.1384	-
15.2564	1190	0.1566	-
15.3846	1200	0.1558	-
15.5128	1210	0.1469	-
15.6410	1220	0.1392	-
15.7692	1230	0.1358	-
15.8974	1240	0.1396	-
16.0	1248	-	0.2504
16.0256	1250	0.1118	-
16.1538	1260	0.136	-
16.2821	1270	0.1333	-
16.4103	1280	0.1332	-
16.5385	1290	0.1261	-
16.6667	1300	0.1238	-
16.7949	1310	0.1309	-
16.9231	1320	0.1256	-
17.0	1326	-	0.2510
17.0513	1330	0.0975	-
17.1795	1340	0.1191	-
17.3077	1350	0.1275	-
17.4359	1360	0.1194	-
17.5641	1370	0.1169	-
17.6923	1380	0.1033	-
17.8205	1390	0.1225	-
17.9487	1400	0.1036	-
18.0	1404	-	0.2552
18.0769	1410	0.102	-
18.2051	1420	0.1131	-
18.3333	1430	0.1109	-
18.4615	1440	0.1152	-
18.5897	1450	0.0969	-
18.7179	1460	0.0983	-
18.8462	1470	0.1104	-
18.9744	1480	0.0948	-
19.0	1482	-	0.2504
19.1026	1490	0.0896	-
19.2308	1500	0.0995	-
19.3590	1510	0.0996	-
19.4872	1520	0.0965	-
19.6154	1530	0.0935	-
19.7436	1540	0.0885	-
19.8718	1550	0.0917	-
20.0	1560	0.0864	0.2510
20.1282	1570	0.0822	-
20.2564	1580	0.0993	-
20.3846	1590	0.0907	-
20.5128	1600	0.1003	-
20.6410	1610	0.0866	-
20.7692	1620	0.0885	-
20.8974	1630	0.0943	-
21.0	1638	-	0.2523
21.0256	1640	0.0701	-
21.1538	1650	0.0886	-
21.2821	1660	0.0889	-
21.4103	1670	0.087	-
21.5385	1680	0.085	-
21.6667	1690	0.083	-
21.7949	1700	0.0836	-
21.9231	1710	0.0831	-
22.0	1716	-	0.2461
22.0513	1720	0.0681	-
22.1795	1730	0.0817	-
22.3077	1740	0.0864	-
22.4359	1750	0.0844	-
22.5641	1760	0.0842	-
22.6923	1770	0.0685	-
22.8205	1780	0.0859	-
22.9487	1790	0.0716	-
23.0	1794	-	0.2473
23.0769	1800	0.072	-
23.2051	1810	0.0796	-
23.3333	1820	0.0815	-
23.4615	1830	0.0847	-
23.5897	1840	0.078	-
23.7179	1850	0.07	-
23.8462	1860	0.0804	-
23.9744	1870	0.0698	-
24.0	1872	-	0.2463
24.1026	1880	0.0674	-
24.2308	1890	0.0809	-
24.3590	1900	0.074	-
24.4872	1910	0.0793	-
24.6154	1920	0.0746	-
24.7436	1930	0.0675	-
24.8718	1940	0.0745	-
25.0	1950	0.0646	0.2437
25.1282	1960	0.0659	-
25.2564	1970	0.0746	-
25.3846	1980	0.0738	-
25.5128	1990	0.08	-
25.6410	2000	0.0704	-
25.7692	2010	0.0741	-
25.8974	2020	0.0733	-
26.0	2028	-	0.2405
26.0256	2030	0.0602	-
26.1538	2040	0.0724	-
26.2821	2050	0.071	-
26.4103	2060	0.0745	-
26.5385	2070	0.0743	-
26.6667	2080	0.0682	-
26.7949	2090	0.0744	-
26.9231	2100	0.0702	-
27.0	2106	-	0.2453
27.0513	2110	0.0573	-
27.1795	2120	0.0706	-
27.3077	2130	0.0721	-
27.4359	2140	0.0709	-
27.5641	2150	0.0711	-
27.6923	2160	0.0642	-
27.8205	2170	0.0762	-
27.9487	2180	0.0629	-
28.0	2184	-	0.2407
28.0769	2190	0.0666	-
28.2051	2200	0.069	-
28.3333	2210	0.0708	-
28.4615	2220	0.072	-
28.5897	2230	0.068	-
28.7179	2240	0.0622	-
28.8462	2250	0.0705	-
28.9744	2260	0.0674	-
29.0	2262	-	0.2403
29.1026	2270	0.062	-
29.2308	2280	0.0728	-
29.3590	2290	0.0699	-
29.4872	2300	0.0745	-
29.6154	2310	0.0687	-
29.7436	2320	0.0671	-
29.8718	2330	0.071	-
30.0	2340	0.061	0.2429

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.10.11
Sentence Transformers: 5.1.0
Transformers: 4.55.2
PyTorch: 2.8.0+cu129
Accelerate: 1.10.0
Datasets: 3.6.0
Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

bibtex

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

	query	answer
type	string	string
details	min: 4 tokens mean: 27.77 tokens max: 615 tokens	min: 3 tokens mean: 22.87 tokens max: 151 tokens