This is a sentence-transformers model finetuned from Snowflake/snowflake-arctic-embed-l. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: Snowflake/snowflake-arctic-embed-l
Maximum Sequence Length: 512 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("suh4s/legal-ft-0cebf579-602c-4040-b967-11629361b744")
# Run inference
sentences = [
    'What benefits are mentioned for those who comply with the terms of the Project Gutenberg agreement?',
    '1.B. "Project Gutenberg" is a registered trademark. It may only be\nused on or associated in any way with an electronic work by people who\nagree to be bound by the terms of this agreement. There are a few\nthings that you can do with most Project Gutenberg-tm electronic works\neven without complying with the full terms of this agreement. See\nparagraph 1.C below. There are a lot of things you can do with Project\nGutenberg-tm electronic works if you follow the terms of this\nagreement and help preserve free future access to Project Gutenberg-tm\nelectronic works. See paragraph 1.E below.',
    'This is about all that he has to say. He will not entreat the judges to\nspare his life; neither will he present a spectacle of weeping\nchildren, although he, too, is not made of “rock or oak.” Some of the\njudges themselves may have complied with this practice on similar\noccasions, and he trusts that they will not be angry with him for not\nfollowing their example. But he feels that such conduct brings\ndiscredit on the name of Athens: he feels too, that the judge has sworn\nnot to give away justice; and he cannot be guilty of the impiety of\nasking the judge to break his oath, when he is himself being tried for\nimpiety.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Information Retrieval

Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5729
cosine_accuracy@3	0.7271
cosine_accuracy@5	0.7792
cosine_accuracy@10	0.8417
cosine_precision@1	0.5729
cosine_precision@3	0.2424
cosine_precision@5	0.1558
cosine_precision@10	0.0842
cosine_recall@1	0.5729
cosine_recall@3	0.7271
cosine_recall@5	0.7792
cosine_recall@10	0.8417
cosine_ndcg@10	0.7066
cosine_mrr@10	0.6635
cosine_map@100	0.6715

Information Retrieval

Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5458
cosine_accuracy@3	0.725
cosine_accuracy@5	0.7833
cosine_accuracy@10	0.8667
cosine_precision@1	0.5458
cosine_precision@3	0.2417
cosine_precision@5	0.1567
cosine_precision@10	0.0867
cosine_recall@1	0.5458
cosine_recall@3	0.725
cosine_recall@5	0.7833
cosine_recall@10	0.8667
cosine_ndcg@10	0.7036
cosine_mrr@10	0.6518
cosine_map@100	0.6594

Training Details

Training Dataset

Unnamed Dataset

Size: 3,562 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 9 tokens
mean: 18.47 tokens
max: 47 tokens
min: 3 tokens
mean: 125.56 tokens
max: 220 tokens

Samples:

sentence_0	sentence_1
`What causes Kay to burst into tears in the passage?`	"Roses bloom and fade away,[object Object] But we the Christ-child see alway."[object Object][object Object]Then Kay burst into tears. He wept so that the splinter of glass swam[object Object]out of his eye. Then he recognized Gerda and said joyfully, "Gerda, dear[object Object]little Gerda, where have you been all this time, and where have I been?"[object Object]And he looked all around him and said, "How cold it is, and how large[object Object]and empty it all looks," and he clung to Gerda, and she laughed and wept[object Object]for joy.
`How does Kay react upon recognizing Gerda?`	"Roses bloom and fade away,[object Object] But we the Christ-child see alway."[object Object][object Object]Then Kay burst into tears. He wept so that the splinter of glass swam[object Object]out of his eye. Then he recognized Gerda and said joyfully, "Gerda, dear[object Object]little Gerda, where have you been all this time, and where have I been?"[object Object]And he looked all around him and said, "How cold it is, and how large[object Object]and empty it all looks," and he clung to Gerda, and she laughed and wept[object Object]for joy.
`What editions were compared to prepare the text mentioned in the context?`	The text itself has been prepared by a comparison of the editions of[object Object]1634 and 1635. It should be borne in mind that Casaubon's is often[object Object]rather a paraphrase than a close translation; and it did not seem worth[object Object]while to notice every variation or amplification of the original. In[object Object]the original editions all that Casaubon conceives as understood, but[object Object]not expressed, is enclosed in square brackets. These brackets are here[object Object]omitted, as they interfere with the comfort of the reader; and so have[object Object]some of the alternative renderings suggested by the translator. In a few[object Object]cases, Latin words in the text have been replaced by English.

Loss: MatryoshkaLoss with these parameters:
json
1{ 2 "loss": "MultipleNegativesRankingLoss", 3 "matryoshka_dims": [ 4 768, 5 512, 6 256, 7 128, 8 64 9 ], 10 "matryoshka_weights": [ 11 1, 12 1, 13 1, 14 1, 15 1 16 ], 17 "n_dims_per_step": -1 18}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
num_train_epochs: 10
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 10
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
tp_size: 0
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin

Training Logs

Click to expand

Epoch	Step	Training Loss	cosine_ndcg@10
0.0561	50	-	0.5573
0.1122	100	-	0.6540
0.1684	150	-	0.6854
0.2245	200	-	0.7066
0.1121	50	-	0.7155
0.2242	100	-	0.7171
0.3363	150	-	0.7278
0.4484	200	-	0.7270
0.5605	250	-	0.7112
0.6726	300	-	0.7238
0.7848	350	-	0.7255
0.8969	400	-	0.7120
1.0	446	-	0.7164
1.0090	450	-	0.7170
1.1211	500	0.4113	0.7293
1.2332	550	-	0.7032
1.3453	600	-	0.7130
1.4574	650	-	0.6967
1.5695	700	-	0.7132
1.6816	750	-	0.7089
1.7937	800	-	0.7106
1.9058	850	-	0.7148
2.0	892	-	0.7118
2.0179	900	-	0.7072
2.1300	950	-	0.7111
2.2422	1000	0.2493	0.7178
2.3543	1050	-	0.7102
2.4664	1100	-	0.7090
2.5785	1150	-	0.7098
2.6906	1200	-	0.7136
2.8027	1250	-	0.7059
2.9148	1300	-	0.7131
3.0	1338	-	0.7166
3.0269	1350	-	0.7076
3.1390	1400	-	0.7101
3.2511	1450	-	0.7095
3.3632	1500	0.1173	0.6990
3.4753	1550	-	0.7016
3.5874	1600	-	0.6802
3.6996	1650	-	0.6917
3.8117	1700	-	0.7028
3.9238	1750	-	0.6976
4.0	1784	-	0.7015
4.0359	1800	-	0.6949
4.1480	1850	-	0.6894
4.2601	1900	-	0.6926
4.3722	1950	-	0.6929
4.4843	2000	0.052	0.6984
4.5964	2050	-	0.7068
4.7085	2100	-	0.7101
4.8206	2150	-	0.6962
4.9327	2200	-	0.6968
5.0	2230	-	0.7083
5.0448	2250	-	0.7026
5.1570	2300	-	0.7023
5.2691	2350	-	0.7032
5.3812	2400	-	0.7087
5.4933	2450	-	0.7171
5.6054	2500	0.0291	0.7056
5.7175	2550	-	0.6969
5.8296	2600	-	0.7000
5.9417	2650	-	0.7012
6.0	2676	-	0.7068
6.0538	2700	-	0.7084
6.1659	2750	-	0.7002
6.2780	2800	-	0.7015
6.3901	2850	-	0.6974
6.5022	2900	-	0.6903
6.6143	2950	-	0.6943
6.7265	3000	0.0304	0.6938
6.8386	3050	-	0.6988
6.9507	3100	-	0.7001
7.0	3122	-	0.7004
7.0628	3150	-	0.7004
7.1749	3200	-	0.6950
7.2870	3250	-	0.6975
7.3991	3300	-	0.6927
7.5112	3350	-	0.6974
7.6233	3400	-	0.6928
7.7354	3450	-	0.6963
7.8475	3500	0.0207	0.6930
7.9596	3550	-	0.6938
8.0	3568	-	0.6952
8.0717	3600	-	0.6936
8.1839	3650	-	0.7045
8.2960	3700	-	0.6964
8.4081	3750	-	0.6940
8.5202	3800	-	0.6989
8.6323	3850	-	0.6991
8.7444	3900	-	0.6992
8.8565	3950	-	0.6997
8.9686	4000	0.0228	0.7004
9.0	4014	-	0.6993
9.0807	4050	-	0.7030
9.1928	4100	-	0.7040
9.3049	4150	-	0.7071
9.4170	4200	-	0.7059
9.5291	4250	-	0.7049
9.6413	4300	-	0.7037
9.7534	4350	-	0.7037
9.8655	4400	-	0.7038
9.9776	4450	-	0.7036
10.0	4460	-	0.7036

Framework Versions

Python: 3.11.12
Sentence Transformers: 4.1.0
Transformers: 4.51.3
PyTorch: 2.6.0+cu124
Accelerate: 1.6.0
Datasets: 3.6.0
Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MatryoshkaLoss

bibtex

@misc{kusupati2024matryoshka,
    title={Matryoshka Representation Learning},
    author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
    year={2024},
    eprint={2205.13147},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

MultipleNegativesRankingLoss

bibtex

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

	sentence_0	sentence_1
type	string	string
details	min: 9 tokens mean: 18.47 tokens max: 47 tokens	min: 3 tokens mean: 125.56 tokens max: 220 tokens