This is a sentence-transformers model finetuned from sentence-transformers/all-mpnet-base-v2 on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/all-mpnet-base-v2
Maximum Sequence Length: 384 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- json

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 384, 'do_lower_case': False, 'architecture': 'MPNetModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
sentences = [
    'tation. A copy of the assessment, denial, or other determination you are disputing must be attached. Representative Information (complete only if you want someone to represent you during the reconsideration process) Power of Attorney (complete only if you want someone to represent you during the reconsideration process) By signing below, the petitioner(s) named in Section 1 appoints the individual named in Section 6 to act as their representative with full authority to receive confidential information and to perform any and all acts the petitioner can perform in connection with matters associated with this petition, except, the representative may not delegate their authority to another individual. If you wish to limit the authority granted by this Power of Attorney, please describe the limitation: A copy of the assessment, denial, or other determination (collectively referred to throughout these instructions as a “Determination”) you are disputing must be attached.',
    'Criminals are targeting human resources and financial professionals across Maine with a new phishing scheme. Don’t fall victim. If you get an email asking you to send employee W-2 or other private/sensitive information, stop to confirm if the request is legitimate before you hit send. Criminals have perfected techniques to trick you into thinking an email is coming from a person you work with. Don’t fall victim to this scam. Connect with the person who you believe sent you the request by phone or by walking over to see them. Do not respond to the email to confirm the sender’s request. The sender could be a criminal, disguising their identity with a fake email address. If you confirm a legitimate request, take steps to protect the information before you send it. and place W-2 Scam in the subject line. • File a complaint with the Internet Crime Complaint Center (IC3), operated by the Federal Bureau of Investigation.',
    'noise',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.2313, -0.0125],
#         [-0.2313,  1.0000,  0.0933],
#         [-0.0125,  0.0933,  1.0000]])

Training Details

Training Dataset

json

Dataset: json
Size: 16,975 training samples
Columns: sentence1, sentence2, score, and type

Approximate statistics based on the first 1000 samples:

	sentence1	sentence2	score	type
type	string	string	float	string
details	min: 36 tokens mean: 179.93 tokens max: 270 tokens	min: 21 tokens mean: 94.85 tokens max: 217 tokens	min: 0.05 mean: 0.49 max: 1.0	min: 3 tokens mean: 4.03 tokens max: 5 tokens

Samples:

sentence1	sentence2	score	type
additional cost incurred providing job coaching, training and supervision on a supported work site. : "Supported Employment/Work" is a consumer-oriented, integrated, and non-segregated employment which is based on the individual's informed choice and which provides appropriate ongoing services to individual s with most significant disabilities in order for the individual to work productively in the community. Specifically, individuals in supported employment/work must: A. be engaged in part-time or full-time employment which pays wages and benefits commensurate with the individuals ability to produce goods or render services and which is based on current competitive rates at or above the minimum wage .	`All industrial facilities must implement a confined space entry program, including written procedures, employee training, and rescue plans, to comply with workplace safety requirements.`	`0.3`	`soft_negative`
additional cost incurred providing job coaching, training and supervision on a supported work site. : "Supported Employment/Work" is a consumer-oriented, integrated, and non-segregated employment which is based on the individual's informed choice and which provides appropriate ongoing services to individual s with most significant disabilities in order for the individual to work productively in the community. Specifically, individuals in supported employment/work must: A. be engaged in part-time or full-time employment which pays wages and benefits commensurate with the individuals ability to produce goods or render services and which is based on current competitive rates at or above the minimum wage .	`“Supported Employment/Work” permits payment of wages below the minimum wage and allows supported workers to be paid without regard to current competitive rates.`	`0.6`	`hard_negative`
vices and which is based on current competitive rates at or above the minimum wage . hat is individualized, and customized, consistent with the unique strengths, abilities, interests, and informed choice of the individual, including with ongoing support services for individuals with the most significant disabilities – For whom competitive integrated employment has not historically occurred, or for whom competitive integrated employment has been interrupted or intermittent as a result of a significant disability; and Who, because of the nature and severity of their disabilities, need intensive supported employment services and extended services after the transition from support provided by the designated State unit, in order to perform this work.	`Licensed businesses that discharge process water must install and operate best available technology to limit nutrient discharges and must submit semiannual effluent monitoring reports to the environmental agency.`	`0.3`	`soft_negative`

Loss: CoSENTLoss with these parameters:
json
1{ 2 "scale": 40.0, 3 "similarity_fct": "pairwise_cos_sim" 4}

Evaluation Dataset

json

Dataset: json
Size: 2,139 evaluation samples
Columns: sentence1, sentence2, score, and type

Approximate statistics based on the first 1000 samples:

	sentence1	sentence2	score	type
type	string	string	float	string
details	min: 44 tokens mean: 181.52 tokens max: 329 tokens	min: 22 tokens mean: 86.31 tokens max: 227 tokens	min: 0.05 mean: 0.5 max: 1.0	min: 3 tokens mean: 3.98 tokens max: 5 tokens

Samples:

sentence1	sentence2	score	type
rce, after an individual has made the transition from support provided by the designated State unit. can produce a diminished or altered state of consciousness resulting in impairment of cognitive abilities or physical functioning; C. can result in the disturbance of behavioral or emotional functioning; E. can cause partial or total functional disability or psychological maladjustment. An applicant for or recipient of Extended Support services who is dissatisfied with any determination made by the Bureau of Rehabilitation Services concerning the furnishing or denial of services may request a timely review of the determination. The Bureau shall make reasonable accessibility accommodations for the individual with disabilities during the appeals process. Whenever possible, the Bureau will attempt to resolve conflicts through Informal Review or through Mediation. An individual may request a Due Process Hearing immediately without having to go through other appeal steps.	Conditions described can impair consciousness, cognition, physical functioning, behavior, or emotional functioning and may cause partial or total functional disability or psychological maladjustment. An applicant or recipient dissatisfied with a Bureau of Rehabilitation Services decision about providing or denying Extended Support may request a timely review. The Bureau must provide reasonable accessibility accommodations during the appeals process, will attempt to resolve disputes through Informal Review or Mediation when possible, and an individual may request a Due Process Hearing immediately without first completing other appeal steps.	`1.0`	`positive`
`The purpose of this program is to provide ongoing extended supports to individuals with brain injuries who are VR consumers with the most significant disabilities once training has been completed. The program provides financial assistance to providers of ongoing support and/or employers to help defray the additional cost incurred providing job coaching, training and supervision on a supported work site.`	`The program provides financial assistance only to individuals prior to completing training and does not fund supports for vocational rehabilitation consumers after training completion.`	`0.6`	`hard_negative`
r assessments and determinations. After I ﬁle a petition, do I still have to pay the amount due? No. However, interest, but not penalties, will continue to accrue during the appeals process. You can minimize additional interest by helping to get your case decided as quickly as possible. For example, if you have any documents that you want MRS to consider, you should attach copies to your petition. You should also be as speciﬁc as possible in explaining why you believe the assessment is not correct. However, if you have received a “jeopardy” assessment, you must immediately pay the assessed amount, or ﬁle a bond or other security, to prevent immediate collection proceedings. You may still ﬁle a petition within 60 days as with any other assessment. For any assessment, if you pay the amount due and MRS later determines that you do not owe some or all of the assessment, MRS will issue you a refund. Yes, if you think meeting with MRS would be helpful.	No, filing a petition does not generally require you to pay the assessed amount while the appeal is pending. However, interest (but not penalties) will continue to accrue during the appeals process. You can help minimize further interest by promptly providing any documents you want MRS to consider and by explaining specifically why the assessment is incorrect. If you received a “jeopardy” assessment, you must immediately pay the assessed amount or post a bond or other security to avoid immediate collection; you may still file a petition within 60 days. If you pay an assessment and MRS later determines you did not owe some or all of it, MRS will issue a refund. If you believe a meeting with MRS would help, you may request one.	`1.0`	`positive`

Loss: CoSENTLoss with these parameters:
json
1{ 2 "scale": 40.0, 3 "similarity_fct": "pairwise_cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: epoch
per_device_train_batch_size: 16
learning_rate: 1e-05
weight_decay: 0.01
num_train_epochs: 6
warmup_ratio: 0.1
fp16: True
load_best_model_at_end: True

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: epoch
prediction_loss_only: True
per_device_train_batch_size: 16
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 1e-05
weight_decay: 0.01
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 6
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: True
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss
0.4713	500	6.2293	-
0.9425	1000	1.8318	-
1.0	1061	-	0.5277
1.4138	1500	1.3701	-
1.8850	2000	0.9076	-
2.0	2122	-	0.4623
2.3563	2500	0.4613	-
2.8275	3000	0.3688	-
3.0	3183	-	0.3232
3.2988	3500	0.1959	-
3.7700	4000	0.1412	-
4.0	4244	-	0.3016
4.2413	4500	0.0838	-
4.7125	5000	0.0647	-
5.0	5305	-	0.3073
5.1838	5500	0.0338	-
5.6550	6000	0.0124	-
6.0	6366	-	0.2879

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.12.12
Sentence Transformers: 5.1.2
Transformers: 4.57.2
PyTorch: 2.9.0+cu126
Accelerate: 1.12.0
Datasets: 4.4.1
Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

CoSENTLoss

bibtex

@article{10531646,
    author={Huang, Xiang and Peng, Hao and Zou, Dongcheng and Liu, Zhiwei and Li, Jianxin and Liu, Kay and Wu, Jia and Su, Jianlin and Yu, Philip S.},
    journal={IEEE/ACM Transactions on Audio, Speech, and Language Processing},
    title={CoSENT: Consistent Sentence Embedding via Similarity Ranking},
    year={2024},
    doi={10.1109/TASLP.2024.3402087}
}

hassaangyt/compliance-emb-v1

SentenceTransformer based on sentence-transformers/all-mpnet-base-v2

Model Details

Model Description

Model Sources

Full Model Architecture

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

json

Evaluation Dataset

json

Training Hyperparameters

Non-Default Hyperparameters

All Hyperparameters

Training Logs

Framework Versions

Citation

BibTeX

Sentence Transformers

CoSENTLoss

hassaangyt/
compliance-emb-v1