This is a sentence-transformers model finetuned from microsoft/mpnet-base on the json dataset. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: microsoft/mpnet-base
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- json

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'MPNetModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sahithkumar7/mpnet-base-finetuned-iter01")
# Run inference
sentences = [
    'What was the most frequently identified pharmaceutical in the groundwater samples?',
    'from one to five compounds. The most frequently identified pharmaceuticals, in decreasing order, were ciprofloxacin 43%\n(3/7), enrofloxacin, norfloxacin, trimethoprim, lincomycin (29% (2/7), abacavir and tetracycline 14% (1/7). The enzyme\ninhibitors, namely clavulanic acid and cilastatin, were detected once in an urban region located well. This catchment point\nshowed the most significant number of pharmaceuticals. West/Tejo and Centre were the regions with the most\nconsiderable number of substances in groundwater, accounting for 43%. All groundwater samples were contaminated by',
    'Pharmacokinetic characteristics may represent key features in understanding antibiotics occurrence [62]. Most antibiotics\nare not completely metabolised in humans and animals; thus, a high percentage of the active substance (40-90%) is\nexcreted in urine/faeces in the unchanged form. These molecules are discharged into water and soil through wastewater,\nanimal manure, and sewage sludge, frequently used as fertilisers to agricultural lands. Also, it is expected that the\nhospital effluent will contribute partly to the pharmaceutical load in the wastewater treatment plant influence [63].',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.4988, -0.0391],
#         [ 0.4988,  1.0000,  0.0047],
#         [-0.0391,  0.0047,  1.0000]])

Evaluation

Metrics

Triplet

Datasets: initial_test, final_test, final_test, final_test, final_test, final_test, final_test, final_test, final_test, final_test, final_test and final_test
Evaluated with TripletEvaluator

Metric	initial_test	final_test
cosine_accuracy	0.96	0.98

Training Details

Training Dataset

json

Dataset: json
Size: 80 training samples
Columns: anchor, positive, and negative
Approximate statistics based on the first 80 samples:
anchor positive negative
type string string string
details
min: 9 tokens
mean: 16.14 tokens
max: 33 tokens
min: 48 tokens
mean: 125.65 tokens
max: 218 tokens
min: 48 tokens
mean: 122.97 tokens
max: 211 tokens

Samples:

anchor	positive	negative
`Which two macrolide antibiotics are frequently detected in surface water samples?`	seems to undertake a similar fate in the environment.[object Object]Nevertheless, due to stronger adsorption, with higher emergence in sediment, its occurrence in the surface water is lower[object Object][71]. The use of tetracyclines, mainly as medicated premix and oral solution for food-producing animals [72], and the very[object Object]low bioavailability (e.g. in pig feed) [43] contribute to increasing its release into the environment. Regarding macrolides,[object Object]erythromycin and clarithromycin exhibit a remarkable frequency of detection in surface water samples. The most	Nonetheless, besides the sorption capacity, these antibiotics have high solubility in water. Crucial routes for these[object Object]substances into the environment are manure from animal production and sewage sludge from wastewater treatment[object Object]plant (WWTP) used as fertilisers. Therefore, these substances have been evidenced in topsoil samples [68]. These[object Object]quinolones and other antibiotics, for instance, norfloxacin and tetracycline, have been identified in groundwater samples[object Object]despite being influenced by sorption processes. They were not readily degraded; instead, the input into groundwater
`What antimicrobial drugs were identified in the survey besides macrolides?`	is one of the most frequently pharmaceutical in representative rivers [74,75]. The three macrolides identified in our[object Object]detection survey are included since 2018 in the first 'watch list' [76].[object Object]Another group of antimicrobial drugs identified in our survey were sulfamethoxazole/trimethoprim and sulfamethazine.[object Object]Sulfamethoxazole/trimethoprim are often used combined since the effectiveness of sulfonamides is enhanced. In the[object Object]present study, the detection of both substances was comparable; however, trimethoprim was detected in groundwater.	upstream samples obtained in rural locations was demonstrated and could be attributed to a low efficiency in the urban[object Object]wastewater treatment plants or due to agricultural pressure.[object Object]The higher frequency of detection for most substances was observed in the Ave river and Ria Formosa, confirming that[object Object]several effluents impact these water bodies from urban wastewater treatment plants and livestock production.[object Object]Pharmacokinetic characteristics may represent key features in understanding antibiotics occurrence [62]. Most antibiotics
`How long was the observational period of the antibiotic survey in Portugal?`	of antibiotics and their metabolites in surface- groundwater. It seeks to reflect the current demographic, spatial, drug[object Object]consumption, and drug profile on an observational period of 3 years in Portugal. The greatest challenge of this survey[object Object]data will be to promote the ecopharmacovigilance framework development shortly to implement measures for avoiding[object Object]misuse/overuse of antibiotics and slow down emission and antibiotic resistance.[object Object]2. Results[object Object]2.1. Frequency of Detections:[object Object]Antibiotics/Enzyme-Inhibitors and Abacavir[object Object]in Surface-Groundwater	despite being influenced by sorption processes. They were not readily degraded; instead, the input into groundwater[object Object]could be due to livestock farming pressure, namely by spreading manure in the soil or the possible sewage sludge[object Object]application in the area. High clay and low sand content in soils can decrease the mobility of pharmaceuticals, which is[object Object]attributed to clay intense exchange capacity. Thus, soil properties (e.g. particle composition) are a significant, influential

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Evaluation Dataset

json

Dataset: json
Size: 20 evaluation samples
Columns: anchor, positive, and negative
Approximate statistics based on the first 20 samples:
anchor positive negative
type string string string
details
min: 11 tokens
mean: 16.4 tokens
max: 25 tokens
min: 76 tokens
mean: 113.65 tokens
max: 148 tokens
min: 89 tokens
mean: 118.8 tokens
max: 162 tokens

Samples:

anchor	positive	negative
`What percentage of unchanged excretion did the most significant number of detected substances show?`	coefficients were not available for lincomycin, clavulanic acid and cilastatin.[object Object]Physicochemical properties of detected pharmaceuticals.[object Object]1 Data retrieved from [16]; 2 Data retrieved from [17]; 3 Data retrieved from [18]; 4 Data retrieved from [19]; 5[object Object]Data retrieved from [20];[object Object]6 Data retrieved from [21]; 7 Data retrieved from [22]; 8 Data retrieved from [23]; 9 Data retrieved from [24]; 10[object Object]Data retrieved from [25];[object Object]NA-not available.[object Object]The most significant number of detected substances showed a percentage of unchanged excretion higher than 40%.	1. Introduction[object Object]Antibiotics are a critical component of human and veterinary modern medicine, developed to produce desirable or[object Object]beneficial effects on infections induced by pathogens. Like most pharmaceuticals, antibiotics tend to be small organic[object Object]polar compounds, generally ionisable, ordinarily subject to a metabolism or biotransformation process by the organism to[object Object]be eliminated more efficiently [1,2]. The excretion of these compounds and their metabolites occurs mainly through urine,
`How many kilograms of abacavir were detected in Portugal in 2017?`	Regarding the different regions, it has been concluded that North and West/Tejo were the regions with the higher[object Object]consuming values. Both regions presented a significant value (33%) for the abacavir. For the detected antiviral abacavir,[object Object]an amount of 1458 kg has been observed.[object Object]Regarding antibiotics used in veterinary medicine, the regional amount was not available. Likewise, due to the reported[object Object]missing quantity for sulfamethazine, the sulfonamides group has been matched.[object Object]Consumption (Kg) of the detected pharmaceuticals in Portugal (2017).	43%[object Object](3/7), enrofloxacin, norfloxacin, trimethoprim, lincomycin (29% (2/7), abacavir and tetracycline[object Object]14% (1/7). The enzyme inhibitors, namely clavulanic acid and cilastatin, were detected once in an urban region located[object Object]well. This catchment point showed the most significant[object Object]number of pharmaceuticals. West/Tejo and Centre were the regions with the most considerable number of substances in[object Object]groundwater, accounting for 43%. All groundwater[object Object]samples were contaminated by at least one antibiotic. Supplemental Tables S2 and S4 contain a detailed description of[object Object]the
`What must marketing authorisation procedures for medicines include since 2006?`	substances in passive samplers [7]. Since 2006, marketing authorisation procedures for both human and veterinary[object Object]medicines must include an environmental risk assessment that comprises a prospective exposure assessment,[object Object]underestimating the possible impact and the occurrence of antibiotics after years of consumption. Ultimately, the potential[object Object]risk may not be correctly anticipated. It becomes urgent to generate new data, mainly to refine exposure assessments.[object Object]As much as the specificities of each member state should be considered this issue has become one of the European	clarithromycin/erythromycin, tetracycline, sulfamethoxazole, and abacavir. In groundwater, enrofloxacin/ciprofloxacin,[object Object]norfloxacin, trimethoprim, lincomycin, abacavir and tetracycline were recovered. Metabolites were not detected in water[object Object]bodies. Noticeable was the detection of enzyme inhibitors, tazobactam and cilastatin, which are both for exclusive[object Object]hospital use. The North region and Algarve (South) were the areas with the most significant frequency of substances in[object Object]surface water. The relatively higher detection of substances downstream of the effluent discharge points compared with a

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
num_train_epochs: 1
warmup_ratio: 0.1
fp16: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Click to expand

Epoch	Step	Training Loss	Validation Loss	initial_test_cosine_accuracy	final_test_cosine_accuracy
-1	-1	-	-	0.7800	-
0.2	1	3.3315	-	-	-
0.4	2	3.0922	-	-	-
0.6	3	3.2635	-	-	-
0.8	4	3.0702	-	-	-
1.0	5	3.3282	-	-	-
-1	-1	-	-	-	0.6800
0.2	1	2.9487	-	-	-
0.4	2	2.9845	-	-	-
0.6	3	2.935	-	-	-
0.8	4	3.0702	-	-	-
1.0	5	3.0039	-	-	-
1.2	6	2.4806	-	-	-
1.4	7	2.2646	-	-	-
1.6	8	1.8101	-	-	-
1.8	9	1.3463	-	-	-
2.0	10	1.942	-	-	-
-1	-1	-	-	-	0.9000
0.2	1	2.2356	-	-	-
0.4	2	1.0123	-	-	-
0.6	3	1.2411	-	-	-
0.8	4	0.9194	-	-	-
1.0	5	0.891	-	-	-
1.2	6	0.602	-	-	-
1.4	7	0.5426	-	-	-
1.6	8	0.5738	-	-	-
1.8	9	0.2678	-	-	-
2.0	10	0.7113	-	-	-
2.2	11	0.2911	-	-	-
2.4	12	0.4745	-	-	-
2.6	13	0.4188	-	-	-
2.8	14	0.3708	-	-	-
3.0	15	0.2882	-	-	-
-1	-1	-	-	-	0.9200
0.2	1	0.5156	-	-	-
0.4	2	0.0749	-	-	-
0.6	3	0.0634	-	-	-
0.8	4	0.0534	-	-	-
1.0	5	0.019	-	-	-
1.2	6	0.0682	-	-	-
1.4	7	0.0381	-	-	-
1.6	8	0.171	-	-	-
1.8	9	0.1188	-	-	-
2.0	10	0.1861	-	-	-
2.2	11	0.0895	-	-	-
2.4	12	0.2492	-	-	-
2.6	13	0.0964	-	-	-
2.8	14	0.2424	-	-	-
3.0	15	0.1096	-	-	-
3.2	16	0.1981	-	-	-
3.4	17	0.1438	-	-	-
3.6	18	0.3454	-	-	-
3.8	19	0.4011	-	-	-
4.0	20	0.1591	0.5567	0.9400	-
-1	-1	-	-	-	0.9400
0.125	1	0.0594	-	-	-
0.25	2	0.0584	-	-	-
0.375	3	0.0146	-	-	-
0.5	4	0.0542	-	-	-
0.625	5	0.0965	-	-	-
0.75	6	0.2209	-	-	-
0.875	7	0.0312	-	-	-
1.0	8	0.1142	-	-	-
-1	-1	-	-	-	0.9600
0.125	1	0.0082	-	-	-
0.25	2	0.004	-	-	-
0.375	3	0.001	-	-	-
0.5	4	0.0118	-	-	-
0.625	5	0.0508	-	-	-
0.75	6	0.0816	-	-	-
0.875	7	0.0149	-	-	-
1.0	8	0.0163	-	-	-
1.125	9	0.038	-	-	-
1.25	10	0.0618	-	-	-
1.375	11	0.0097	-	-	-
1.5	12	0.0368	-	-	-
1.625	13	0.0212	-	-	-
1.75	14	0.0072	-	-	-
1.875	15	0.0037	-	-	-
2.0	16	0.128	-	-	-
-1	-1	-	-	-	0.9600
0.125	1	0.0012	-	-	-
0.25	2	0.0003	-	-	-
0.375	3	0.0008	-	-	-
0.5	4	0.0008	-	-	-
0.625	5	0.0013	-	-	-
0.75	6	0.0743	-	-	-
0.875	7	0.0024	-	-	-
1.0	8	0.001	-	-	-
1.125	9	0.0024	-	-	-
1.25	10	0.01	-	-	-
1.375	11	0.0009	-	-	-
1.5	12	0.1912	-	-	-
1.625	13	0.0024	-	-	-
1.75	14	0.002	-	-	-
1.875	15	0.0038	-	-	-
2.0	16	0.1492	-	-	-
2.125	17	0.004	-	-	-
2.25	18	0.0123	-	-	-
2.375	19	0.0348	-	-	-
2.5	20	0.0068	0.5351	0.9600	-
2.625	21	0.1679	-	-	-
2.75	22	0.0123	-	-	-
2.875	23	0.0934	-	-	-
3.0	24	0.0048	-	-	-
-1	-1	-	-	-	0.9600
0.2	1	0.0763	-	-	-
0.4	2	0.0119	-	-	-
0.6	3	0.0019	-	-	-
0.8	4	0.0034	-	-	-
1.0	5	0.001	-	-	-
-1	-1	-	-	-	0.9800

Framework Versions

Python: 3.11.13
Sentence Transformers: 5.0.0
Transformers: 4.52.4
PyTorch: 2.6.0+cu124
Accelerate: 1.8.1
Datasets: 3.6.0
Tokenizers: 0.21.2

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

bibtex

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

	anchor	positive	negative
type	string	string	string
details	min: 9 tokens mean: 16.14 tokens max: 33 tokens	min: 48 tokens mean: 125.65 tokens max: 218 tokens	min: 48 tokens mean: 122.97 tokens max: 211 tokens