SentenceTransformer based on microsoft/mdeberta-v3-base

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: microsoft/mdeberta-v3-base
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: DebertaV2Model 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Information Retrieval

• Dataset: eval
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 110,575 training samples
• Columns: sentence_0, sentence_1, and label
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	label
  type	string	string	int
  details	min: 3 tokens mean: 6.46 tokens max: 19 tokens	min: 27 tokens mean: 394.71 tokens max: 512 tokens	1: 100.00%

• Samples:

  sentence_0	sentence_1	label
  chronic diseases	Role of antioxidants in cancer therapy. Oxidative stress is a key component in linking environmental toxicity to the multistage carcinogenic process. Reactive oxygen species (ROS) are generated in response to both endogenous and exogenous stimuli. To counterbalance ROS-mediated injury, an endogenous antioxidants defense system exists; however, when oxidation exceeds the control mechanisms, oxidative stress arises. Chronic and cumulative oxidative stress induces deleterious modifications to a variety of macromolecular components, such as DNA, lipids, and proteins. A primary mechanism of many chemotherapy drugs against cancer cells is the formation of ROS, or free radicals. Radiotherapy is based on the fact that ionizing radiation destroys tumor cells. Radiotherapy induces direct lesions in the DNA or biological molecules, which eventually affect DNA. Free radicals produced by oncology therapy are often a source of serious side effects as well. The objective of this review is to provide information about the effects of antioxidants during oncology treatments and to discuss the possible events and efficacy. Much debate has arisen about whether antioxidant supplementation alters the efficacy of cancer chemotherapy. There is still limited evidence in both quality and sample size, suggesting that certain antioxidant supplements may reduce adverse reactions and toxicities. Significant reductions in toxicity may alleviate dose-limiting toxicities so that more patients are able to complete prescribed chemotherapy regimens and thus, in turn, improve the potential for success in terms of tumor response and survival. Copyright © 2013 Elsevier Inc. All rights reserved.	1
  plant-based diets	Diet, infection and wheezy illness: lessons from adults. An increase in asthma and atopic disease has been recorded in many countries where society has become more prosperous. We have investigated two possible explanations: a reduction in childhood infections and a change in diet. In a cohort of people followed up since 1964, originally selected as a random sample of primary school children, we have investigated the relevance of family size and the common childhood infectious diseases to development of eczema, hay fever and asthma. Although membership of a large family reduced risks of hay fever and eczema (but not asthma), this was not explained by the infections the child had suffered. Indeed, the more infections the child had had, the greater the likelihood of asthma, although measles gave a modest measure of protection. We have investigated dietary factors in two separate studies. In the first, we have shown the risks of bronchial hyper-reactivity are increased seven-fold among those with the lowest intake of vitamin C, while the lowest intake of saturated fats gave a 10-fold protection. In the second, we have shown that the risk of adult-onset wheezy illness is increased five-fold by the lowest intake of vitamin E and doubled by the lowest intake of vitamin C. These results were supported by direct measurements of the vitamins and triglycerides in plasma. We have proposed that changes in the diet of pregnant women may have reflected those observed in the population as a whole and that these may have resulted in the birth of cohorts of children predisposed to atopy and asthma. The direct test of this is to study the diet and nutritional status of a large cohort of pregnant women and to follow their offspring forward. This is our current research.	1
  liver health	Effect of a very-high-fiber vegetable, fruit, and nut diet on serum lipids and colonic function. We tested the effects of feeding a diet very high in fiber from fruit and vegetables. The levels fed were those, which had originally inspired the dietary fiber hypothesis related to colon cancer and heart disease prevention and also may have been eaten early in human evolution. Ten healthy volunteers each took 3 metabolic diets of 2 weeks duration. The diets were: high-vegetable, fruit, and nut (very-high-fiber, 55 g/1,000 kcal); starch-based containing cereals and legumes (early agricultural diet); or low-fat (contemporary therapeutic diet). All diets were intended to be weight-maintaining (mean intake, 2,577 kcal/d). Compared with the starch-based and low-fat diets, the high-fiber vegetable diet resulted in the largest reduction in low-density lipoprotein (LDL) cholesterol (33% +/- 4%, P <.001) and the greatest fecal bile acid output (1.13 +/- 0.30 g/d, P =.002), fecal bulk (906 +/- 130 g/d, P <.001), and fecal short-chain fatty acid outputs (78 +/- 13 mmol/d, P <.001). Nevertheless, due to the increase in fecal bulk, the actual concentrations of fecal bile acids were lowest on the vegetable diet (1.2 mg/g wet weight, P =.002). Maximum lipid reductions occurred within 1 week. Urinary mevalonic acid excretion increased (P =.036) on the high-vegetable diet reflecting large fecal steroid losses. We conclude that very high-vegetable fiber intakes reduce risk factors for cardiovascular disease and possibly colon cancer. Vegetable and fruit fibers therefore warrant further detailed investigation. Copyright 2001 by W.B. Saunders Company	1

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 32
• fp16: True
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.1.1
• Transformers: 4.44.2
• PyTorch: 2.4.1+cu121
• Accelerate: 0.34.2
• Datasets: 3.0.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss