SentenceTransformer based on llm-semantic-router/mmbert-embed-32k-2d-matryoshka

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: llm-semantic-router/mmbert-embed-32k-2d-matryoshka
• Maximum Sequence Length: 32768 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 492 training samples
• Columns: sentence_0, sentence_1, and sentence_2
• Approximate statistics based on the first 492 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  details	min: 36 tokens mean: 219.2 tokens max: 791 tokens	min: 49 tokens mean: 227.77 tokens max: 819 tokens	min: 51 tokens mean: 159.34 tokens max: 430 tokens

• Samples:

  sentence_0	sentence_1	sentence_2
  A 48-year-old woman presents to the emergency department because of increasingly severe right upper abdominal pain, fever, and non-bloody vomiting for the last 5 hours. The pain is dull, intermittent, and radiates to her right shoulder. During the past 3 months, she has had recurring abdominal discomfort after meals. The patient underwent an appendectomy more than 30 years ago. She has hypertension, diabetes mellitus type 2, and chronic back pain. She takes bisoprolol, metformin, and ibuprofen daily. She is 171 cm (5 ft 6 in) tall and weighs 99 kg (218 lb). Her BMI is 35.2 kg/m2. She appears uncomfortable and is clutching her abdomen. Her temperature is 38.5°C (101.3°F), pulse is 108/min, and blood pressure is 150/82 mm Hg. Abdominal examination shows right upper quadrant abdominal tenderness and guarding. Upon deep palpation of the right upper quadrant, the patient pauses during inspiration. Laboratory studies show the following:[object Object]Blood[object Object]Hemoglobin 13.1 g/dL[object Object]Leukocyte count 10,900/mm3[object Object]P...	A 48-year-old woman presents to the emergency department because of increasingly severe right upper abdominal pain, fever, and non-bloody vomiting for the last 5 hours. The pain is dull, intermittent, and radiates to her right shoulder. During the past 3 months, she has had recurring abdominal discomfort after meals. The patient underwent an appendectomy more than 30 years ago. She has hypertension, diabetes mellitus type 2, and chronic back pain. She takes bisoprolol, metformin, and ibuprofen daily. She is 171 cm (5 ft 6 in) tall and weighs 99 kg (218 lb). Her BMI is 35.2 kg/m2. She appears uncomfortable and is clutching her abdomen. Her temperature is 38.5°C (101.3°F), pulse is 108/min, and blood pressure is 150/82 mm Hg. Abdominal examination shows right upper quadrant abdominal tenderness and guarding. Upon deep palpation of the right upper quadrant, the patient pauses during inspiration. Laboratory studies show the following:[object Object]Blood[object Object]Hemoglobin 13.1 g/dL[object Object]Leukocyte count 10,900/mm3[object Object]P...	A 27-year-old woman presents with acute abdominal pain in her right upper quadrant. The pain came on suddenly while she was eating dinner. After this pain she began feeling dizzy and came to the emergency department. In the ED, her blood pressure is 75/40 mmHg, pulse is 100/minute, and she is afebrile. On physical exam, she feels too light-headed to ambulate. She demonstrates normal bowel sounds with tenderness upon palpation in the right upper quadrant. The patient is deemed too unstable for imaging. An abdominal radiograph and CT are reviewed from a recent previous visit to the ED for mild abdominal pain, and are shown in Figures A and B, respectively. Which of the following specific additional findings in her history supports the most likely diagnosis?[object Object][object Object]Answer: Use of oral contraceptives (OCPs) for birth control
  A 42-year-old man presents to his primary care provider for abdominal pain. He reports that for several months he has been experiencing a stabbing pain above the umbilicus during meals. He denies associated symptoms of nausea, vomiting, or diarrhea. The patient’s past medical history is significant for hypertension and hyperlipidemia for which he takes amlodipine and atorvastatin. His family history is significant for lung cancer in his father. The patient is a current smoker with a 20 pack-year smoking history and drinks 3-5 beers per week. Initial laboratory testing is as follows:[object Object][object Object]Serum:[object Object]Na+: 141 mEq/L[object Object]K+: 4.6 mEq/L[object Object]Cl-: 102 mEq/L[object Object]HCO3-: 25 mEq/L[object Object]Urea nitrogen: 14 mg/dL[object Object]Creatinine: 1.1 mg/dL[object Object]Glucose: 120 mg/dL[object Object]Calcium: 8.4 mg/dL[object Object]Alkaline phosphatase: 66 U/L[object Object]Aspartate aminotransferase (AST): 40 U/L[object Object]Alanine aminotransferase (ALT): 52 U/L[object Object]Gastrin: 96 pg/mL (<100 pg/mL)[object Object]Lipase: 90 U/L (<160 U/L)[object Object][object Object]The patient is started on a proton pump inhibitor without symptomatic improvement after 6 w...	A 42-year-old man presents to his primary care provider for abdominal pain. He reports that for several months he has been experiencing a stabbing pain above the umbilicus during meals. He denies associated symptoms of nausea, vomiting, or diarrhea. The patient’s past medical history is significant for hypertension and hyperlipidemia for which he takes amlodipine and atorvastatin. His family history is significant for lung cancer in his father. The patient is a current smoker with a 20 pack-year smoking history and drinks 3-5 beers per week. Initial laboratory testing is as follows:[object Object][object Object]Serum:[object Object]Na+: 141 mEq/L[object Object]K+: 4.6 mEq/L[object Object]Cl-: 102 mEq/L[object Object]HCO3-: 25 mEq/L[object Object]Urea nitrogen: 14 mg/dL[object Object]Creatinine: 1.1 mg/dL[object Object]Glucose: 120 mg/dL[object Object]Calcium: 8.4 mg/dL[object Object]Alkaline phosphatase: 66 U/L[object Object]Aspartate aminotransferase (AST): 40 U/L[object Object]Alanine aminotransferase (ALT): 52 U/L[object Object]Gastrin: 96 pg/mL (<100 pg/mL)[object Object]Lipase: 90 U/L (<160 U/L)[object Object][object Object]The patient is started on a proton pump inhibitor without symptomatic improvement after 6 w...	A 42-year-old man comes to his primary care physician complaining of abdominal pain. He describes intermittent, burning, epigastric pain over the past 4 months. He reports that the pain worsens following meals. He had an upper gastrointestinal endoscopy done 2 months ago that showed a gastric ulcer without evidence of malignancy. The patient was prescribed pantoprazole with minimal improvement in symptoms. He denies nausea, vomiting, diarrhea, or melena. The patient has no other medical problems. He had a total knee replacement 3 years ago following a motor vehicle accident for which he took naproxen for 2 months for pain management. He has smoked 1 pack per day since the age 22 and drinks 1-2 beers several nights a week with dinner. He works as a truck driver, and his diet consists of mostly of fast food. His family history is notable for hypertension in his paternal grandfather and coronary artery disease in his mother. On physical examination, the abdomen is soft, nondistended, and ...
  A 62-year-old woman comes to the physician in June for a routine check-up. She has chronic back pain and underwent an appendectomy at the age of 27. She is married and has two kids. The patient recently got back from a cruise to Mexico where she celebrated her 40th wedding anniversary. Her last mammogram was 6 months ago and showed no abnormalities. Her last Pap smear was 2 years ago and unremarkable. A colonoscopy 5 years ago was normal. Her mother died of breast cancer last year and her father has arterial hypertension. Her immunization records show that she has never received a pneumococcal or a shingles vaccine, her last tetanus booster was 6 years ago, and her last influenza vaccine was 2 years ago. She drinks 1– 2 alcoholic beverages every weekend. She takes a multivitamin daily and uses topical steroids. She regularly attends water aerobic classes and physical therapy for her back pain. She is 168 cm (5 ft 6 in) tall and weighs 72 kg (160 lb); BMI is 26 kg/m2. Her temperature is...	A 62-year-old woman comes to the physician in June for a routine check-up. She has chronic back pain and underwent an appendectomy at the age of 27. She is married and has two kids. The patient recently got back from a cruise to Mexico where she celebrated her 40th wedding anniversary. Her last mammogram was 6 months ago and showed no abnormalities. Her last Pap smear was 2 years ago and unremarkable. A colonoscopy 5 years ago was normal. Her mother died of breast cancer last year and her father has arterial hypertension. Her immunization records show that she has never received a pneumococcal or a shingles vaccine, her last tetanus booster was 6 years ago, and her last influenza vaccine was 2 years ago. She drinks 1– 2 alcoholic beverages every weekend. She takes a multivitamin daily and uses topical steroids. She regularly attends water aerobic classes and physical therapy for her back pain. She is 168 cm (5 ft 6 in) tall and weighs 72 kg (160 lb); BMI is 26 kg/m2. Her temperature is...	A 66-year-old G3P3 presents with an 8-year-history of back pain, perineal discomfort, difficulty urinating, recurrent malaise, and low-grade fevers. These symptoms have recurred regularly for the past 5–6 years. She also says that there are times when she experiences a feeling of having a foreign body in her vagina. With the onset of symptoms, she was evaluated by a physician who prescribed her medications after a thorough examination and recommended a vaginal pessary, but she was non-compliant. She had 3 vaginal deliveries She has been menopausal since 51 years of age. She does not have a history of malignancies or cardiovascular disease. She has type 2 diabetes mellitus that is controlled with diet and metformin. Her vital signs include: blood pressure 110/60 mm Hg, heart rate 91/min, respiratory rate 13/min, and temperature 37.4℃ (99.3℉). On physical examination, there is bilateral costovertebral angle tenderness. The urinary bladder is non-palpable. The gynecologic examination reve...

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Training Hyperparameters

Non-Default Hyperparameters

• num_train_epochs: 2
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Framework Versions

• Python: 3.12.10
• Sentence Transformers: 5.2.2
• Transformers: 5.0.0
• PyTorch: 2.7.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss