SentenceTransformer based on google/embeddinggemma-300m

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • med_embed-training-triplets-v1

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Triplet

• Dataset: medembed-triplets-dev-100
• Evaluated with TripletEvaluator

Training Details

Training Dataset

med_embed-training-triplets-v1

• Dataset: med_embed-training-triplets-v1 at 0b344f0
• Size: 230,357 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 4 tokens mean: 10.65 tokens max: 25 tokens	min: 5 tokens mean: 35.86 tokens max: 191 tokens	min: 6 tokens mean: 36.06 tokens max: 144 tokens

• Samples:

  anchor	positive	negative
  MTHFR homozygous A223V mutation symptoms	The patient was admitted to our institution where MTHFR homozygous A223V mutation was identified. Folic acid intake was increased to 800 mcg/d, and no other coagulation tests were abnormal.	The patient had several symptoms of MM, including hypercalcemia, bone fractures, anemia, and renal insufficiency. A biopsy showed atypical clonal plasma cells with Cluster of Differentiation (CD)138 positive infiltration.
  Causes of spindle cell malignancy in the duodenal wall	Histological analysis revealed a spindle cell malignancy that was positive for CD21, CD23, and vimentin, but negative for CD20, CD34, CD35, CD117, DOG 1, and smooth muscle actin.	Based on immunohistochemical analysis of the tumor cells, the primary buttock tumor was diagnosed as a skeletal muscle metastasis of the primary small intestine gastrointestinal stromal tumor (GIST).
  What was the patient's main complaint during hospital admission?	This 27-year-old pregnant woman was admitted to the hospital at 36 weeks gestation with acute vision loss in her left eye and severe onset headache.	The patient was discharged the next day

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Evaluation Dataset

med_embed-training-triplets-v1

• Dataset: med_embed-training-triplets-v1 at 0b344f0
• Size: 100 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 100 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 5 tokens mean: 10.95 tokens max: 32 tokens	min: 7 tokens mean: 33.45 tokens max: 94 tokens	min: 8 tokens mean: 36.28 tokens max: 90 tokens

• Samples:

  anchor	positive	negative
  What was the initial presentation of the patient?	The 45-year-old female patient presented to the department with an enlarging lesion in her upper abdomen.	The patient was transferred to this hospital for further evaluation.
  giant omphalocele symptoms	The patient, a 9-year-old female, presented to the hospital with a large lump in the anterior abdominal wall extending from the xiphisternum to the level of iliac crest.	The patient presented with bilateral nasovestibular lumps which grew in size over several months, occluding nasal entrance and protruding outside the nose.
  granulomatous lymphocytic interstitial lung disease treatment	The patient had clubbing and chronic lung findings, and thorax CT revealed extended and severe bronchiectasis with thickened bronchial walls, some granulomatous nodules and mosaic appearance, compatible with granulomatous lymphocytic interstitial lung disease (GLILD). Regular intravenous immunoglobulin (IVIG) replacement was started.	The patient was treated with methylprednisolone pulse therapy followed by oral prednisolone (PSL) and cyclophosphamide intravenously. After treatment, arthralgia, renal function, proteinuria, and skin manifestations improved.

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 8
• learning_rate: 2e-05
• weight_decay: 0.01
• num_train_epochs: 1
• warmup_ratio: 0.1
• dataloader_num_workers: 4
• load_best_model_at_end: True
• ddp_find_unused_parameters: False
• batch_sampler: no_duplicates

All Hyperparameters

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 5.1.1
• Transformers: 4.57.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.9.0
• Datasets: 4.1.1
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss