SentenceTransformer based on abhinand/MedEmbed-large-v0.1

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: abhinand/MedEmbed-large-v0.1
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Triplet

• Dataset: val-triplets
• Evaluated with TripletEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 7,603 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 6 tokens mean: 29.42 tokens max: 256 tokens	min: 4 tokens mean: 9.38 tokens max: 25 tokens	min: 4 tokens mean: 8.89 tokens max: 21 tokens

• Samples:

  anchor	positive	negative
  O2: 94% 4Lnc.	The patient is on 4L of oxygen via nasal cannula	The patient’s oxygen saturation is 100% on room air
  The patient has received 500 mg of intravenously levofloxacin given at the outside hospital, and the patient has received intravenous vancomycin as well as ceftazidime in our Emergency Department.	The patient has received broad spectrum antibiotics.	The patient has pneumonia.
  Cardiac enzymes done at OSH showed CK 363, CK-MB 33, TropI 6.78.	The patient has cardiac ischemia.	The patient has normal cardiac perfusion.

• Loss: TripletLoss with these parameters:
  json

  1{ 2 "distance_metric": "TripletDistanceMetric.COSINE", 3 "triplet_margin": 0.3 4}

Evaluation Dataset

Unnamed Dataset

• Size: 938 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 938 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 6 tokens mean: 31.03 tokens max: 187 tokens	min: 4 tokens mean: 9.08 tokens max: 21 tokens	min: 4 tokens mean: 8.76 tokens max: 19 tokens

• Samples:

  anchor	positive	negative
  History of acute renal failure.	The patient has had kidney damage.	The patient has always had normal functioning kidneys.
  Of note, pt had recent workup for intermittent abd discomfort and bloating, CT abd showed cholelithiasis and endometrial thickening, due for endometrial biopsy with Gyn.	Patient has findings warranting biopsy on imaging	Patient has endometrial cancer
  She states she drinks about [[object Object]] glasses of wine per night, but also admits to drinking up to a full bottle of wine during the day when she is home alone.	the patient consumes alcohol	the patient denies alcohol use

• Loss: TripletLoss with these parameters:
  json

  1{ 2 "distance_metric": "TripletDistanceMetric.COSINE", 3 "triplet_margin": 0.3 4}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 32
• learning_rate: 2e-05
• weight_decay: 0.01
• num_train_epochs: 40.0
• warmup_ratio: 0.1
• load_best_model_at_end: True
• push_to_hub: True
• hub_model_id: youssefkhalil320/MedEmbed-large-v0.1-medical-nli_v1

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.10.19
• Sentence Transformers: 3.3.1
• Transformers: 4.44.2
• PyTorch: 2.8.0+cu128
• Accelerate: 1.12.0
• Datasets: 4.0.0
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

TripletLoss