SentenceTransformer based on google/embeddinggemma-300m

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: google/embeddinggemma-300m
• Maximum Sequence Length: 2048 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 2048, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 3072, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (3): Dense({'in_features': 3072, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity'})
  (4): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 200 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 200 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 11 tokens mean: 81.76 tokens max: 806 tokens	min: 11 tokens mean: 80.08 tokens max: 806 tokens	min: 12 tokens mean: 79.01 tokens max: 797 tokens

• Samples:

  anchor	positive	negative
  $(6?3) + 4 - (2 - 1) = 5.$ To make this statement true, the question mark between the 6 and the 3 should be replaced by	What is the degree measure of the smaller angle formed by the hands of a clock at 10 o'clock?	An insect lives on the surface of a regular tetrahedron with edges of length 1. It wishes to travel on the surface of the tetrahedron from the midpoint of one edge to the midpoint of the opposite edge. What is the length of the shortest such trip? (Note: Two edges of a tetrahedron are opposite if they have no common endpoint.)
  What is the degree measure of the smaller angle formed by the hands of a clock at 10 o'clock?	Which triplet of numbers has a sum NOT equal to 1?	Corners are sliced off a unit cube so that the six faces each become regular octagons. What is the total volume of the removed tetrahedra?
  Which triplet of numbers has a sum NOT equal to 1?	What is the degree measure of the smaller angle formed by the hands of a clock at 10 o'clock?	How many pairs of positive integers $(a,b)$ are there such that $\text{gcd}(a,b)=1$ and $\frac{a}{b} + \frac{14b}{9a}$ is an integer? $\mathrm {(A)}\ 4\quad\mathrm {(B)}\ 6\quad\mathrm {(C)}\ 9\quad\mathrm {(D)}\ 12\quad\mathrm {(E)}\ \text{infinitely many}$

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 1
• learning_rate: 2e-05
• num_train_epochs: 5
• warmup_ratio: 0.1
• prompts: task: classification | query:

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.12.12
• Sentence Transformers: 5.1.2
• Transformers: 4.57.2
• PyTorch: 2.9.0+cu126
• Accelerate: 1.12.0
• Datasets: 4.0.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss