This is a sentence-transformers model finetuned from google/embeddinggemma-300m. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/embeddinggemma-300m
Maximum Sequence Length: 256 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'Gemma3TextModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ahmedHamdi/IR-fr-en-gemma_masked")
# Run inference
sentences = [
    "Tuscany, 1984. Mario and GPE find themselves stuck at a level crossing, and the two friends confide in each other about their lives. GPE, among other things, is worried about his sister PERSON, who has fallen into depression because of her relationship with an American. As the wait drags on, the two men decide to take a detour and end up stranded in the middle of the countryside. They take refuge in an inn for the night and wake up the next morning in a Tuscan village called PERSON, sometime between 1400 and 1500. Initially believing it to be a big joke, the two men find themselves having to confront the reality of the era. They find shelter with PERSON, the brother of the PERSON's guest who died the day before. They begin working at the ORG owned by PERSON's mother, PERSON, and while GPE adapts immediately to his new lifestyle, PERSON refuses to adjust. Soon, PERSON, the daughter of a wealthy family, with whom he begins a relationship. Meanwhile, PERSON is arrested, and GPE tries to write a letter to ORG to secure his release. GPE makes no secret of his jealousy of ORG relationship and convinces his friend to set off for GPE to join PERSON.\nDuring their journey to GPE, they encounter a beautiful ORG, PERSON, who frightens them by shooting an arrow at their chariot. The story then divides into two parts, depending on whether it is presented in the television or film version.",
    'Tuscan countryside. The janitor PERSON and the teacher GPE are stopped at a level crossing, waiting for the train to pass. The two are friends and trust each other. Among other things, GPE is worried about his sister PERSON, who fell into depression for the failure of her relationship with an American boy. The wait continues and decide to travel a road in the fields. After a while they remain in cream with the car in the middle of the countryside. You do evening, it rains. The two find accommodation in an inn for the night, in a room that already hosts another person. The next morning as soon as they wake up they see, having fun, the guest urinating from the window, but their laughter are immediately truncated by the hiss of a spear that kills him. Mario and GPE see fonding people in the black cloak on horseback, rush to the ground floor and find other people, dressed in a very strange way. Increducts make themselves be said by a man where they find themselves and discover that they are fried, an imaginary Tuscan village, in 1492. considering it first a terrible joke, they must resign themselves to the harsh reality making themselves hosting from PERSON, the brother of the killed man, ORG, who tells them of a terrible feud with such a ORG, who is exterminating his family. Once in the village they know PERSON, a mother of PERSON and the late ORG and begin to work in their slaughter shop. In the ORG context of the village, the most disparate episodes happen. GPE immediately seems comfortable, while PERSON does not want to settle down; Soon, however, during a religious function, PERSON makes the acquaintance of Pia, a girl of a rich family, with whom he begins to see himself looking out from the surrounding wall of her house. In the meantime, PERSON is arrested and GPE and PERSON write a letter to ORG in vain to obtain his liberation. GPE (who continues to tell PERSON to ask PERSON if he has an friend to introduce him) does not hide a certain annoyance for the meetings between PERSON and the young PERSON and complains that he always remains and only him to work in the butcher shop. Driven by his political-intellectual ardor, he convinces his friend to travel to GPE, in order to reach PERSON and dissuade him from leaving for the ORG and discovering GPE, so that, in the future, his sister cannot meet the American boy who left her. In an unspecified place the two come across a beautiful ORG, PERSON, which intimidates them by throwing an arrow against their wagon. At this point the story differs depending on the version, standard or extended.',
    "World-famous symphony conductor Sir PERSON returns from a visit to his native GPE and discovers that his rich brother-in-law, August Henshler, having misunderstood PERSON's casual instruction to watch over his much younger wife PERSON while he was away, hired a private detective named ORG to follow her. PERSON is livid, and when August hands him the detective's report he tears it to pieces. Despite himself, the idea of infidelity on ORG's part hounds PERSON's mind, and he acquires ORG's address from August, ostensibly so he can destroy any copies of the report which ORG may have kept on file. ORG summarizes the report for PERSON: his wife was spied late at night wearing only a nightgown as she went to the hotel room of PERSON's secretary, PERSON, a man closer in age to her own, where she stayed for thirty-eight minutes. Distressed by the news, PERSON quarrels with PERSON before proceeding to his concert, where he conducts three pieces of classical music, envisioning scenarios appropriate to each one: a perfect crime scenario in which he murders his wife with a straight razor and frames PERSON (to the ORG to PERSON's Semiramide), accepting the situation and giving PERSON a generous check and his blessing (to the Prelude to ORG's Tannhäuser), and a game of Russian roulette with a blubbering PERSON and PERSON, that ends in PERSON's death with the first trigger pull (to GPE's Francesca da Rimini). After the concert, PERSON tries to stage his murder fantasy, but makes a mess of their apartment while looking for the recording device which he fantasized using to lure PERSON to the crime scene, then accidentally records at the wrong speed. Daphne returns home and questions her husband about his ill temperament, but devotedly bandages his thumb when he cuts it testing the sharpness of his razor. He next tries the forgiveness fantasy, but spills ink all over his checkbook. Russian roulette also fizzles out when he cannot remember where he left the bullets. Without revealing his suspicions, he asks if she has ever been to PERSON's room. She admits it, saying she was searching for her sister, PERSON, August's wife, suspecting she was having an affair with PERSON; she became trapped there when she saw ORG spying on the room, fearful that if he saw her walk out she would be suspected of having an affair. Alfred begs PERSON's forgiveness for his irrational behavior, which she gladly gives, ascribing it to the creative temperament of a great artist.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000,  0.5453, -0.0518],
#         [ 0.5453,  1.0000,  0.0562],
#         [-0.0518,  0.0562,  1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 16,276 training samples
Columns: sentence_0 and sentence_1
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1
type string string
details
min: 8 tokens
mean: 122.12 tokens
max: 256 tokens
min: 13 tokens
mean: 226.96 tokens
max: 256 tokens

Samples:

sentence_0	sentence_1
`General Overview[object Object]The story takes place in GPE, a fantasy world inhabited by humans and dragons. PERSON, a teenager searching for his identity, flees his castle and wanders the countryside after killing his father. He will meet LOC, the young ORG, and a fearsome sorcerer. Arren gradually discovers how to overcome his fears and find his own voice.`	As a war galley sails through a storm, two dragons fight above the clouds, ending in one's death. In the kingdom of GPE, the royal wizard ORG proclaims the dragons to be a sign of a loss of balance in the world. The king is dealing with disease sweeping across his kingdom, and the disappearance of his son, PERSON. In the middle of the night, ORG kills his father, steals his father's sword, and flees the castle. Arren travels through the desert and is rescued from wolves by the archmage Sparrowhawk. Together they travel to the city of LOC. When ORG explores the town alone, he rescues a young girl named ORG from slavers, but is later captured by the same slave master, GPE. His sword is dumped in the sea. Sparrowhawk rescues Arren from the slave caravan and takes him to a farm run by ORG's oldest and greatest friend PERSON, who lives with ORG. ORG's intervention against ORG's slave caravan angers Lord Cob, a powerful warlock and the ruler of Hort Town, who wants the archmage brought to th...
`Inspired by the true story of ORG (PERSON), Fighter in the PERSON tells the story of how a Korean immigrant became a legend in GPE by creating a new martial art: Taekkyon.`	At the tail-end of World War II, PERSON is a young Korean man who longs to be able to fly fighter planes. Stowing away to GPE in order to join their air force, Bae-dal's first experience of the country is when a con-man tries to steal his money. Bae-dal discovers that the man is a fellow Korean called ORG (PERSON), who has survived the harsh treatment of Koreans in GPE by turning to petty crime. With their different motives: Bae-dal driven by desire for action and Chun-bae needing to escape from some gangsters, the two Koreans stow away in a truck to the air force training camp. The commander in charge of the camp is a pompous imperialist called PERSON (PERSON). Having mistreated the two Koreans, he is amused by Bae-dal's fighting spirit and says that if Bae-dal can beat him with his inferior foreign fighting style (Taekkyon), he will release them. The two men fight with PERSON easily defeating Bae-dal, but an American attack on the airforce base allows Bae-dal and PERSON to escape. La...
`Ten-year-old PERSON is thrilled when his grandfather moves into his house. He's less than thrilled when he has to give up his bedroom and move into the attic. With the help of his friends, PERSON decides to declare war on his grandfather, who is much smarter than he looks...`	After accidentally stealing from a grocery store due to having trouble with the self-checkouts and causing a scene with the store manager, recently widowed PERSON (PERSON) is visited by his daughter PERSON (PERSON) who wants him to move in with her family. PERSON does not want to leave his house because he built it himself. PERSON convinces PERSON to move in with her and gives him her son PERSON's (PERSON) bedroom. PERSON is not happy about giving his room to his grandfather and being moved to the attic. PERSON is welcomed by PERSON's husband PERSON (PERSON) and two daughters, PERSON (PERSON) and PERSON (Poppy Gagnon). During his first day, PERSON spends most of his time in his new room, sitting in his chair and looking at the sky while still thinking about his late wife. PERSON then tells his friends PERSON (PERSON), PERSON (PERSON) and PERSON (PERSON) about his grandfather moving in with his family and living in his room. After a miserable first night in his new room, PERSON decides ...

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 4
per_device_eval_batch_size: 4
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 4
per_device_eval_batch_size: 4
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 3
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.1229	500	0.0396
0.2458	1000	0.0699
0.3686	1500	0.1463
0.4915	2000	0.1551
0.6144	2500	0.1705
0.7373	3000	0.1281
0.8602	3500	0.1261
0.9830	4000	0.1362
1.1059	4500	0.09
1.2288	5000	0.0709
1.3517	5500	0.1023
1.4746	6000	0.0672
1.5974	6500	0.0847
1.7203	7000	0.0734
1.8432	7500	0.0687
1.9661	8000	0.0602
2.0890	8500	0.0431
2.2118	9000	0.0298
2.3347	9500	0.0317
2.4576	10000	0.0218
2.5805	10500	0.0178
2.7034	11000	0.0211
2.8262	11500	0.0171
2.9491	12000	0.0227

Framework Versions

Python: 3.9.18
Sentence Transformers: 5.1.2
Transformers: 4.57.6
PyTorch: 2.8.0+cu128
Accelerate: 1.10.1
Datasets: 4.5.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss

bibtex

@misc{henderson2017efficient,
    title={Efficient Natural Language Response Suggestion for Smart Reply},
    author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
    year={2017},
    eprint={1705.00652},
    archivePrefix={arXiv},
    primaryClass={cs.CL}
}

	sentence_0	sentence_1
type	string	string
details	min: 8 tokens mean: 122.12 tokens max: 256 tokens	min: 13 tokens mean: 226.96 tokens max: 256 tokens