SentenceTransformer based on thebajajra/RexBERT-base-embed-pf-v0.1

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: thebajajra/RexBERT-base-embed-pf-v0.1
• Maximum Sequence Length: 1024 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • nomic-embed-supervised-data

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False, 'architecture': 'ModernBertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

nomic-embed-supervised-data

• Dataset: nomic-embed-supervised-data at 13eef8a
• Size: 1,611,024 training samples
• Columns: query, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	query	positive	negative
  type	string	string	list
  details	min: 4 tokens mean: 37.16 tokens max: 1024 tokens	min: 5 tokens mean: 99.03 tokens max: 1024 tokens	min: 20 elements mean: 155.14 elements max: 209 elements

• Samples:

  query	positive	negative
  Japan's biggest daily newspaper, Asahi Shimbun, published an interview Sunday with Woody Allen, who, asked to sum up the 20 th century in a single word, replied, Disappointing.	Woody Allen described the 20th century as "disappointing'' in an interview with the Japaense newspaper, Asahi Shimbun.	['Billy Joel described the 20th century as "outlandish" in a recent interview with Japan's leading newspaper. ', "aside for the kids' education and--BOP--I have so much trouble with that the", 'An old man is jogging.', 'It was my favorite spot.', 'Two women are playing volleyball.', ...]
  Sedimentation is the term for when clumped solids sink to the bottom of the water.	Deposition occurs where the water motion slows.	['The law of superposition is best described by: in undisturbed layers of sedimentary rock, the lowest layers contain the older rocks.', "Review: I felt this film - throughout. I waas impressed with Russell Crowe's talent in developing his relationship with Lillie, such a typical Aussie blend of softly softly approach, a bit self depreciating and very persistent. Really loved the cinematography and direction. Pace was just right and the portrayals of nearly all characters was impressive.Gosh, didn't Russell's talent even in 1993 shine! .. and I have yet to see Gladiator. Question: is it a negative review?", 'Question: How many times did Chopin and Liszy perform together in public? (Answer: seven).', 'Fact 1: Birth may be followed by a period of parental care of the offspring. \nFact 2: Human birth is the rarest of all births.', 'acts. Id. at 963. The Lawson court stated that a "critical distinction” of Pylant from the facts before it in Lawson was that in Pylant, there was "not a factu...
  Volleyball involves techniques like jumping.	Volleyball Volleyball is a team sport in which two teams of six players are separated by a net . Each team tries to score points by grounding a ball on the other team 's court under organized rules . It has been a part of the official program of the Summer Olympic Games since 1964 . The complete rules are extensive . But simply , play proceeds as follows : a player on one of the teams begins a ` rally ' by serving the ball ( tossing or releasing it and then hitting it with a hand or arm ) , from behind the back boundary line of the court , over the net , and into the receiving team 's court . The receiving team must not let the ball be grounded within their court . The team may touch the ball up to 3 times but individual players may not touch the ball twice consecutively . Typically , the first two touches are used to set up for an attack , an attempt to direct the ball back over the net in such a way that the serving team is unable to prevent it from being grounded in their court . ...	['Volleyball jump serve The Volleyball jump Serve is a type of volleyball serve where the player increases the power and height of their serve by jumping into the hit . The Jump Serve itself was popularized by the brazilian national volleyball team in 1984 on the Olympics .', "Volleyball variations As volleyball is one of the world 's most popular team sports , second only to football ( soccer ) in the number of players , there are numerous variations of the basic rules . The rules have changed around the world since its creation in 1895 , as skills have developed , to make the game more suited for spectators , for learning or other special needs . Some variations have been included as a change in the international rules by Fédération Internationale de Volleyball ( FIVB ) , others have resulted in variations with specific names . The more notable variations include :", 'Volleyball (ball) A volleyball is a ball used to play indoor volleyball , beach volleyball , or other less common v...

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Evaluation Dataset

nomic-embed-supervised-data

• Dataset: nomic-embed-supervised-data at 13eef8a
• Size: 84,795 evaluation samples
• Columns: query, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	query	positive	negative
  type	string	string	list
  details	min: 4 tokens mean: 34.78 tokens max: 1024 tokens	min: 5 tokens mean: 91.63 tokens max: 1024 tokens	min: 15 elements mean: 151.78 elements max: 209 elements

• Samples:

  query	positive	negative
  I think people are upset that they're "copying" a mechanic. Which is ridiculous. Every platformer copied from Mario. That doesn't mean they weren't good games. And this says nothing of the end result of the flow of the game.	The people who say spiderman is copying another game. Are the same retarded little children who think any game that has drivable vehicles in it, is a gta rip off. [object Object][object Object]You can't take them serious.	['i unfollowed all of them earlier this year theres too many and all their tweets and instaposts made up damn near half my feed i said id just wait for the music and then iridesence came out and was pretty mid so ', 'There are five, FIVE fucking boxes of doughnuts by the printer in the office this morning and I am overtired and hungry and in the worst mood, and I’m sorry r/1200isplenty, tea is not a goddamn dessert substitute; I keep drinking it and not only am I not satiated, I have to pee every ten minutes, so I’m here guzzling tea and growling [object Object] under my breath and if I sound crazy, it’s because I am. \n\nEdit: down to one box. Just checked the calories for one doughnut—380. Three hundred. And eighty. Calories. For one doughnut. ', 'What a strange thing to call your cock.', "It doesn't. That's a very personal decision. You're not right or wrong on that. I was just replying because you were clear you don't want to raise the child as your own....
  duties of a medical assistant for resume	Include clinical skills in a health care resume. Those that apply to a medical assistant job may be: 1 Preparing patients, including taking of medical histories. 2 The ability to take vital signs. 3 Preparing medication and treatments. 4 Basic first aid, CPR and infection control knowledge. Assisting physicians with exams.	['Include clinical skills in a health care resume. Those that apply to a medical assistant job may be: 1 Preparing patients, including taking of medical histories. 2 The ability to take vital signs. 3 Preparing medication and treatments. Basic first aid, CPR and infection control knowledge.', 'Clinical Medical Assistant Resume. The position of a medical assistant is a key position in any hospital, clinic or a health care center. A medical assistant may perform a variety of duties, pertaining to administrative, front office or clinical duties. Here we will discuss the job profile of a clinical medical assistant. Also, there is a clinical medical assistant resume example provided below, for your further reference.', "Community Q&A. A medical assistant is a member of a health care team that undertakes administrative and clinical job duties. A health care resume must emphasize skills as well as education and job experience. Learn how to write a resume for a medical assistant job and yo...
  when did british rule end in south africa	History of South Africa British colonies: Cape Colony, Natal Colony, Transvaal Colony, and Orange River Colony. The country became a self-governing nation state within the British Empire, in 1934 following enactment of the Status of the Union Act. The dominion came to an end on 31 May 1961 as the consequence of a 1960 referendum, which legitimised the country becoming a sovereign state named Republic of South Africa. A republican constitution was adopted. From 1948–1994, South African politics were dominated by Afrikaner nationalism. Racial segregation and white minority rule known officially as apartheid, an Afrikaans word meaning "separateness”, came into existence in 1948 (under	["South Africa 1910. The Union was a dominion that included the former territories of the Cape, Transvaal and Natal colonies, as well as the Orange Free State republic. The Natives' Land Act of 1913 severely restricted the ownership of land by blacks; at that stage natives controlled only 7% of the country. The amount of land reserved for indigenous peoples was later marginally increased. In 1931, the union was fully sovereign from the United Kingdom with the passage of the Statute of Westminster, which abolished the last powers of the British Government on the country. In 1934, the South African Party and", 'History of South Africa conducted raids in Free State territories. Both sides adopted scorched-earth tactics, with large swathes of pasturage and cropland being destroyed. Faced with starvation, Moshoeshoe signed a peace treaty on 15 October 1858, though crucial boundary issues remained unresolved. War broke out again in 1865. After an unsuccessful appeal for aid from the British ...

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 256
• per_device_eval_batch_size: 128
• learning_rate: 2e-06
• num_train_epochs: 20
• warmup_ratio: 0.1
• bf16: True
• dataloader_num_workers: 10
• dataloader_prefetch_factor: 10

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.11.13
• Sentence Transformers: 5.1.2
• Transformers: 4.57.1
• PyTorch: 2.8.0+cu129
• Accelerate: 1.11.0
• Datasets: 4.3.0
• Tokenizers: 0.22.1

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss