SentenceTransformer based on ibm-granite/granite-embedding-107m-multilingual

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: ibm-granite/granite-embedding-107m-multilingual
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 34,441 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 3 tokens mean: 99.93 tokens max: 255 tokens	min: 14 tokens mean: 245.16 tokens max: 512 tokens

• Samples:

  anchor	positive
  March Lecture Solving using by Svensson1 In this following: We describe Multiplicative Hedge) • We this method to solve is these lecture are on of “Lecture 11 of in 2015” written and Simon Rodriguez and on by Kaul that the lecture previous we to use the majority method order to fairly general with days N experts as For t . , gives advice: 2. advice the expert, of and the decides 4. observes suffers was majority parameterized by ε “learning rate”), now as follows: • each i weight initialized 1. are trustworthy the ning.) each t: • Predict based on w(t) After observing the vector, i expert the lecture we case = following any sequence of i of WM mistake	Advanced Algorithms March 22, 2022 Lecture 9: Solving LPs using Multiplicative Weights Notes by Ola Svensson1 In this lecture we do the following: • We describe the Multiplicative Weight Update (actually Hedge) method. • We then use this method to solve covering LPs. • This is a very fast and simple (i.e., very attractive) method for solving these LPs approximately. These lecture notes are partly based on an updated version of “Lecture 11 of Topics in TCS, 2015” that were written by Vincent Eggerling and Simon Rodriguez and on the lecture notes by Shiva Kaul that we used in the last lecture. 1 Recall last lecture In the previous lecture, we saw how to use the weighted majority method in order to fairly smartly follow the advice of experts. Recall that the general game-setting with T days and N experts was as follows: For t = 1, . . . , T: 1. Each expert i ∈[N] gives some advice: UP or DOWN 2. Aggregator (you) predicts, based on the advice of the expert, UP or DOWN. 3. Adversary, with k...
  Last ε The same proof the For duration expert i ∈[N], of WM mistakes ε) · (# i’s mistakes) + O(log(N)/ε) 1Disclaimer: notes They not been and may typos,	Last lecture we analyzed the case when ε = 1/2. The same proof gives the following Theorem 1 For any sequence of outcomes, duration T, and expert i ∈[N], # of WM mistakes ≤2(1 + ε) · (# of i’s mistakes) + O(log(N)/ε) . 1Disclaimer: These notes were written as notes for the lecturer. They have not been peer-reviewed and may contain inconsistent notation, typos, and omit citations of relevant works. 1
  [Sketch] The proof done by potential function: for each = 1, . , 1, Φ(t) = i We lower potential the mistakes of i. We it in of our mistakes. The weight of expert down by a −ε) i does. As weight is 1, Φ(T +1) = +1) ≥w(T +1) = (1 −ε)# of . Every the experts was (since majority weights are (1 −ε). that the factor every time Φ(T −ε/2)# mistakes = N −ε/2)# , equality used that = was initialized with a weight above bounds give us (1 mistakes ≤N · (1 of . sides, allowing for randomized strategies In the exercises, you proved that are instances for weighted This overcome this we allow random instead of always making prediction the to create A is often general is often good the of adversaries. Allowing randomized leads to following with T t . . ,	Proof [Sketch] The proof was done by defining a potential function: for each t = 1, . . . , T + 1, let Φ(t) = X i∈[N] w(t) i . We now lower bound the “final” potential Φ(T +1) using the number of mistakes of i. We then upper bound it in terms of our number of mistakes. Lower bound: The weight of expert i goes down by a factor (1 −ε) for each mistake i does. As the initial weight of i is 1, Φ(T +1) = X j∈[N] w(T +1) j ≥w(T +1) i = (1 −ε)# of i’s mistakes . Upper bound: Every time WM errs, at least half the weight of the experts was wrong (since weighted majority was wrong). These weights are then decreased by (1 −ε). It follows that the potential goes down by at least a factor (1 −ε/2) every time WM errs. And so Φ(T +1) ≤Φ(1) · (1 −ε/2)# of WM mistakes = N · (1 −ε/2)# of WM mistakes , where for the equality we used that Φ(1) = N since each expert was initialized with a weight of 1. The above bounds give us (1 −ε)# of i’s mistakes ≤Φ(T +1) ≤N · (1 −ε/2)# of WM mistakes . Taking logs on b...

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Evaluation Dataset

Unnamed Dataset

• Size: 3,827 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 15 tokens mean: 174.64 tokens max: 266 tokens	min: 55 tokens mean: 432.79 tokens max: 512 tokens

• Samples:

  anchor	positive
  CHAPTER 39 Adrenocorticosteroids Adrenocortical Antagonists occurs. of /d of or in intermediate-, long-acting glucocorticoids greater growth-suppressing the steroid in larger than amounts, as cortisone hydrocortisone, which mineralocorticoid effects addition to glucocorticoid and fluid and loss of potassium. patients this a hypokalemic, and in blood pressure. hypoproteinemia, renal disease, liver disease, also occur. In patients with disease, small of may These by using non-salt-retaining and supplements. C. Suppression corticosteroids adrenal suppression occur. weeks the given appropriate at times dosage 24–48 hours) or stress ten-fold for or costeroid dosage be it slowly. If to reduction be slow levels. It take 2–12 to and cortisol may not to normal The suppression not treatment ACTH does time for normal function. the too receiving a certain disorder, the	CHAPTER 39 Adrenocorticosteroids & Adrenocortical Antagonists 707 hypertension also occurs. In dosages of 45 mg/m 2 /d or more of hydrocortisone or its equivalent, growth retardation occurs in children. Medium-, intermediate-, and long-acting glucocorticoids have greater growth-suppressing potency than the natural steroid at equivalent doses. When given in larger than physiologic amounts, steroids such as cortisone and hydrocortisone, which have mineralocorticoid effects in addition to glucocorticoid effects, cause some sodium and fluid retention and loss of potassium. In patients with normal cardiovas- cular and renal function, this leads to a hypokalemic, hypochloremic alkalosis and eventually to a rise in blood pressure. In patients with hypoproteinemia, renal disease, or liver disease, edema may also occur. In patients with heart disease, even small degrees of sodium retention may lead to heart failure. These effects can be minimized by using synthetic non-salt-retaining steroids, ...
  is a treatment not reduce the return function. dosage rapidly a certain the symptoms the in patients an disorder patients Cushing’s disease) symptoms with rapid symptoms include anorexia, vomit- ing, weight loss, postural reflect true glucocorticoid deficiency, occur in the normal or even plasma levels, sug- gesting glucocorticoids must carefully the hyperglycemia, sodium with edema hypertension, hypokalemia, peptic osteopo- rosis, and and intermittent alternate-day) can on this Even patients may of stress, surgical are or or acci- occur. B. with with peptic hypertension with failure, cer- as varicella tuberculosis, psycho- ses, osteoporosis, Glucocorticoid differ respect relative anti- inflammatory and mineralocorticoid of available ( Table and these factors should be in drug to used. ACTH Adrenocortical Steroids patients normal	is not a pituitary problem, and treatment with ACTH does not reduce the time required for the return of normal function. If the dosage is reduced too rapidly in patients receiving gluco- corticoids for a certain disorder, the symptoms of the disorder may reappear or increase in intensity. However, patients without an underlying disorder (eg, patients cured surgically of Cushing’s disease) also develop symptoms with rapid reductions in cortico- steroid levels. These symptoms include anorexia, nausea or vomit- ing, weight loss, lethargy, headache, fever, joint or muscle pain, and postural hypotension. Although many of these symptoms may reflect true glucocorticoid deficiency, they may also occur in the presence of normal or even elevated plasma cortisol levels, sug- gesting glucocorticoid dependence. Contraindications & Cautions A. Special Precautions Patients receiving glucocorticoids must be monitored carefully for the development of hyperglycemia, glycosuria, sodium retention with ede...
  ( Table and these should be taken in be A. ACTH ACTH used past production to However, when is able, ACTH therapeutic agent has abandoned. which claimed be effective than were due of of were dosage Dosage the regimen physician consider the disease, amount likely to required the effect, therapy. required for the dose to obtain initial the for needed effect be until a small or symptoms is When it is continuously plasma levels to ACTH, paren- preparation oral doses frequent The situation with respect use of inflammatory allergic The same total quantity few be effective many smaller slowly absorbed autoimmune involving organs aggressively, is as treatment. complexes macrophages, of predni- divided doses dosage is serious dosage can gradually large required prolonged time, after control When used manner, large amounts	available ( Table 39–1 ), and these factors should be taken into account in selecting the drug to be used. A. ACTH versus Adrenocortical Steroids In patients with normal adrenals, ACTH was used in the past to induce the endogenous production of cortisol to obtain similar effects. However, except when an increase in androgens is desir- able, the use of ACTH as a therapeutic agent has been abandoned. Instances in which ACTH was claimed to be more effective than glucocorticoids were probably due to the administration of smaller amounts of corticosteroids than were produced by the dosage of ACTH. B. Dosage In determining the dosage regimen to be used, the physician must consider the seriousness of the disease, the amount of drug likely to be required to obtain the desired effect, and the duration of therapy. In some diseases, the amount required for maintenance of the desired therapeutic effect is less than the dose needed to obtain the initial effect, and the lowest possible dosage for th...

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 128
• per_device_eval_batch_size: 128
• learning_rate: 2e-05
• num_train_epochs: 5
• warmup_ratio: 0.1
• fp16: True
• dataloader_drop_last: True
• dataloader_num_workers: 2
• load_best_model_at_end: True
• push_to_hub: True
• hub_model_id: RikoteMaster/embedder-granite
• hub_strategy: end
• hub_private_repo: True

All Hyperparameters

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.10.17
• Sentence Transformers: 4.1.0
• Transformers: 4.52.3
• PyTorch: 2.7.0+cu126
• Accelerate: 1.7.0
• Datasets: 3.6.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss