SentenceTransformer based on intfloat/multilingual-e5-large-instruct

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: intfloat/multilingual-e5-large-instruct
• Maximum Sequence Length: 256 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Training Dataset:
  • clef-me5-triplets-hard-negatives

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'XLMRobertaModel'})
  (1): Pooling({'embedding_dimension': 1024, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Information Retrieval

• Dataset: CT26-dev-split
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

clef-me5-triplets-hard-negatives

• Dataset: clef-me5-triplets-hard-negatives
• Size: 18,280 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 26 tokens mean: 59.6 tokens max: 112 tokens	min: 22 tokens mean: 188.95 tokens max: 256 tokens	min: 27 tokens mean: 194.91 tokens max: 256 tokens

• Samples:

  anchor	positive	negative
  query: Peer-reviewed scientific studies project universal mask use would spare nearly 130,000 U.S. lives by February. But Utah carries on with a baffling, lax patchwork of feeble mask rules. If elected, I’ll shield your family. #PetersonProtects #utpol	passage: 469,578–578,347) lives could be lost to COVID-19 across the United States by 28 February 2021.[object Object][object Object]title: Modeling COVID-19 scenarios for the United States[object Object]We find that achieving universal mask use (95% mask use in public) could be sufficient to ameliorate the worst effects of epidemic resurgences in many states. Universal mask use could save an additional 129,574 (85,284–170,867) lives from September 22, 2020 through the end of February 2021, or an additional 95,814 (60,731–133,077) lives assuming a lesser adoption of mask wearing (85%), when compared to the reference scenario.	passage: title: Universal Masking to Prevent SARS-CoV-2 Transmission—The Time Is Now[object Object]abstract: Xiaowen Wang, MD; Enrico G. Ferro, MD; Guohai Zhou, PhD; Dean Hashimoto, MD, JD; Deepak L. Bhatt, MD, MPH
  query: @KathMLee1 Jab wrecks their innate immunity- dampening IL6 VD8 NK’s- leaving you exposed not only to Covid but other hidden viruses and cancers- IME	passage: -term transcriptional changes in immune cells after vaccination, as assessed by RNA sequencing.[object Object][object Object]title: The impact of BNT162b2 mRNA vaccine on adaptive and innate immune responses[object Object]Additionally, vaccination with BNT162b2 modulated innate immune responses as measured by the production of inflammatory cytokines when stimulated with various microbial stimuli other than SARS-CoV-2, including higher IL-1/IL-6 release and decreased production of IFN-α. Altogether, these data expand our knowledge regarding the overall immunological effects of this new class of vaccines and underline the need of additional studies to elucidate their effects on both innate and adaptive immune responses.	passage: the genetic modifications introduced by the vaccine are likely the source of these differential responses.

  title: Innate Immune Suppression by SARS-CoV-2 mRNA Vaccinations: The role of G-quadruplexes, exosomes and microRNAs In this paper, we present the evidence that vaccination, unlike natural infection, induces a profound impairment in type I interferon signaling, which has diverse adverse consequences to human health. We explain the mechanism by which immune cells release into the circulation large quantities of exosomes containing spike protein along with critical microRNAs that induce a signaling response in recipient cells at distant sites. We also identify potential profound disturbances in regulatory control of protein synthesis and cancer surveillance. These disturbances are shown to have a potentially direct causal link to neurodegenerative disease, myocarditis, immune thrombocytopenia, Bell’s palsy, liver disease, impaired adaptive immunity, increased tumorigenesis... | | query: Rebuilding of October #Arctic sea ice volume spanning over the past 100 years... [Side-by-side look between PIOMAS-20C and PIOMAS data sets now refreshed through October 2021. Model details available at |

  passage: title: Arctic Sea Ice Volume Variability over 1901–2010: A Model-Based Reconstruction
  abstract: Abstract PIOMAS-20C, an Arctic sea ice reconstruction for 1901–2010, is produced by forcing the Pan-Arctic Ice Ocean Modeling and Assimilation System (PIOMAS) with ERA-20C atmospheric data. ERA-20C performance over Arctic sea ice is assessed by comparisons with measurements and data from other reanalyses. ERA-20C performs similarly with respect to the annual cycle of downwelling radiation, air temperature, and wind speed compared to reanalyses with more extensive data assimilation such as ERA-Interim and MERRA. PIOMAS-20C sea ice thickness and volume are then compared with in situ and aircraft remote sensing observations for the period of ~1950–2010. Error statistics are similar to those for PIOMAS. We compare the magnitude and patterns of sea ice variability between the first half of the twentieth century (1901–40) and the more recent period (1980–2010), both marked by sea ice decl...

  | passage: title: Variability of Arctic Sea Ice Thickness Using PIOMAS and the CESM Large Ensemble[object Object]abstract: Abstract Because of limited high-quality satellite and in situ observations, less attention has been given to the trends in Arctic sea ice thickness and therefore sea ice volume than to the trends in sea ice extent. This study evaluates the spatial and temporal variability in Arctic sea ice thickness using the Pan-Arctic Ice Ocean Modeling and Assimilation System (PIOMAS). Additionally, the Community Earth System Model Large Ensemble Project (LENS) is used to quantify the forced response and internal variability in the model. A dipole spatial pattern of sea ice thickness variability is shown in both PIOMAS and LENS with opposite signs of polarity between the East Siberian Sea and near the Fram Strait. As future sea ice thins, this dipole structure of variability is reduced, and the largest interannual variability is found only along the northern Greenland coastline. |
• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false, 5 "directions": [ 6 "query_to_doc" 7 ], 8 "partition_mode": "joint", 9 "hardness_mode": null, 10 "hardness_strength": 0.0 11}

Evaluation Dataset

clef-me5-triplets-hard-negatives

• Dataset: clef-me5-triplets-hard-negatives
• Size: 963 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 963 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 25 tokens mean: 58.92 tokens max: 121 tokens	min: 30 tokens mean: 191.84 tokens max: 256 tokens	min: 20 tokens mean: 197.65 tokens max: 256 tokens

• Samples:

  anchor	positive	negative
  query: Hot off the press via @CDCgov's @CDCMMWR Efficacy of #COVID19Vaccines in Staving off #SARSCoV2 Infection Among essential staff pre- and amid #B16172 (#DeltaVariant) dominance — Eight U.S. sites, December 2020 – August 2021	passage: title: Effectiveness of COVID-19 Vaccines in Preventing SARS-CoV-2 Infection Among Frontline Workers Before and During B.1.617.2 (Delta) Variant Predominance — Eight U.S. Locations, December 2020–August 2021
  abstract: During December 14, 2020-April 10, 2021, data from the HEROES-RECOVER Cohorts,* a network of prospective cohorts among frontline workers, showed that the Pfizer-BioNTech and Moderna mRNA COVID-19 vaccines were approximately 90% effective in preventing symptomatic and asymptomatic infection with SARS-CoV-2, the virus that causes COVID-19, in real-world conditions (1,2). This report updates vaccine effectiveness (VE) estimates including all COVID-19 vaccines available through August 14, 2021, and examines whether VE differs for adults with increasing time since completion of all recommended vaccine doses. VE before and during SARS-CoV-2 B.1.617.2 (Delta) variant predominance, which coincided with an increase in reported COVID-19 vaccine breakthrough infections, wer...	passage: horts of health care personnel, first responders, and other essential and frontline workers* in eight U.S.[object Object][object Object]title: Interim Estimates of Vaccine Effectiveness of BNT162b2 and mRNA-1273 COVID-19 Vaccines in Preventing SARS-CoV-2 Infection Among Health Care Personnel, First Responders, and Other Essential and Frontline Workers — Eight U.S. Locations, December 2020–March 2021[object Object]locations during December 14, 2020-March 13, 2021, CDC routinely tested for SARS-CoV-2 infections every week regardless of symptom status and at the onset of symptoms consistent with COVID-19-associated illness.
  query: Un #lait d'allaitement azur ou vert? Ceci n'est pas une plaisanterie, cela provient d'une recherche post injection de 178 #dames qui tétinent. 3 dames après la 1ère injection (Moderna ou Pfizer) et 2 dames après la 2e💉. Mais les médias persister à prôner d'injecter...	passage: title: Maternal and child outcomes reported by breastfeeding women following mRNA COVID-19 vaccination
  abstract: Table of Contents Summary What’s known on this subject One published U.S. study with 31 breastfeeding women who were vaccinated with either of the two available mRNA COVID-19 vaccines found that 67% experienced any side effects after dose one and 61% after dose two of the vaccine. What this study adds More than 85% of 180 breastfeeding women who received an mRNA COVID-19 vaccine reported local or systemic symptoms, with higher frequency following the second dose. Few symptoms were reported in their breastfed children. No serious adverse events were noted. Contributors’ Statement Page Dr. Chambers and Ms. Bertrand designed the study and supervised the collection of data used in the study. Mr. Honerkamp-Smith performed the statistical analysis. All authors were involved in preparing the manuscript. All authors approved the final manuscript as submitted and agree to be...	passage: title: Codominant IgG and IgA expression with minimal vaccine mRNA in milk of BNT162b2 vaccinees[object Object]abstract: Lactating women can produce protective antibodies in their milk after vaccination, which has informed antenatal vaccination programs for diseases such as influenza and pertussis. However, whether SARS-CoV-2-specific antibodies are produced in human milk as a result of COVID-19 vaccination is still unclear. In this study, we show that lactating mothers who received the BNT162b2 vaccine secreted SARS-CoV-2-specific IgA and IgG antibodies into milk, with the most significant increase at 3-7 days post-dose 2. Virus-specific IgG titers were stable out to 4-6 weeks after dose 2. In contrast, SARS-CoV-2-specific IgA levels showed substantial decay. Vaccine mRNA was detected in few milk samples (maximum of 2 ng/ml), indicative of minimal transfer. Additionally, infants who consumed post-vaccination human milk had no reported adverse effects up to 28 days post-ingestion.
  query: 📰Les scientifiques tentent présentement d'évaluer l'effet des récifs coralliens fabriqués sur les poissons. À ce stade, les données à modeste envergure pour le vivaneau rouge et le baliste gris dans le golfe du Mexique sont prometteurs.	passage: title: Fine-scale movement of northern Gulf of Mexico red snapper and gray triggerfish estimated with three-dimensional acoustic telemetry[object Object]abstract: Abstract Red snapper and gray triggerfish are ecologically, economically, and culturally important reef fishes in the northern Gulf of Mexico (nGOM). Scientists and managers have sought to understand the effects of artificial reefs on reef fish ecology by focusing on fish residency and movement at artificial reefs with less attention paid to broader spatial and temporal patterns in reef fish movements among a seascape of artificial reefs and other natural habitats. We used novel large-scale (> 15 km 2 ) geopositioning acoustic telemetry arrays to track the 3-dimensional movements of tagged red snapper (n = 59) and gray triggerfish (n = 15) among multiple nGOM artificial reefs up to 333 days.	passage: .[object Object][object Object]title: A new species of[object Object]Milne-Edwards, 1879 (Isopoda: Cirolanidae) from the southern Gulf of Mexico with a redescription of[object Object]Kou, Chen and Li, 2017 from off Pratas Island, Taiwan[object Object]Bathynomus kensleyi, previously known from the South China Sea as well as the south-east Swain Reefs, Coral Sea, was primarily differentiated by the elongate and upturned pleotelson spines, but that character is now shown to also occur in mature Bathynomus jamesi. Two specimens from the Gulf of Mexico (obtained from the Enoshima Aquarium in Japan) were compared to species of Bathynomus from the western North Atlantic. Sequence data showed that one of two samples was not B. giganteus Milne-Edwards, 1879, as had been assumed, and it did not match any other species of Bathynomus. That specimen was collected off the Yucatán Peninsula and is morphologically distinct from both B.

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false, 5 "directions": [ 6 "query_to_doc" 7 ], 8 "partition_mode": "joint", 9 "hardness_mode": null, 10 "hardness_strength": 0.0 11}

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 256
• per_device_eval_batch_size: 256
• learning_rate: 6.4e-05
• num_train_epochs: 20
• warmup_ratio: 0.1
• bf16: True
• dataloader_num_workers: 8
• load_best_model_at_end: True
• optim: adamw_8bit
• gradient_checkpointing: True
• gradient_checkpointing_kwargs: {'use_reentrant': False}

All Hyperparameters

Training Logs

• The bold row denotes the saved checkpoint.

Training Time

• Training: 20.6 minutes

Framework Versions

• Python: 3.12.6
• Sentence Transformers: 5.4.1
• Transformers: 4.56.0
• PyTorch: 2.8.0+cu129
• Accelerate: 1.10.1
• Datasets: 4.8.5
• Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss