SentenceTransformer

Model Details

Model Description

• Model Type: Sentence Transformer

• Maximum Sequence Length: 1024 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 105,064 training samples
• Columns: anchor, positive, negative, negative_2, negative_3, and negative_4
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative	negative_2	negative_3	negative_4
  type	string	string	string	string	string	string
  details	min: 6 tokens mean: 11.81 tokens max: 26 tokens	min: 17 tokens mean: 169.73 tokens max: 986 tokens	min: 12 tokens mean: 181.7 tokens max: 759 tokens	min: 22 tokens mean: 184.76 tokens max: 817 tokens	min: 14 tokens mean: 186.03 tokens max: 859 tokens	min: 12 tokens mean: 179.54 tokens max: 759 tokens

• Samples:

  anchor	positive	negative	negative_2	negative_3	negative_4
  When was quantum field theory developed?	The third thread in the development of quantum field theory was the need to handle the statistics of many-particle systems consistently and with ease. In 1927, Pascual Jordan tried to extend the canonical quantization of fields to the many-body wave functions of identical particles using a formalism which is known as statistical transformation theory; this procedure is now sometimes called second quantization. In 1928, Jordan and Eugene Wigner found that the quantum field describing electrons, or other fermions, had to be expanded using anti-commuting creation and annihilation operators due to the Pauli exclusion principle (see Jordan–Wigner transformation). This thread of development was incorporated into many-body theory and strongly influenced condensed matter physics and nuclear physics.	The application of the new quantum theory to electromagnetism resulted in quantum field theory, which was developed starting around 1930. Quantum field theory has driven the development of more sophisticated formulations of quantum mechanics, of which the ones presented here are simple special cases.	Two classic text-books from the 1960s, James D. Bjorken, Sidney David Drell, "Relativistic Quantum Mechanics" (1964) and J. J. Sakurai, "Advanced Quantum Mechanics" (1967), thoroughly developed the Feynman graph expansion techniques using physically intuitive and practical methods following from the correspondence principle, without worrying about the technicalities involved in deriving the Feynman rules from the superstructure of quantum field theory itself. Although both Feynman's heuristic and pictorial style of dealing with the infinities, as well as the formal methods of Tomonaga and Schwinger, worked extremely well, and gave spectacularly accurate answers, the true analytical nature of the question of "renormalizability", that is, whether ANY theory formulated as a "quantum field theory" would give finite answers, was not worked-out until much later, when the urgency of trying to formulate finite theories for the strong and electro-weak (and gravitational interactions) demanded i...	It was evident from the beginning that a proper quantum treatment of the electromagnetic field had to somehow incorporate Einstein's relativity theory, which had grown out of the study of classical electromagnetism. This need to put together relativity and quantum mechanics was the second major motivation in the development of quantum field theory. Pascual Jordan and Wolfgang Pauli showed in 1928 that quantum fields could be made to behave in the way predicted by special relativity during coordinate transformations (specifically, they showed that the field commutators were Lorentz invariant). A further boost for quantum field theory came with the discovery of the Dirac equation, which was originally formulated and interpreted as a single-particle equation analogous to the Schrödinger equation, but unlike the Schrödinger equation, the Dirac equation satisfies both the Lorentz invariance, that is, the requirements of special relativity, and the rules of quantum mechanics. [object Object]The Dirac equa...	Through the works of Born, Heisenberg, and Pascual Jordan in 1925-1926, a quantum theory of the free electromagnetic field (one with no interactions with matter) was developed via canonical quantization by treating the electromagnetic field as a set of quantum harmonic oscillators. With the exclusion of interactions, however, such a theory was yet incapable of making quantitative predictions about the real world.
  Was there a year 0?	Cassini gave the following reasons for using a year 0:[object Object]Fred Espanak of NASA lists 50 phases of the moon within year 0, showing that it is a full year, not an instant in time. Jean Meeus gives the following explanation:[object Object]Although he used the usual French terms "avant J.-C." (before Jesus Christ) and "après J.-C." (after Jesus Christ) to label years elsewhere in his book, the Byzantine historian Venance Grumel used negative years (identified by a minus sign, −) to label BC years and unsigned positive years to label AD years in a table. He did so possibly to save space and put no year 0 between them.	Games Def Interceptions Fumbles Sacks & Tackles [object Object]Year Age Tm Pos No. G GS Int Yds TD Lng PD FF Fmb FR Yds TD Sk Tkl Ast Sfty AV[object Object]2004 23 NWE ss 42 13 2 0 0 0 0 2 1 0 1 0 0 15 8 2[object Object]2005 24 IND 36 16 0 1 0 1 0 0 8 2 1[object Object]2006 25 IND ss 36 10 1 0 0 0 0 2 11 0 1[object Object]Career 39 3 0 0 0 0 4 2 0 2 0 0 34 10 4[object Object]2 yrs IND 26 1 0 0 0 0 2 1 0 1 0 0 19 2 2[object Object]1 yr NWE 13 2 0 0 0 0 2 1 0 1 0 0 15 8 2[object Object]After pleading guilty in January 2008 to drug charges in Virginia Beach, VA stemming from a March 2007 incident, Reid was initially sentenced to two years in prison for possessing marijuana with the intent to distribute but had the sentence suspended with the agreement he would stay out of trouble for two years. His license was also suspended for six months and ordered to attend drug treatment and counseling.	This enzyme belongs to the family of oxidoreductases, specifically those acting on paired donors, with O2 as oxidant and incorporation or reduction of oxygen. The oxygen incorporated need not be derived from O2 with 2-oxoglutarate as one donor, and incorporation of one atom o oxygen into each donor. The systematic name of this enzyme class is N6,N6,N6-trimethyl-L-lysine,2-oxoglutarate:oxygen oxidoreductase (3-hydroxylating). Other names in common use include trimethyllysine alpha-ketoglutarate dioxygenase, TML-alpha-ketoglutarate dioxygenase, TML hydroxylase, 6-N,6-N,6-N-trimethyl-L-lysine,2-oxoglutarate:oxygen oxidoreductase, and (3-hydroxylating). This enzyme participates in lysine degradation and L-carnitine biosynthesis and requires the presence of iron and ascorbate.	ㅜ is one of the Korean hangul. The Unicode for ㅜ is U+315C.	ㅌ is one of the Korean hangul. The Unicode for ㅌ is U+314C.
  When is the dialectical method used?	The Dialect Test was created by A.J. Ellis in February 1879, and was used in the fieldwork for his work "On Early English Pronunciation". It stands as one of the earliest methods of identifying vowel sounds and features of speech. The aim was to capture the main vowel sounds of an individual dialect by listening to the reading of a short passage. All the categories of West Saxon words and vowels were included in the test so that comparisons could be made with the historic West Saxon speech as well as with various other dialects.	Karl Popper has attacked the dialectic repeatedly. In 1937, he wrote and delivered a paper entitled "What Is Dialectic?" in which he attacked the dialectical method for its willingness "to put up with contradictions". Popper concluded the essay with these words: "The whole development of dialectic should be a warning against the dangers inherent in philosophical system-building. It should remind us that philosophy should not be made a basis for any sort of scientific system and that philosophers should be much more modest in their claims. One task which they can fulfill quite usefully is the study of the critical methods of science" (Ibid., p. 335).	He was one of the first to apply Labovian methods in Britain with his research in 1970-1 on the speech of Bradford, Halifax and Huddersfield. He concluded that the speech detailed in most of dialectology (e.g. A. J. Ellis, the Survey of English Dialects) had virtually disappeared, having found only one speaker out of his sample of 106 speakers who regularly used dialect. However, he found that differences in speech persisted as an indicator of social class, age and gender. This PhD dissertation was later adapted into a book, "Dialect and Accent in Industrial West Yorkshire". The work was criticised by Graham Shorrocks on the grounds that the sociolinguistic methods used were inappropriate for recording the traditional vernacular and that there was an inadequate basis for comparison with earlier dialect studies in West Yorkshire.	The Institute also attempted to reformulate dialectics as a concrete method. The use of such a dialectical method can be traced back to the philosophy of Hegel, who conceived dialectic as the tendency of a notion to pass over into its own negation as the result of conflict between its inherent contradictory aspects. In opposition to previous modes of thought, which viewed things in abstraction, each by itself and as though endowed with fixed properties, Hegelian dialectic has the ability to consider ideas according to their movement and change in time, as well as according to their interrelations and interactions.	For Marx, dialectics is not a formula for generating predetermined outcomes but is a method for the empirical study of social processes in terms of interrelations, development, and transformation. In his introduction to the Penguin edition of Marx's "Capital", Ernest Mandel writes, "When the dialectical method is applied to the study of economic problems, economic phenomena are not viewed separately from each other, by bits and pieces, but in their inner connection as an integrated totality, structured around, and by, a basic predominant mode of production."

• Loss: CachedGISTEmbedLoss with these parameters:
  json

  1{'guide': SentenceTransformer( 2 (0): Transformer({'max_seq_length': 1024, 'do_lower_case': False}) with Transformer model: XLMRobertaModel 3 (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True}) 4 (2): Normalize() 5), 'temperature': 0.01}

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 1024
• learning_rate: 3e-05
• weight_decay: 0.01
• num_train_epochs: 8
• warmup_ratio: 0.05
• bf16: True
• batch_sampler: no_duplicates

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.4.1
• Transformers: 4.49.0
• PyTorch: 2.5.1+cu124
• Accelerate: 1.4.0
• Datasets: 2.21.0
• Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers