CrossEncoder based on microsoft/MiniLM-L12-H384-uncased

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: microsoft/MiniLM-L12-H384-uncased
• Maximum Sequence Length: 512 tokens
• Number of Output Labels: 1 label
• Supported Modality: Text
• Training Dataset:
  • ms_marco
• Language: en

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Full Model Architecture

CrossEncoder(
  (0): Transformer({'transformer_task': 'sequence-classification', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'logits'}}, 'module_output_name': 'scores', 'architecture': 'BertForSequenceClassification'})
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Cross Encoder Reranking

• Datasets: NanoMSMARCO_R100, NanoNFCorpus_R100 and NanoNQ_R100
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:
  json

  1{ 2 "at_k": 10, 3 "always_rerank_positives": true 4}

Cross Encoder Nano BEIR

• Dataset: NanoBEIR_R100_mean
• Evaluated with CrossEncoderNanoBEIREvaluator with these parameters:
  json

  1{ 2 "dataset_names": [ 3 "msmarco", 4 "nfcorpus", 5 "nq" 6 ], 7 "dataset_id": "sentence-transformers/NanoBEIR-en", 8 "rerank_k": 100, 9 "at_k": 10, 10 "always_rerank_positives": true 11}

Training Details

Training Dataset

ms_marco

• Dataset: ms_marco at a47ee7a
• Size: 78,704 training samples
• Columns: query, docs, and labels
• Approximate statistics based on the first 100 samples:

  	query	docs	labels
  type	string	list	list
  modality	text
  details	min: 5 tokens mean: 9.2 tokens max: 16 tokens	min: 3 elements mean: 8.26 elements max: 10 elements	min: 3 elements mean: 8.26 elements max: 10 elements

• Samples:

  query	docs	labels
  transcription definition biology	['Transcription. Definition. noun. It is the process of transcribing or making a copy of genetic information stored in a DNA strand into a complementary strand of RNA (messenger RNA or mRNA) with the aid of RNA polymerases. Supplement', 'See also: RNA, protein synthesis. User Contributions / Comments: Transcription is the first step of gene expression, in which a particular segment of DNA is copied into RNA by the enzyme RNA polymerase. It is the process of transcribing or making a copy of genetic information stored in a DNA strand into a complementary strand of RNA (messenger RNA or mRNA) with the aid of RNA polymerases.', 'Transcription is the process by which genetic information from DNA is trans ferred into RNA. DNA sequence is enzymatically copied by RNA polymerase to produce a complement ary nucleotide RNA strand. Transcription and Translation. Transcription of RNA from DNA. One of the central concepts of biology is that RNA is trans cribed from DNA in a process called transcription. There are several types of RNA that can be trans cribed including mRNA, rRNA and tRNA.', 'This complex initiates transcription, and the RNA polymerase begins mRNA synthesis by matching complementary bases to the original DNA strand. The mRNA molecule is elongated and, once the strand is completely synthesized, transcription is terminated. Transcription factors can bind to specific DNA sequences called enhancer and promoter sequences in order to recruit RNA polymerase to an appropriate transcription site. Together, the transcription factors and RNA polymerase form a complex called the transcription initiation complex.', "Transcription is the first step of gene expression, in which a particular segment of DNA is copied into RNA (mRNA) by the enzyme RNA polymerase. Both RNA and DNA are nucleic acids, which use base pairs of nucleotides as a complementary language. In bacteria, transcription begins with the binding of RNA polymerase to the promoter in DNA. RNA polymerase is a core enzyme consisting of five subunits: 2 α subunits, 1 β subunit, 1 β' subunit, and 1 ω subunit.", ...]	[1, 0, 0, 0, 0, ...]
  how to type backslash on US keyboard	["Zacy5000 said: Or you could use 'Alt Gr + The normal US button for backslash'. Alt Gr is the one to the right hand side of the space bar and the normal US button for the backslash is next to the enter button and shift key (may slightly differ on different keyboards). This is much easier then using the Alt code. This is more for my own reference than anything else but today I was needing to enter a backslash key while using a US keyboard but with UK keyboard settings. After trying pretty much every key on the keyboard in all combinations, I realised there was no key combination that would do this directly.", "Our computers had Korean keyboards, with no obvious backslash. The solution is to use the character code. Hold down the ALT key and type 92 using the numeric keypad (not the numbers on the top of the main letters section). When you release the ALT key you'll get a backslash. The same trick can be used for accented characters such as e (é alt) 130 or (E ë). Alt · 137 neil 4. years ago", "When using windows auth usernames usually contain a backslash \, I can't type it, when I do the textbox (or textarea) just loose focus, and no backslash is added. I tried switching to input mode, but it doesn't help. I'm using: 1 Vimum 1.30. 2 Chrome 15.0.874.121. 3 Windows 7. 4 Spanish Latin Keyboard. ", 'Alternatively referred to as a backward slash and reverse solidus, the backslash is a character \ first introduced by Bob Bemer in 1960. A backslash can be used to represent a file path in MS-DOS and Windows, for example, c:\windows is an example of the backslash being used. ', "3. I'm trying to use a UK keymap on a physical US keyboard on my notebook (Asus Zenbook UX31E). My keyboard has vertical bar and backslash (	and \) above the Enter key, but I get hash and tilde instead. On UK keyboards, pipe and backslash seem to be just to the left of the Z key. I use a European keyboard and have only a forward-slash / . Using the alt to the right of the space bar (alt gr) together with the key for / gives
  how many oz in a shot	['(*) A shotglass is usually 1.5 ounces, but sometimes 2 ounces with a measuring line at 1.5 ounces. You can also buy (in US) short shot glasses or pony shots which are 1 ounce. Pony shots are usually used with martinis, manhattans, and rob roys.', 'In the US, a shot is 1.5 ounces and a small shot is 1 ounce of liquor for an alcoholic beverage such as whiskey. However, this varies internationally and there is no absolutely defined unit, and various sizes of glasses are used.', 'Other shot glass sizes are called short shot glasses or pony shots both of which are 1 ounce. Alcoholism experts define a standard drink as 12 ounces of beer, 1.5 ounces of 72-proof distilled spirits, or 5 ounces of wine, all of which contain approximately .54 ounces of alcohol.', 'Since a shot is typically a spirit with 40 percent alcohol, this makes the shot 1.5 oz. or 42.62 mL (though many establishments serve a standard drink of 1 oz). A double shot in North America may be either 2.5 or 3 fluid ounces.', 'Using a standard 1.5 ounce shot glass filled to the brim results in the following number of standard drinks: 1 1.5 ounces of 72-proof = 1 standard drink. 2 1.5 ounces of 80-proof = 1.2 standard drinks. 3 1.5 ounces of 100-proof = 1.5', ...]	[1, 0, 0, 0, 0, ...]

• Loss: ListMLELoss with these parameters:
  json

  1{ 2 "activation_fn": "torch.nn.modules.linear.Identity", 3 "mini_batch_size": 16, 4 "respect_input_order": true 5}

Evaluation Dataset

ms_marco

• Dataset: ms_marco at a47ee7a
• Size: 1,000 evaluation samples
• Columns: query, docs, and labels
• Approximate statistics based on the first 100 samples:

  	query	docs	labels
  type	string	list	list
  modality	text
  details	min: 5 tokens mean: 9.01 tokens max: 18 tokens	min: 3 elements mean: 8.07 elements max: 10 elements	min: 3 elements mean: 8.07 elements max: 10 elements

• Samples:

  query	docs	labels
  average daily commute	['The average daily commute is about 25 minutes. By Heather Huhman. + More. Heather R. Huhman With the average daily U.S. commute at about 25 minutes, many workers are spending at least an hour traveling each day to and from work. For some densely populated cities, such as Chicago or New York, this average is 30 to 35 minutes, sometimes more. Heather R. Huhman.', "About 8% of workers in the USA commute an hour or longer; 4.3% have it much easier because they work from home. A quick look at the highs and lows of commuting: Source: Census Bureau's American Community Survey estimates, 2011 data. The average travel time for workers who commute by public transportation is higher than that of workers who use other modes, said Census Bureau statistician Brian McKenzie. For some workers, using transit is a necessity, but others simply choose a longer travel time over sitting in traffic..", "Americans' commutes aren't getting longer. The national average, one-way daily commute is 25.5 minutes, and 4.3% of the nation's workers work from home. The average travel time for workers who commute by public transportation is higher than that of workers who use other modes, said Census Bureau statistician Brian McKenzie. For some workers, using transit is a necessity, but others simply choose a longer travel time over sitting in traffic..", 'But the number spending a total of three hours or more travelling has risen to 1.84million-or nine per cent of the workforce. According to the results, accountants have the longest average daily commute – 75.6 minutes – followed by IT workers at 65.6 minutes. The number of Britons spending at least three hours a day on their work commute has risen by 50 per cent. The average total daily commuting distance has dropped from 18.5 miles in 2008 to 16.7 miles-a fall of 10 per cent-as people live slightly closer to their workplaces.', "* = Less than 0.5%. According to the survey, American workers say they spend an average of 45.6 minutes commuting to and from work in a typical day. (This includes workers who report zero minutes commuting; excluding these workers yields an average commute time of 48.1 minutes). PRINCETON, NJ -- Gallup's annual Work and Education survey finds that American workers report spending an average of 4", ...]	[1, 0, 0, 0, 0, ...]
  should hulu charge fee for service	['Hulu is going to charge for their services, which is their right. However Hulu told the public, [Hulu] think[s] what [they] need to do is deliver [their] content to consumers in a way where they will appreciate the value.. I am assured the public already greatly appreciate the value of their service. If hulu begins to charge: I will upgrade my tivo, get back to Netflix, and probably read more. Seem like a pretty thin attempt at padding the bottom line. Maybe the entertainment bubble has burst and the industry should stop paying actors ten million dollars per movie or 1 million dollars per episode.', 'Price : Netflix subscriptions currently cost $7.99 per month, but that price will rise by $1 or $2 for new members later this quarter. Existing members will keep their current price for a generous period of time. Assuming the $2 increase, Netflix will soon cost $119.88 per year.', "Hulu Plus is the subscription based big brother to Hulu's free video streaming platform. It costs $7.99 a month, and really does open up the catalog of available titles. In fact, for subscribers, there are many shows that have the complete catalog of episodes from every season. ", 'You can try out the service for a one-month free trial, after which you have to fork up $79. But if you don’t want the service, you can opt out of it at no charge. Since Amazon uses a yearly subscription model, you will be charged an early termination fee if you want to suddenly cancel your subscription. Hulu Plus. Netflix. For $7.99, Netflix subscribers get to enjoy video on demand streaming on two devices. The new Family plan that costs $11.99, entitles you to on-demand streaming on four devices simultaneously. What this means is that four people can watch four different shows at the same time, on four different device', 'When Google starts paying for my hulu, Netflix or Amazon prime, I will change to them. :). Netflix service runs $8.99 per month — which comes out to $107.88 a year — while Amazon Prime is $99 per year. Hulu Plus is $7.99 a month, for the year that equals 95.88. 1 Amazon Prime charges $99 per year for a membership (which equals $8.25 per month), making it very similar to the above options in terms of price. 2 You also get additional benefits including free two-day shipping for purchases made on Amazon.com.', ...]	[1, 0, 0, 0, 0, ...]
  when are the newsboys playing at the state fair	["See NewsBoys at the NC State Fair. HIS Radio is your Host Station for this year's Christian concert night at the NC State Fair. We're happy to welcome the NewsBoys back to the Triangle...at Dorton Arena on Friday night, Oct. 24th. Tickets to this show are SOLD OUT! The ONLY place you may still obtain tickets is here on HIS Radio, but we're not selling them.", 'Please try again later. newsboys play jesus freak at the state fair of wv. duncan phillips on drums plays rotating drums, awesome! all rights reserved to Newsboys. Category. License.', 'The NEWSBOYS will be appearing at the Nebraska State Fair on Thursday, September 3, at 7:30pm. Tickets are on sale now for a $6 convenience fee plus gate at Etix.com or at the Heartland Events Center Box Office in Grand Island. Fan Club premier packages are also available for $30 plus gate and fees.', "One of Christian rock's most enduring acts, the Newsboys will appear at the Nebraska State Fair in 2015."]	[1, 0, 0, 0]

• Loss: ListMLELoss with these parameters:
  json

  1{ 2 "activation_fn": "torch.nn.modules.linear.Identity", 3 "mini_batch_size": 16, 4 "respect_input_order": true 5}

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• num_train_epochs: 1
• learning_rate: 2e-05
• warmup_steps: 0.1
• bf16: True
• per_device_eval_batch_size: 16
• load_best_model_at_end: True
• seed: 12

All Hyperparameters

Training Logs

• The bold row denotes the saved checkpoint.

Training Time

• Training: 39.7 minutes
• Evaluation: 6.7 minutes
• Total: 46.4 minutes

Framework Versions

• Python: 3.11.6
• Sentence Transformers: 5.7.0.dev0
• Transformers: 5.10.2
• PyTorch: 2.10.0+cu128
• Accelerate: 1.14.0
• Datasets: 4.8.4
• Tokenizers: 0.22.2

Additional Resources

• Training and Finetuning Reranker Models with Sentence Transformers: the end-to-end guide for training or finetuning Cross Encoder (reranker) models.
• Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video reranker models through the same API.
• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: training multimodal Cross Encoders.

Citation

BibTeX

Sentence Transformers

ListMLELoss