CrossEncoder based on microsoft/MiniLM-L12-H384-uncased

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: microsoft/MiniLM-L12-H384-uncased
• Maximum Sequence Length: 512 tokens
• Number of Output Labels: 1 label
• Training Dataset:
  • ms_marco
• Language: en

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Cross Encoder Reranking

• Datasets: NanoMSMARCO_R100, NanoNFCorpus_R100 and NanoNQ_R100
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:
  json

  1{ 2 "at_k": 10, 3 "always_rerank_positives": true 4}

Cross Encoder Nano BEIR

• Dataset: NanoBEIR_R100_mean
• Evaluated with CrossEncoderNanoBEIREvaluator with these parameters:
  json

  1{ 2 "dataset_names": [ 3 "msmarco", 4 "nfcorpus", 5 "nq" 6 ], 7 "rerank_k": 100, 8 "at_k": 10, 9 "always_rerank_positives": true 10}

Training Details

Training Dataset

ms_marco

• Dataset: ms_marco at a47ee7a
• Size: 78,704 training samples
• Columns: query, docs, and labels
• Approximate statistics based on the first 1000 samples:

  	query	docs	labels
  type	string	list	list
  details	min: 12 characters mean: 33.99 characters max: 98 characters	min: 3 elements mean: 6.50 elements max: 10 elements	min: 3 elements mean: 6.50 elements max: 10 elements

• Samples:

  query	docs	labels
  what does a business development consultant do	['Duties and Responsibilities. An organizational development consultant is a person called in to a company, be it a large corporation or a small business, to evaluate how it operates and make recommendations for improvement.', 'Many sales businesses use business development consultants to help generate leads and show them how to do so. In a business such as sales, lead generation can make or break a company. Having someone show a business owner how to successfully acquire this key piece of information is very important.', 'Development of a marketing strategy is another area covered by a business development consultant. Many businesses struggle with devising ways to effectively market their business to prospective clients.', 'A Good Business Consultant Has Extensive Experience. A good Business Consultant has experience working in and working with a broad range of businesses. It is the accumulated business history of a Business Consultant which makes the consultant valuable.', "A busines...	[1, 0, 0, 0, 0, ...]
  did soren kjeldsen ever play in the masters	["Recent News. Soeren Søren kjeldsen stalled on the back nine and finished joint-runner up in The British masters At woburn with a-2-under par-32=37. 69 The, dane who Captured'may S Irish, open looked good for his second win of the season when shooting four birdies against a single bogey on The' marquess'course s front. nine", "Latest News. Soeren Søren kjeldsen stalled on the back nine and finished joint-runner up in The British masters At woburn with a-2-under par-32=37. 69 The, dane who Captured'may S Irish, open looked good for his second win of the season when shooting four birdies against a single bogey on The' marquess'course s front. nine", "Soren Kjeldsen of Denmark (L) celebrates winning the Irish Open with Austria's Bernd Wiesberger …. It is amazing to be holding the trophy but then I felt good coming into the tournament, he said. I played well in my last two tournaments and while I was not in contention I had the chance today to change all that.", "Denmark. Soeren Søren kje...	[1, 0, 0, 0, 0, ...]
  what is a sell limit order	["Sell Limit Order. A Sell Limit Order is an order to sell a specified number of shares of a stock that you own at a designated price or higher, at a price that is above the current market price. This is your limit price, in other words, the minimum price you are willing to accept to sell your shares. The main benefit of a Sell Limit Order is that you may be able to sell the shares that you own at a minimum price that you specify IF the stock's price raises to that price. Sell Limit Orders are great for maximizing profit-taking.", 'Stop-Limit Order. A stop-limit order is an order to buy or sell a stock that combines the features of a stop order and a limit order. Once the stop price is reached, a stop-limit order becomes a limit order that will be executed at a specified price (or better)', "You place a Sell Limit Order @ $50 on 100 shares of TGT. Now suppose the price trades up to $50. As long as the price remains above $50 per share, your shares would then be sold at the next best av...	[1, 0, 0, 0, 0, ...]

• Loss: PListMLELoss with these parameters:
  json

  1{ 2 "lambda_weight": "sentence_transformers.cross_encoder.losses.PListMLELoss.PListMLELambdaWeight", 3 "activation_fct": "torch.nn.modules.linear.Identity", 4 "mini_batch_size": null, 5 "respect_input_order": true 6}

Evaluation Dataset

ms_marco

• Dataset: ms_marco at a47ee7a
• Size: 1,000 evaluation samples
• Columns: query, docs, and labels
• Approximate statistics based on the first 1000 samples:

  	query	docs	labels
  type	string	list	list
  details	min: 11 characters mean: 33.46 characters max: 108 characters	min: 2 elements mean: 6.00 elements max: 10 elements	min: 2 elements mean: 6.00 elements max: 10 elements

• Samples:

  query	docs	labels
  how much to spend on wordpress hosting	['Shared server. This will cost as little as $3 per month to around $10 per month depending on how you want to pay for it (by the month or by the year). The performance of your site will suffer from shared hosting. It’s a good choice for a personal blog or for getting you started. 1 Courses – You can take courses online that are free for extremely basic information or spend up to $200 or more for mid to advanced topics. 2 Some courses cost from $20 – $50 for a monthly subscription. 3 This allows you to pay for as much training as you want. 4 This can still cost hundreds of dollars', 'You may also choose to invest in customization, SEO or other factors along the way. If your interest is simply to start a blog on WordPress, you can start with a minimal cost of $60 for unlimited hosting and free domain with Bluehost. You can learn all about which hosting service is best for WordPress here. Domain: Cost – $10. The first element you need to shop for is a domain. Having a domain name is g...	[1, 1, 0, 0, 0, ...]
  what type blood is the universal donor	["At one time, type O negative blood was considered the universal blood donor type. This implied that anyone — regardless of blood type — could receive type O negative blood without risking a transfusion reaction. Even then, small samples of the recipient's and donor's blood are mixed to check compatibility in a process known as crossmatching. In an emergency, however, type O negative red blood cells may be given to anyone — especially if the situation is life-threatening or the matching blood type is in short supply.", 'People with type O Rh D negative blood are often called universal donors. O Rh D negative is the universal donor because it does not contain any antigens (markers). When you … get donated blood that has antigens that are not the same as those of the recipient the blood will clot in the body. AB is a universal acceptor because RBC (red blood cells) contain the A and B antigen (simply put, it is a marker on the cell) so the body a … ccepts any blood type because it recog...	[1, 0, 0, 0, 0, ...]
  dental crown costs average	['The prices for dental crowns range from $500 to $2,500 per crown and are dependent upon the materials used, location of tooth and geographic location. The average cost of a crown is $825, with or without dental insurance coverage. The cheapest cost of a dental crown is $500 for a simple metal crown. Dental crowns are specifically shaped shells that fit over damaged or broken teeth for either cosmetic or structural purposes. 1 People with insurance typically paid $520 – $1,140 out of pocket with an average of $882 per crown. 2 Those without insurance generally paid between $830 and $2,465 per crown with an average cost of $1,350.', '1 All-porcelain crowns require a higher level of skill and take more time to install than metal or porcelain-fused-to-metal crowns, and can cost $800-$3,000 or more per tooth. 2 CostHelper readers without insurance report paying $860-$3,000, at an average cost of $1,430. 1 CostHelper readers without insurance report paying $860-$3,000, at an average cost...	[1, 0, 0, 0, 0, ...]

• Loss: PListMLELoss with these parameters:
  json

  1{ 2 "lambda_weight": "sentence_transformers.cross_encoder.losses.PListMLELoss.PListMLELambdaWeight", 3 "activation_fct": "torch.nn.modules.linear.Identity", 4 "mini_batch_size": null, 5 "respect_input_order": true 6}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• learning_rate: 2e-05
• num_train_epochs: 1
• warmup_ratio: 0.1
• seed: 12
• bf16: True
• load_best_model_at_end: True

All Hyperparameters

Click to expand

• overwrite_output_dir: False
• do_predict: False
• eval_strategy: steps
• prediction_loss_only: True
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• per_gpu_train_batch_size: None
• per_gpu_eval_batch_size: None
• gradient_accumulation_steps: 1
• eval_accumulation_steps: None
• torch_empty_cache_steps: None
• learning_rate: 2e-05
• weight_decay: 0.0
• adam_beta1: 0.9
• adam_beta2: 0.999
• adam_epsilon: 1e-08
• max_grad_norm: 1.0
• num_train_epochs: 1
• max_steps: -1
• lr_scheduler_type: linear
• lr_scheduler_kwargs: {}
• warmup_ratio: 0.1
• warmup_steps: 0
• log_level: passive
• log_level_replica: warning
• log_on_each_node: True
• logging_nan_inf_filter: True
• save_safetensors: True
• save_on_each_node: False
• save_only_model: False
• restore_callback_states_from_checkpoint: False
• no_cuda: False
• use_cpu: False
• use_mps_device: False
• seed: 12
• data_seed: None
• jit_mode_eval: False
• use_ipex: False
• bf16: True
• fp16: False
• fp16_opt_level: O1
• half_precision_backend: auto
• bf16_full_eval: False
• fp16_full_eval: False
• tf32: None
• local_rank: 0
• ddp_backend: None
• tpu_num_cores: None
• tpu_metrics_debug: False
• debug: []
• dataloader_drop_last: False
• dataloader_num_workers: 0
• dataloader_prefetch_factor: None
• past_index: -1
• disable_tqdm: False
• remove_unused_columns: True
• label_names: None
• load_best_model_at_end: True
• ignore_data_skip: False
• fsdp: []
• fsdp_min_num_params: 0
• fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
• fsdp_transformer_layer_cls_to_wrap: None
• accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
• deepspeed: None
• label_smoothing_factor: 0.0
• optim: adamw_torch
• optim_args: None
• adafactor: False
• group_by_length: False
• length_column_name: length
• ddp_find_unused_parameters: None
• ddp_bucket_cap_mb: None
• ddp_broadcast_buffers: False
• dataloader_pin_memory: True
• dataloader_persistent_workers: False
• skip_memory_metrics: True
• use_legacy_prediction_loop: False
• push_to_hub: False
• resume_from_checkpoint: None
• hub_model_id: None
• hub_strategy: every_save
• hub_private_repo: None
• hub_always_push: False
• gradient_checkpointing: False
• gradient_checkpointing_kwargs: None
• include_inputs_for_metrics: False
• include_for_metrics: []
• eval_do_concat_batches: True
• fp16_backend: auto
• push_to_hub_model_id: None
• push_to_hub_organization: None
• mp_parameters:
• auto_find_batch_size: False
• full_determinism: False
• torchdynamo: None
• ray_scope: last
• ddp_timeout: 1800
• torch_compile: False
• torch_compile_backend: None
• torch_compile_mode: None
• dispatch_batches: None
• split_batches: None
• include_tokens_per_second: False
• include_num_input_tokens_seen: False
• neftune_noise_alpha: None
• optim_target_modules: None
• batch_eval_metrics: False
• eval_on_start: False
• use_liger_kernel: False
• eval_use_gather_object: False
• average_tokens_across_devices: False
• prompts: None
• batch_sampler: batch_sampler
• multi_dataset_batch_sampler: proportional

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.11.11
• Sentence Transformers: 3.5.0.dev0
• Transformers: 4.49.0
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.2
• Datasets: 3.4.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

PListMLELoss