CrossEncoder based on BAAI/bge-reranker-v2-m3

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: BAAI/bge-reranker-v2-m3
• Maximum Sequence Length: 512 tokens
• Number of Output Labels: 1 label
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Full Model Architecture

CrossEncoder(
  (0): Transformer({'transformer_task': 'sequence-classification', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'logits'}}, 'module_output_name': 'scores', 'architecture': 'XLMRobertaForSequenceClassification'})
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Cross Encoder Reranking

• Datasets: query_note_test and query_note_val
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:
  json

  1{ 2 "at_k": 10 3}

Training Details

Training Dataset

Unnamed Dataset

• Size: 769,572 training samples
• Columns: sentence1, sentence2, and label
• Approximate statistics based on the first 100 samples:

  	sentence1	sentence2	label
  type	string	string	float
  modality	text	text
  details	min: 5 tokens mean: 7.92 tokens max: 13 tokens	min: 18 tokens mean: 208.42 tokens max: 512 tokens	min: 0.0 mean: 0.0 max: 0.0

• Samples:

  sentence1	sentence2	label
  infp男和infj女	谁懂啊啊啊‼️💗INFP×INFJ相处好戳我[object Object]绿老头和小蝴蝶都是高敏易碎人群，消息秒回啊，朋友圈点赞啊，这种细微且偏爱的行为，都能让对方感到安心和减少内耗。[object Object]小蝴蝶的话，有事情就不要藏着掖着，对绿老头也不要有过多的试探行为，有问题就直接问，直接说，只要把事情拖到情绪都上来了，那这问题就更难解决了。因为绿老头只是看起来淡然，但其实他们的内心是很火热躁动的，他们只是不好意思主动表达。绿老头得学会调整，明知道小蝴蝶容易想多内耗，那就多主动表现情绪，不要让小蝴蝶觉得你对他淡淡的，要是能偶尔打个只球，明确的表达一下内心，那就更好了。#心理[话题]# #mbti[话题]# #infp[话题]# #infp日常[话题]# #infp和infj[话题]# #INFJ[话题]# #infj的世界[话题]# #我的日常[话题]# #温暖治愈[话题]#	0.0
  电动车远光灯刺眼反击	电动车在这些情况下是全责哦！注意了哦！[object Object]骑电动车一定要小心这些情况哦！[暗中观察R]#交通事故理赔[话题]# #交通事故[话题]# #法律求助[话题]# #电动车[话题]# #法律咨询[话题]# #交通安全[话题]#	0.0
  电动车远光灯刺眼反击	支付宝上这个骑行险有用吗[object Object]#车险[话题]# #电动车骑行险[话题]# #支付宝[话题]#	0.0

• Loss: BinaryCrossEntropyLoss with these parameters:
  json

  1{ 2 "activation_fn": "torch.nn.modules.linear.Identity", 3 "pos_weight": null 4}

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 32
• learning_rate: 0.0001
• weight_decay: 0.01
• num_train_epochs: 1
• lr_scheduler_type: cosine
• warmup_ratio: 0.05
• seed: 3407
• bf16: True

All Hyperparameters

Training Logs

Training Time

• Training: 57.2 minutes

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.5.1
• Transformers: 4.56.2
• PyTorch: 2.11.0+cu128
• Accelerate: 1.13.0
• Datasets: 4.3.0
• Tokenizers: 0.22.2

Additional Resources

• Training and Finetuning Reranker Models with Sentence Transformers: the end-to-end guide for training or finetuning Cross Encoder (reranker) models.
• Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video reranker models through the same API.
• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: training multimodal Cross Encoders.

Citation

BibTeX

Sentence Transformers