ettin-reranker-150m-v1

Model Details

Model Description

• Model Type: Cross Encoder
• Base model: jhu-clsp/ettin-encoder-150m
• Maximum Sequence Length: 7999 tokens
• Number of Output Labels: 1 label
• Supported Modality: Text
• Training Dataset: cross-encoder/ettin-reranker-v1-data
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Documentation: Cross Encoder Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Cross Encoders on Hugging Face

Full Model Architecture

CrossEncoder(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'ModernBertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
  (2): Dense({'in_features': 768, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.activation.GELU', 'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
  (3): LayerNorm({'dimension': 768})
  (4): Dense({'in_features': 768, 'out_features': 1, 'bias': True, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'sentence_embedding', 'module_output_name': 'scores'})
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

MTEB(eng, v2) Retrieval

Speed

Metrics

Cross Encoder Reranking

• Datasets: NanoMSMARCO_R100, NanoNFCorpus_R100, NanoNQ_R100, NanoFiQA2018_R100, NanoTouche2020_R100, NanoSciFact_R100, NanoHotpotQA_R100, NanoArguAna_R100, NanoFEVER_R100, NanoDBPedia_R100, NanoClimateFEVER_R100, NanoSCIDOCS_R100 and NanoQuoraRetrieval_R100
• Evaluated with CrossEncoderRerankingEvaluator with these parameters:
  json

  1{ 2 "at_k": 10, 3 "always_rerank_positives": true 4}

Cross Encoder Nano BEIR

• Dataset: NanoBEIR_R100_mean
• Evaluated with CrossEncoderNanoBEIREvaluator with these parameters:
  json

  1{ 2 "dataset_names": [ 3 "msmarco", 4 "nfcorpus", 5 "nq", 6 "fiqa2018", 7 "touche2020", 8 "scifact", 9 "hotpotqa", 10 "arguana", 11 "fever", 12 "dbpedia", 13 "climatefever", 14 "scidocs", 15 "quoraretrieval" 16 ], 17 "dataset_id": "sentence-transformers/NanoBEIR-en", 18 "rerank_k": 100, 19 "at_k": 10, 20 "always_rerank_positives": true 21}

[!NOTE] The release blogpost quotes a slightly higher NanoBEIR mean NDCG@10 of 0.7086 for this model, computed in fp32 rather than the bfloat16 used by the training-time evaluation above. Both numbers are valid.

Training Details

Training Dataset

ettin-reranker-v1-data

• Dataset: cross-encoder/ettin-reranker-v1-data
• Size: 143,393,475 training samples
• Columns: query, document, and label
• Approximate statistics based on the first 1000 samples:

  	query	document	label
  type	string	string	float
  details	min: 26 characters mean: 55.52 characters max: 249 characters	min: 63 characters mean: 659.91 characters max: 3975 characters	min: -2.94 mean: 8.51 max: 13.88

• Samples:

  query	document	label
  Help me with my Reborn performance	I was reading the comment section for Dotacinema's world of dota video, and a bunch of people were complaining how there were a lot of bugs and some talked about PERFORMANCE ISSUES. But there were also people saying that reborn has actually IMPROVED their gameplay?[object Object][object Object][object Object]I am one of those people who is running into performance issues and would desperately like to know how some are getting BETTER performance while others like me are getting worse. I'm not complaining about bugs, I'm complaing about framerate, I use to get 60 fps solid in source 1 but I now have 40 or at worst 30 fps in source 2.[object Object]I have an i3 processor/gtx560ti/16gb RAM[object Object][object Object]i dont think it's a potato pc, so I dont know what's happening, I cleaned my computer recently so dust isnt affecting anything in anyway.[object Object]So if you gained or had IMPROVED performance in source 2 please list the settings you are enabling, so I can see where I am at fault. (v sync is off btw)[object Object][object Object]TLDR: Have bad performance now from source 2, if you have good p...	9.5
  Really wanna try out the game and expansion, ~$60 is hefty. Likelihood of sales?	As per title, steam sells the game and its expansions for $60 total. Heavy price to drop. Are there sales on any other website? This game looks fantastic to immerse in otherwise and I'm pleased that this subreddit has at least some attention to help out new folks!	9.25
  Your Avatar. [MGSV Spoilers]	Was anyone else suprised he actually replaces the snake model in some cutscenes. I've only tried the first Quiet cutscenes, i was just amazed I haven't seen anybody else say this yet.[object Object]Sorry if repost.	5.25

• Loss: MSELoss with these parameters:
  json

  1{ 2 "activation_fn": "torch.nn.modules.linear.Identity" 3}

Evaluation Dataset

ettin-reranker-v1-data

• Dataset: cross-encoder/ettin-reranker-v1-data
• Size: 5,000 evaluation samples
• Columns: query, document, and label
• Approximate statistics based on the first 1000 samples:

  	query	document	label
  type	string	string	float
  details	min: 14 characters mean: 52.62 characters max: 168 characters	min: 11 characters mean: 50.12 characters max: 184 characters	min: 4.44 mean: 13.49 max: 18.62

• Samples:

  query	document	label
  Why do we need binomial distribution?	Why is the binomial distribution important?	11.375
  I already have Windows 10, can I delete Windows.old?	After resetting windows 10, can I safely delete the "old windows" folder?	10.875
  How can guys last longer during sex?	How do men last longer in bed?	10.8125

• Loss: MSELoss with these parameters:
  json

  1{ 2 "activation_fn": "torch.nn.modules.linear.Identity" 3}

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 3
• num_train_epochs: 1
• learning_rate: 1.5e-05
• warmup_steps: 0.03
• bf16: True
• per_device_eval_batch_size: 3
• load_best_model_at_end: True
• seed: 12

All Hyperparameters

Training Logs

• The bold row denotes the saved checkpoint.

Training Time

• Training: 18.5 hours
• Evaluation: 14.5 minutes
• Total: 18.7 hours

Framework Versions

• Python: 3.11.15
• Sentence Transformers: 5.4.1
• Transformers: 5.7.0
• PyTorch: 2.7.0+cu126
• Accelerate: 1.13.0
• Datasets: 4.8.5
• Tokenizers: 0.22.2

Citation

BibTeX

Ettin Reranker Blogpost

Sentence Transformers