SentenceTransformer based on hkunlp/instructor-xl

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: hkunlp/instructor-xl
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'T5EncoderModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': False})
  (2): Dense({'in_features': 1024, 'out_features': 768, 'bias': False, 'activation_function': 'torch.nn.modules.linear.Identity', 'module_input_name': 'sentence_embedding', 'module_output_name': 'sentence_embedding'})
  (3): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 31,800 training samples
• Columns: anchor, positive, negative, instruction_text, boost_words, and claim
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative	instruction_text	boost_words	claim
  type	string	string	string	string	list	string
  details	min: 11 tokens mean: 25.07 tokens max: 43 tokens	min: 30 tokens mean: 53.01 tokens max: 86 tokens	min: 24 tokens mean: 56.77 tokens max: 120 tokens	min: 4 tokens mean: 5.75 tokens max: 9 tokens	size: 3 elements	min: 8 tokens mean: 20.32 tokens max: 35 tokens

• Samples:

  anchor	positive	negative	instruction_text	boost_words	claim
  Extract statements advocating for Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.	The claim is true because Alfred Nobel and Bertha von Suttner maintained a long and close correspondence in which she persistently advocated for peace, providing a direct and sustained channel for her to influence his thinking.	The claim is false because Nobel's philosophy on peace was notably different from Suttner's. Nobel held a more cynical and pragmatic view, once remarking that his dynamite factories would end war sooner than her peace congresses. The prize's criteria reflect his practical mindset, not a wholesale adoption of her idealistic pacifism.	Extract statements advocating for	['suttner', 'nobel', 'bertha']	Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.
  Retrieve arguments in favor of Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.	The claim is true because their letters contain direct discussions about creating a peace prize. In an 1893 letter to Suttner, Nobel expressed his desire to establish a prize for the person who had done the most to advance the pacification of Europe, showing a clear causal link between her advocacy and his idea.	The claim is false because although their letters contain direct discussions about creating a peace prize, and in an 1893 letter to Suttner, Nobel expressed his desire to establish a prize for the person who had done the most to advance the pacification of Europe, these private exchanges fail to demonstrate a definitive, exclusive causal link between her advocacy and his final institutional idea.	Retrieve arguments in favor of	['suttner', 'nobel', 'bertha']	Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.
  Find evidence backing Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.	The claim is true because Nobel explicitly read and praised Suttner's influential anti-war novel, "Die Waffen nieder!" ("Lay Down Your Arms!"), demonstrating that her specific work and pacifist message directly impacted his views on the subject.	The claim of 'direct' influence is false because the final text of Nobel's will and the creation of the prizes were a surprise to nearly everyone, including Suttner. If her influence had been so direct and formative, it is likely she would have had more specific knowledge of his ultimate plans before they were revealed.	Find evidence backing	['suttner', 'nobel', 'bertha']	Bertha von Suttner, a 19th-century peace activist and author, directly influenced Alfred Nobel's decision to create the Nobel Peace Prize.

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false, 5 "directions": [ 6 "query_to_doc" 7 ], 8 "partition_mode": "joint", 9 "hardness_mode": null, 10 "hardness_strength": 0.0 11}

Training Hyperparameters

Non-Default Hyperparameters

• num_train_epochs: 2
• learning_rate: 2e-05
• warmup_steps: 0.1
• gradient_accumulation_steps: 2
• bf16: True
• gradient_checkpointing: True
• remove_unused_columns: False

All Hyperparameters

Training Logs

Training Time

• Training: 1.1 hours

Framework Versions

• Python: 3.12.3
• Sentence Transformers: 5.4.1
• Transformers: 5.5.4
• PyTorch: 2.9.1+cu128
• Accelerate: 1.12.0
• Datasets: 4.5.0
• Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss