SentenceTransformer

Model Details

Model Description

• Model Type: Sentence Transformer

• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 16,199 training samples
• Columns: sentence and label
• Approximate statistics based on the first 1000 samples:

  	sentence	label
  type	string	int
  details	min: 11 tokens mean: 18.73 tokens max: 72 tokens	0: ~0.30% 1: ~0.30% 2: ~0.30% 3: ~0.30% 4: ~2.40% 5: ~0.30% 6: ~0.30% 7: ~0.30% 8: ~0.30% 9: ~0.30% 10: ~0.30% 11: ~0.40% 12: ~0.30% 13: ~0.30% 14: ~0.40% 15: ~0.30% 16: ~0.30% 17: ~0.30% 18: ~0.90% 19: ~0.30% 20: ~1.30% 21: ~0.30% 22: ~1.10% 23: ~0.30% 24: ~0.30% 25: ~0.30% 26: ~0.30% 27: ~0.30% 28: ~0.30% 29: ~0.30% 30: ~0.30% 31: ~0.30% 32: ~0.30% 33: ~0.30% 34: ~0.30% 35: ~0.30% 36: ~0.30% 37: ~0.30% 38: ~0.30% 39: ~0.30% 40: ~0.40% 41: ~0.30% 42: ~0.30% 43: ~0.30% 44: ~0.60% 45: ~0.70% 46: ~0.30% 47: ~0.30% 48: ~0.30% 49: ~0.30% 50: ~0.30% 51: ~0.30% 52: ~0.30% 53: ~0.30% 54: ~0.30% 55: ~0.30% 56: ~0.30% 57: ~0.80% 58: ~0.30% 59: ~0.30% 60: ~0.60% 61: ~0.30% 62: ~0.30% 63: ~0.30% 64: ~0.50% 65: ~0.30% 66: ~0.30% 67: ~0.30% 68: ~0.30% 69: ~0.50% 70: ~0.60% 71: ~0.30% 72: ~0.30% 73: ~0.30% 74: ~0.30% 75: ~0.30% 76: ~0.30% 77: ~0.30% 78: ~0.30% 79: ~0.30% 80: ~0.30% 81: ~0.30% 82: ~0.30% 83: ~0.30% 84: ~0.80% 85: ~0.60% 86: ~0.50% 87: ~0.30% 88: ~0.30% 89: ~16.30% 90: ~0.30% 91: ~0.30% 92: ~0.30% 93: ~0.30% 94: ~0.30% 95: ~0.30% 96: ~0.30% 97: ~0.30% 98: ~0.50% 99: ~0.30% 100: ~0.30% 101: ~0.30% 102: ~0.30% 103: ~0.30% 104: ~0.30% 105: ~0.30% 106: ~0.30% 107: ~0.70% 108: ~0.30% 109: ~3.20% 110: ~0.30% 111: ~0.40% 112: ~2.30% 113: ~0.30% 114: ~0.30% 115: ~0.50% 116: ~0.50% 117: ~0.50% 118: ~0.40% 119: ~0.30% 120: ~0.30% 121: ~0.30% 122: ~0.80% 123: ~0.30% 124: ~0.30% 125: ~0.30% 126: ~0.30% 127: ~0.30% 128: ~0.30% 129: ~0.30% 130: ~0.30% 131: ~0.50% 132: ~0.30% 133: ~0.40% 134: ~0.30% 135: ~0.30% 136: ~0.30% 137: ~0.30% 138: ~0.30% 139: ~0.30% 140: ~0.30% 141: ~0.30% 142: ~0.30% 143: ~0.30% 144: ~0.40% 145: ~0.30% 146: ~0.30% 147: ~0.30% 148: ~0.30% 149: ~0.30% 150: ~0.30% 151: ~0.70% 152: ~0.30% 153: ~0.30% 154: ~0.30% 155: ~1.30% 156: ~0.30% 157: ~0.40% 158: ~0.30% 159: ~0.30% 160: ~0.30% 161: ~1.50% 162: ~0.30% 163: ~0.30% 164: ~0.30% 165: ~0.30% 166: ~0.30% 167: ~0.30% 168: ~0.30% 169: ~1.50% 170: ~0.30% 171: ~0.30% 172: ~7.20% 173: ~0.30% 174: ~1.00% 175: ~0.30% 176: ~0.30% 177: ~0.30% 178: ~1.80% 179: ~0.30% 180: ~0.50% 181: ~0.70% 182: ~0.30% 183: ~0.30%

• Samples:

  sentence	label
  科目：コンクリート。名称：免震基礎天端ｸﾞﾗｳﾄ注入。	0
  科目：コンクリート。名称：免震基礎天端ｸﾞﾗｳﾄ注入。	0
  科目：コンクリート。名称：免震基礎天端ｸﾞﾗｳﾄ注入。	0

• Loss: sentence_transformer_lib.custom_batch_all_trip_loss.CustomBatchAllTripletLoss

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 512
• per_device_eval_batch_size: 512
• learning_rate: 1e-05
• weight_decay: 0.01
• num_train_epochs: 250
• warmup_ratio: 0.1
• fp16: True
• batch_sampler: group_by_label

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.11.12
• Sentence Transformers: 3.4.1
• Transformers: 4.51.3
• PyTorch: 2.6.0+cu124
• Accelerate: 1.5.2
• Datasets: 3.5.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

CustomBatchAllTripletLoss