SentenceTransformer based on BAAI/bge-large-en-v1.5

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-large-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 1024 tokens
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Semantic Similarity

• Evaluated with EmbeddingSimilarityEvaluator

Training Details

Training Dataset

Unnamed Dataset

• Size: 132 training samples
• Columns: sentence1, sentence2, and score
• Approximate statistics based on the first 1000 samples:

  	sentence1	sentence2	score
  type	string	string	float
  details	min: 10 tokens mean: 41.99 tokens max: 126 tokens	min: 14 tokens mean: 42.72 tokens max: 162 tokens	min: 0.25 mean: 0.93 max: 1.0

• Samples:

  sentence1	sentence2	score
  "Gross receipts as defined in Section 952.3 shall not include receipts from any sales of real property with respect to which the Real Property Transfer Tax imposed by Article 12-C has been paid to the City."	"Receipts from the sale of real property are exempt from the gross receipts tax if the Real Property Transfer Tax imposed by Article 12-C has been paid to the City."	1.0
  For tax years beginning on or after January 1, 2025, any person or combined group, except for a lessor of residential real estate, whose gross receipts within the City did not exceed $5,000,000, adjusted annually in accordance with the increase in the Consumer Price Index: All Urban Consumers for the San Francisco/Oakland/Hayward Area for All Items as reported by the United States Bureau of Labor Statistics, or any successor to that index, as of December 31 of the calendar year two years prior to the tax year, beginning with tax year 2026, and rounded to the nearest $10,000.	For taxable years ending on or before December 31, 2024, using the rules set forth in Sections 956.1 and 956.2, in the manner directed in Sections 953.1 through 953.7, inclusive, and in Section 953.9 of this Article 12-A-1; and	0.95
  "San Francisco Gross Receipts" refers to the revenue generated from sales and services within the city limits of San Francisco.	"Revenue generated from sales and services within the city limits of San Francisco"	1.0

• Loss: CoSENTLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "pairwise_cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 5
• warmup_ratio: 0.1
• fp16: True

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.0.1
• Transformers: 4.42.0.dev0
• PyTorch: 2.3.0+cu121
• Accelerate: 0.31.0
• Datasets: 2.19.2
• Tokenizers: 0.19.1

Citation

BibTeX

Sentence Transformers

CoSENTLoss