BGE base Financial Matryoshka

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: BAAI/bge-base-en-v1.5
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • json
• Language: en
• License: apache-2.0

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Information Retrieval

• Datasets: dim_768, dim_512, dim_256, dim_128 and dim_64
• Evaluated with InformationRetrievalEvaluator

Training Details

Training Dataset

json

• Dataset: json
• Size: 6,300 training samples
• Columns: positive and anchor
• Approximate statistics based on the first 1000 samples:

  	positive	anchor
  type	string	string
  details	min: 6 tokens mean: 45.49 tokens max: 371 tokens	min: 7 tokens mean: 20.33 tokens max: 41 tokens

• Samples:

  positive	anchor
  In their 2023 forward-looking statements, Goldman Sachs mentioned that results, financial condition, liquidity, and capital actions may differ, possibly materially, from the anticipated results. Important factors include those described in "Risk Factors" in Part I, Item 1A and "Forward-Looking Statements" in Part I, Item 1.	What factors could potentially alter Goldman Sachs' anticipated financial outcomes according to their 2023 forward-looking statements?
  Visa Direct is part of Visa’s strategy beyond C2B payments and helps facilitate the delivery of funds to eligible cards, deposit accounts and digital wallets across more than 190 countries and territories. Visa Direct supports multiple use cases, such as P2P payments and account-to-account transfers, business and government payouts to individuals or small businesses, merchant settlements and refunds.	What is the purpose of Visa Direct?
  The Company's international operations are subject to different, and sometimes more stringent, legal and regulatory requirements, which vary widely by jurisdiction, including anti-corruption laws; economic sanctions laws; various privacy, insurance, tax, tariff and trade laws and regulations; corporate governance, privacy, data protection (including the EU's General Data Protection Regulation which began to apply across the EU during 2018), data mining, data transfer, labor and employment, intellectual property, consumer protection and investment laws and regulations; discriminatory licensing procedures; compulsory cessions of reinsurance; required localization of records and funds; higher premium and income taxes; limitations on dividends and repatriation of capital; and requirements for local participation in an insurer's ownership.	What types of laws and regulations govern the international operations of a company?

• Loss: MatryoshkaLoss with these parameters:
  json

  1{ 2 "loss": "MultipleNegativesRankingLoss", 3 "matryoshka_dims": [ 4 768, 5 512, 6 256, 7 128, 8 64 9 ], 10 "matryoshka_weights": [ 11 1, 12 1, 13 1, 14 1, 15 1 16 ], 17 "n_dims_per_step": -1 18}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 2
• per_device_eval_batch_size: 1
• learning_rate: 2e-05
• num_train_epochs: 4
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• bf16: True
• tf32: False
• load_best_model_at_end: True
• optim: adamw_torch_fused
• batch_sampler: no_duplicates

All Hyperparameters

Training Logs