SentenceTransformer based on sentence-transformers/all-MiniLM-L12-v2

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/all-MiniLM-L12-v2
• Maximum Sequence Length: 128 tokens
• Output Dimensionality: 384 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • pre-finetune

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 128, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

pre-finetune

• Dataset: pre-finetune at 5e8c10c
• Size: 14,388 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 120 tokens mean: 127.97 tokens max: 128 tokens	min: 21 tokens mean: 122.36 tokens max: 128 tokens

• Samples:

  anchor	positive
  The ETF Series Solutions AAM Transf (TRFM) employs a passive management strategy to track the Pence Transformers Index, focusing on U.S.-listed equities, including ADRs, that are set to benefit from significant disruptions in consumer behavior and technological innovation. The index is rules-based and modified equal-weighted, emphasizing companies with substantial R&D spending. It targets sectors like autonomous driving, electric vehicles, the digital economy, 5G, low-carbon technologies, and renewable energy. Eligible companies must have a market cap of at least $2 billion and meet analyst rating criteria. The portfolio is tiered by market cap and domicile, with 75% allocated to U.S. companies, and is reconstituted quarterly.	Nova Ltd. designs, develops, produces, and sells process control systems used in the manufacture of semiconductors in Israel, Taiwan, the United States, China, Korea, and internationally. Its product portfolio includes a set of metrology platforms for dimensional, films, and materials and chemical metrology measurements for process control for various semiconductor manufacturing process steps, including lithography, etch, chemical mechanical planarization, deposition, electrochemical plating, and advanced packaging. The company serves various sectors of the integrated circuit manufacturing industry, including logic, foundries, and memory manufacturers, as well as process equipment manufacturers. Nova Ltd. was formerly known as Nova Measuring Instruments Ltd. and changed its name to Nova Ltd. in July 2021. The company was incorporated in 1993 and is headquartered in Rehovot, Israel.
  The U.S. Global Jets ETF (JETS) employs a passive management strategy to track the U.S. Global Jets Index, focusing on U.S. and international airline companies, including passenger airlines, aircraft manufacturers, and airport services. The fund is non-diversified and uses a tiered weighting scheme primarily based on market cap and passenger load. Approximately 70% of its portfolio is allocated to large-cap U.S. passenger airlines, with the top four companies receiving 10% each. The next five largest U.S. or Canadian airlines receive 4% each, while other companies meeting trading and liquidity criteria are weighted based on fundamental factors like cash flow return on capital and sales growth.	United Airlines Holdings, Inc., through its subsidiaries, provides air transportation services in North America, Asia, Europe, Africa, the Pacific, the Middle East, and Latin America. The company transports people and cargo through its mainline and regional fleets. It also offers catering, ground handling, training, and maintenance services for third parties. The company was formerly known as United Continental Holdings, Inc. and changed its name to United Airlines Holdings, Inc. in June 2019. United Airlines Holdings, Inc. was incorporated in 1968 and is headquartered in Chicago, Illinois.
  The SPDR S&P Bank ETF (KBE) aims to deliver investment results that correspond to the total return performance of the S&P Banks Select Industry Index, which is part of the S&P Total Market Index tracking the broad U.S. equity market. KBE invests at least 80% of its total assets in securities within this index, focusing on the bank segment, including sub-industries like Asset Management & Custody Banks, Diversified Banks, Regional Banks, Other Diversified Financial Services, and Thrifts & Mortgage Finance. The fund employs an equal-weighted strategy, rebalancing quarterly to ensure equal emphasis on both large and small banking firms, thus providing diversified exposure across the banking sector. Additionally, KBE may hold equity securities outside the index, cash, and money market instruments to maintain liquidity and flexibility.	MGIC Investment Corporation, through its subsidiaries, provides private mortgage insurance, other mortgage credit risk management solutions, and ancillary services to lenders and government sponsored entities in the United States, Puerto Rico, and Guam. The company offers primary mortgage insurance that provides mortgage default protection on individual loans, as well as covers unpaid loan principal, delinquent interest, and various expenses associated with the default and subsequent foreclosure. It also provides contract underwriting services, as well as reinsurance. The company serves originators of residential mortgage loans, including savings institutions, commercial banks, mortgage brokers, credit unions, mortgage bankers, and other lenders. MGIC Investment Corporation was founded in 1957 and is headquartered in Milwaukee, Wisconsin.

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Evaluation Dataset

pre-finetune

• Dataset: pre-finetune at 5e8c10c
• Size: 3,597 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 1000 samples:

  	anchor	positive
  type	string	string
  details	min: 120 tokens mean: 127.99 tokens max: 128 tokens	min: 22 tokens mean: 122.47 tokens max: 128 tokens

• Samples:

  anchor	positive
  The Goldman Sachs Future Health Care ETF (GDOC) seeks long-term capital growth by investing at least 80% of its net assets in equity investments of U.S. and non-U.S. healthcare companies. This actively managed, non-diversified fund targets innovators and disruptors in the healthcare sector, focusing on key themes such as genomics, precision medicine, technology-enabled procedures, and digital healthcare. GDOC may invest in companies of any market capitalization and may use derivatives like futures and options to achieve its investment goals. The fund's adviser employs a fundamental investment process that may integrate ESG factors, utilizing company disclosures, third-party research, and engagement to inform decisions. The fund's thematic allocations can vary over time at the adviser's discretion.	Insulet Corporation develops, manufactures, and sells insulin delivery systems for people with insulin-dependent diabetes. It offers Omnipod System, a self-adhesive disposable tubeless Omnipod device that is worn on the body for up to three days at a time, as well as its wireless companion, the handheld personal diabetes manager. The company sells its products primarily through independent distributors and pharmacy channels, as well as directly in the United States, Canada, Europe, the Middle East, and Australia. Insulet Corporation was incorporated in 2000 and is headquartered in Acton, Massachusetts.
  The J.P. Morgan Exchange-Traded Fund (JPRE) aims to provide high total investment return through capital appreciation and current income by investing at least 80% of its net assets in equity securities of real estate investment trusts (REITs), including both equity and mortgage REITs across various market capitalizations. As an actively managed, non-diversified fund, JPRE focuses on U.S. REITs with strong financials, operating revenues, and growth potential. The fund employs a disciplined investment process, evaluating securities based on their ability to generate long-term earnings and growth, while also considering ESG factors. On May 20, 2022, JPRE acquired the assets and liabilities of the JPMorgan Realty Income Fund, which had $2.2 billion in assets, enhancing its investment strategy and historical performance data.	Extra Space Storage Inc., headquartered in Salt Lake City, Utah, is a self-administered and self-managed REIT and a member of the S&P 500. As of September 30, 2020, the Company owned and/or operated 1,906 self-storage stores in 40 states, Washington, D.C. and Puerto Rico. The Company's stores comprise approximately 1.4 million units and approximately 147.5 million square feet of rentable space. The Company offers customers a wide selection of conveniently located and secure storage units across the country, including boat storage, RV storage and business storage. The Company is the second largest owner and/or operator of self-storage stores in the United States and is the largest self-storage management company in the United States.
  The First Trust Indxx Metaverse ETF (ARVR) aims to replicate the performance of the Indxx Metaverse Index, investing at least 80% of its net assets in securities within the index. This non-diversified fund targets companies globally that are integral to the Metaverse, focusing on those generating at least 50% of their revenue from five key sub-themes: IP & Contents, Platforms, Payment, Optics & Display, and Semiconductor, Hardware & 5G. The portfolio, comprising 50 companies selected by market-cap, is weighted using revenue thresholds, favoring firms with higher Metaverse-related revenue. Stocks are equally weighted, capped at 2%, and adjusted for market-cap, with the index rebalanced quarterly and reconstituted semi-annually.	Adobe Inc. operates as a diversified software company worldwide. It operates through three segments: Digital Media, Digital Experience, and Publishing and Advertising. The Digital Media segment offers products, services, and solutions that enable individuals, teams, and enterprises to create, publish, and promote content; and Document Cloud, a unified cloud-based document services platform. Its flagship product is Creative Cloud, a subscription service that allows members to access its creative products. This segment serves content creators, workers, marketers, educators, enthusiasts, communicators, and consumers. The Digital Experience segment provides an integrated platform and set of applications and services that enable brands and businesses to create, manage, execute, measure, monetize, and optimize customer experiences from analytics to commerce. This segment serves marketers, advertisers, agencies, publishers, merchandisers, merchants, web analysts, data scientists, developers, ...

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: steps
• num_train_epochs: 10
• warmup_ratio: 0.1
• bf16: True
• dataloader_drop_last: True
• load_best_model_at_end: True
• batch_sampler: no_duplicates

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 4.0.2
• Transformers: 4.51.2
• PyTorch: 2.1.0+cu118
• Accelerate: 1.6.0
• Datasets: 3.5.0
• Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss