SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: Qwen/Qwen3-Embedding-0.6B
• Maximum Sequence Length: 32768 tokens
• Output Dimensionality: 1024 dimensions
• Similarity Function: Cosine Similarity
• Training Dataset:
  • ssf-train-valid-full-synthetic-batch10-cleaned

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

ssf-train-valid-full-synthetic-batch10-cleaned

• Dataset: ssf-train-valid-full-synthetic-batch10-cleaned at 7ed5d68
• Size: 1,125 training samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 1000 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 53 tokens mean: 172.38 tokens max: 364 tokens	min: 54 tokens mean: 163.91 tokens max: 348 tokens	min: 16 tokens mean: 85.45 tokens max: 164 tokens

• Samples:

  anchor	positive	negative
  The Audit Associate/Audit Assistant Associate undertakes specific stages of audit work under supervision. He/She begins to appreciate the underlying principles behind the tasks assigned to him as part of the audit plan. He is also able to make adjustments to the application of skills to improve the work tasks or solve non-complex issues. The Audit Associate/Audit Assistant Associate operates in a structured work environment. He is able to build relationships, work in a team and identify ethical issues with reference to the code of professional conduct and ethics. He is able to select and apply from a range of known solutions to familiar problems and takes responsibility for his own learning and performance. He is a trustworthy and meticulous individual.	The Audit Associate/Audit Assistant Associate is responsible for executing specific stages of the audit process under guidance. They begin to understand the fundamental principles behind their assigned tasks as part of the audit plan. Additionally, they can adapt their skills to enhance work tasks or address simple issues. The Audit Associate/Audit Assistant Associate works within a structured environment, builds relationships, collaborates effectively in a team, and recognizes ethical concerns in accordance with the professional code of conduct. They are capable of selecting and implementing known solutions to familiar challenges and take ownership of their learning and performance. They are a reliable and detail-oriented professional.	The Marketing Coordinator develops promotional strategies and manages social media campaigns for brand awareness. They focus on creating engaging content and analyzing market trends to attract new customers. The Marketing Coordinator collaborates with various departments, ensuring alignment in messaging and objectives. They are responsible for organizing promotional events and measuring the effectiveness of marketing initiatives. They are an innovative and creative thinker.
  The Audit Senior Manager/Audit Manager manages a portfolio of engagements to deliver high quality audit services. He/she also provides leadership on audit engagements which includes client acceptance process, engagement planning, execution and finalisation of an audit engagement. He is fully accountable for the audit engagement and ensures that the engagement progress against budget and timeline is closely monitored. He also serves to develop and maintain long-term client relationships and value-add to the audit firm by identifying new business development opportunities. The Audit Senior Manager/Audit Manager reviews and provides key technical expertise to ensure the quality of audit work performed is in compliance with professional standards and requirements. He contributes towards continuous improvement in audit methodology and process. He will also assume a greater role in professional development activities such as training, staff recruitment and resource planning.	The Audit Senior Manager/Audit Manager is responsible for overseeing a portfolio of audit engagements to deliver high-quality audit services. He/she leads the audit process, which includes client acceptance, planning, execution, and finalization of audits. The Audit Senior Manager/Audit Manager is fully accountable for each engagement and ensures that progress aligns with budget and timelines. Additionally, he/she focuses on developing and maintaining long-term client relationships while identifying opportunities for new business development to enhance the audit firm's value. The role also involves reviewing audit work to ensure compliance with professional standards and contributing to the continuous improvement of audit methodologies and processes. Furthermore, he/she plays a significant role in professional development activities, including training, staff recruitment, and resource planning.	The Marketing Coordinator organizes promotional events and campaigns to enhance brand awareness. He/she collaborates with various teams to create engaging content and ensure consistent messaging across all channels. The Marketing Coordinator is responsible for analyzing market trends and customer feedback to optimize marketing strategies. Additionally, he/she manages social media accounts and develops relationships with influencers to expand the company's reach. The role also involves tracking campaign performance and reporting on key metrics to measure success.
  The Audit Partner/Audit Director is a transformational leader who steers the organisation to achieve its business goals and objectives by formulating technical and strategic directions to drive change. He/She provides strategic vision and leadership to the organisation in order to develop and strengthen organisational capabilities and culture. The Audit Partner/Audit Director is expected to promote new ideas and business solutions that result in extended services to existing clients. He constantly seeks to expand client base and support business development activities. He also establishes consistent and rigorous quality and risk management processes and procedures. The Audit Partner/Audit Director uses a multitude of controls and procedures consisting professional, regulatory, business, economic, social and environmental conditions to manage risk exposure.	The Audit Partner/Audit Director acts as a transformational leader who guides the organization towards achieving its business objectives by developing strategic and technical approaches to facilitate change. He/She offers strategic vision and leadership to enhance the organization's capabilities and culture. The Audit Partner/Audit Director is tasked with introducing innovative ideas and business solutions that extend services to current clients. He actively seeks to grow the client base and supports initiatives for business development. Additionally, he establishes thorough and consistent quality and risk management processes and procedures. The Audit Partner/Audit Director utilizes a variety of controls and procedures that encompass professional, regulatory, business, economic, social, and environmental factors to manage risk exposure.	The Software Developer designs and implements new applications for mobile devices, focusing on user interface and experience. He/She collaborates with teams to ensure that software products meet customer needs and industry standards. The Software Developer is responsible for debugging and troubleshooting issues to enhance product performance. He also conducts code reviews and provides mentorship to junior developers, ensuring high-quality software development practices.

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Evaluation Dataset

ssf-train-valid-full-synthetic-batch10-cleaned

• Dataset: ssf-train-valid-full-synthetic-batch10-cleaned at 7ed5d68
• Size: 282 evaluation samples
• Columns: anchor, positive, and negative
• Approximate statistics based on the first 282 samples:

  	anchor	positive	negative
  type	string	string	string
  details	min: 66 tokens mean: 131.97 tokens max: 308 tokens	min: 63 tokens mean: 127.59 tokens max: 265 tokens	min: 19 tokens mean: 76.96 tokens max: 208 tokens

• Samples:

  anchor	positive	negative
  The Engineering Head (Power) is a subject matter expert on rail power systems. He/She leads the organisation to implement rail power systems maintenance regime and improvement strategies. His duties also include translating and aligning established industry standards into department Key Performance Indicators (KPIs). He possesses a strong understanding of leading engineering practices, operational best practices, industry developments and regulatory requirements and he translates these into organisation practices and performance requirements. He possesses strong leadership skills, is able to cultivate a culture of continuous improvement and demonstrates excellent management skills to achieve the departments operational and functional goals.	The Engineering Head (Power) serves as a subject matter expert on rail power systems, guiding the organization in implementing maintenance regimes and strategies for improvement. His responsibilities also involve translating and aligning established industry standards into departmental Key Performance Indicators (KPIs). With a deep understanding of leading engineering practices, operational best practices, industry developments, and regulatory requirements, he effectively integrates these aspects into organizational practices and performance expectations. He demonstrates strong leadership abilities, fosters a culture of continuous improvement, and showcases excellent management skills to meet the operational and functional objectives of the department.	The Marketing Coordinator develops engaging social media content to enhance brand visibility and customer engagement. He/She collaborates with creative teams to design promotional materials and ensures timely delivery of marketing campaigns. With a keen eye for detail, he/she analyzes customer feedback to optimize future strategies and improve audience reach. Strong communication skills are essential for this role, as well as a passion for market trends and consumer behavior analysis.
  The Engineering Head (Rolling Stock/Engineering Train) is a technical subject matter expert on passenger trains and engineering trains within the organisation. He/She represents the organisation when interfacing with internal and/or external stakeholders to implement enhancement strategies to improve rail reliability and maintainability. His duties also include translating and aligning established industry standards into department Key Performance Indicators (KPIs). He possesses a strong understanding of leading engineering practices, operational best practices, industry developments and regulatory requirements and he translates these into organisation practices and performance requirements. He possesses strong leadership skills, is able to foster a culture of continuous improvement and demonstrates excellent management skills to achieve the departments operational and functional objectives.	The Engineering Head (Rolling Stock/Engineering Train) serves as a technical expert on passenger and engineering trains within the organization. He/She represents the organization in interactions with both internal and external stakeholders to implement strategies aimed at enhancing rail reliability and maintainability. His responsibilities also involve translating established industry standards into department Key Performance Indicators (KPIs). He possesses a deep understanding of leading engineering practices, operational best practices, industry trends, and regulatory obligations, which he effectively integrates into organizational practices and performance standards. He demonstrates strong leadership abilities, promotes a culture of continuous improvement, and exhibits excellent management skills to meet the department's operational and functional goals.	The Marketing Specialist develops creative advertising strategies and manages promotional campaigns for various products. He/She conducts market research to understand consumer behavior and trends, ensuring that marketing efforts are effectively aligned with target audiences. His responsibilities include designing engaging content for social media platforms and collaborating with graphic designers to produce visually appealing materials. He possesses strong analytical skills, is adept at using marketing software, and demonstrates exceptional communication abilities to enhance brand visibility and customer engagement.
  The Engineering Head (Signal and Communications) is a subject matter expert of signal, communication and control systems. He/She drives the organisation's signal, communication and control systems maintenance regime and engineering standards, as well as translating and aligning established industry standards and regulatory requirements into department Key Performance Indicators (KPIs). He maintains a close link with internal and external stakeholders to implement enhancement strategies to improve rail reliability and maintainability. He possesses strong leadership skills, is able to cultivate a culture of continuous improvement through an in-depth appreciation of leading engineering best- practices and industry developments. He also demonstrates excellent management skills to achieve the departments operational and functional objectives.	The Engineering Head (Signal and Communications) is an expert in signal, communication, and control systems. He/She leads the maintenance regime and engineering standards for the organization’s signal, communication, and control systems, while also translating industry standards and regulatory requirements into departmental Key Performance Indicators (KPIs). He maintains strong relationships with both internal and external stakeholders to implement strategies that enhance rail reliability and maintainability. With strong leadership abilities, he fosters a culture of continuous improvement by deeply understanding leading engineering best practices and industry advancements. Additionally, he showcases excellent management skills to fulfill the operational and functional objectives of the department.	The Marketing Specialist develops creative advertising strategies to engage potential customers and enhance brand visibility. He/She works closely with the design team to produce eye-catching promotional materials and ensures that all marketing campaigns align with the company’s vision and values. With a strong focus on market research, he analyzes consumer behavior to optimize outreach efforts and increase customer engagement.

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false 5}

Training Hyperparameters

Non-Default Hyperparameters

• eval_strategy: epoch
• per_device_train_batch_size: 32
• per_device_eval_batch_size: 16
• gradient_accumulation_steps: 16
• learning_rate: 2e-05
• weight_decay: 0.01
• max_grad_norm: 0.5
• num_train_epochs: 5
• lr_scheduler_type: cosine
• warmup_ratio: 0.1
• warmup_steps: 1500
• bf16: True
• tf32: False
• load_best_model_at_end: True
• gradient_checkpointing: True
• batch_sampler: no_duplicates

All Hyperparameters

Training Logs

• The bold row denotes the saved checkpoint.

Framework Versions

• Python: 3.12.11
• Sentence Transformers: 5.1.0
• Transformers: 4.55.0
• PyTorch: 2.8.0+cu128
• Accelerate: 1.10.0
• Datasets: 4.0.0
• Tokenizers: 0.21.4

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss