This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B on the massive_triplet_v3 dataset. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: Qwen/Qwen3-Embedding-0.6B
Maximum Sequence Length: 32768 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity
Training Dataset:
- massive_triplet_v3

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 32768, 'do_lower_case': False}) with Transformer model: Qwen3Model 
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("CocoRoF/POLAR-Qwen3-0.6b-linq-gist")
# Run inference
sentences = [
    'create list of spiders that obeys the visible projects list, through use of the spider selection menu',
    "def create_spiders_list():\n    spiders_lst = [obj for obj in globals().values() if\n                   inspect.isclass(obj) and str(obj).split('.')[2] == 'spiders' and 'BaseSpider' not in str(obj)]\n    visible_projects = find_visible_projects()\n    spiders_dict = {i.split('.')[0]: [obj for obj in spiders_lst if i.split('.')[0] in str(obj)] for i in\n                    os.listdir('HousingPriceScraper/HousingPriceScraper/spiders/SpiderGroups')[:-1] if i.split('.')[0] in visible_projects}\n    if len(list(spiders_dict.keys())) > 0:\n        spiders_lst = select_spiders(spiders_dict)\n    else:\n        print('There are no visible projects, got to set_visible_projects to set defaults')\n        return False\n    return spiders_lst",
    'def instantiate_pipelines(settings, simulator_settings):\n    pipelines = []\n    # lock to manage race parallel processes race conditions \n    lock = Lock()\n\n    logger.info("\\nVALIDATING PIPELINES\\n")\n    for p_idx, pipeline_settings in enumerate(settings.runs):\n\n        # turn a pipeline off by specifying num_runs as 0\n        num_runs = pipeline_settings.get("num_runs", 0)\n\n        # start_idx determines the first dataset name\'s starting idx\n        start_idx = pipeline_settings.get("start_idx", 0)\n\n        if num_runs:\n            logger.info("Validating run: {}\\n".format(p_idx))\n        else:\n            logger.info("Skipping run: {}\\n".format(p_idx))\n            \n        for idx in range(start_idx, start_idx + num_runs):           \n            logger.info("Pipeline sub index: {}\\n".format(idx))\n            # class factory and instantiate pipeline object\n            Pipeline = pipeline_factory(pipeline_settings["pipeline_name"])\n            p = Pipeline(pipeline_settings, idx, simulator_settings)\n            \n            # give each pipeline an idependent logger\n            log_name = "dSim_{}".format(p.pipeline_settings["dataset_name"])\n            log_path = os.path.join(p.pipeline_settings["outdir"],\n                                    p.pipeline_settings["dataset_name"]+\'.log\')\n            fh = logging.FileHandler(log_path, mode=\'w\')\n            fh.setLevel(logging.DEBUG)\n            format = "%(asctime)-6s: %(name)s - %(levelname)s - %(message)s"\n            fmt = logging.Formatter(format)\n            fh.setFormatter(fmt)\n            local_logger = logging.getLogger(log_name)\n            local_logger.addHandler(fh)\n            logger.info("Init local logging: {}".format(log_path))\n            p.logger = local_logger\n\n            # pipeline/ dataset directory\n            p.pipeline_settings["lock"] = lock\n\n            # validate all submodules for each pipeline is ready (use local logger) \n            p.instantiate_modules()\n\n            # append to list of instantiated pipelines\n            pipelines.append(p)\n    return pipelines',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

massive_triplet_v3

Dataset: massive_triplet_v3 at 51266de
Size: 500,000 training samples
Columns: query, positive, and negative
Approximate statistics based on the first 1000 samples:
query positive negative
type string string string
details
min: 6 tokens
mean: 22.57 tokens
max: 67 tokens
min: 8 tokens
mean: 132.85 tokens
max: 1160 tokens
min: 4 tokens
mean: 122.89 tokens
max: 1758 tokens

Samples:

query	positive	negative
`방학기간에 소외지역의 청소년을 대상으로 청춘누리 봉사단이 할 수 있는 캠프의 이름은 뭐야`	주요 수상기관 교육기부프로그램 개요[object Object]4. 대학생 동아리 ｢청춘누리 봉사단｣[object Object]□ 청춘누리축제[object Object]◦ (참가대상) 전국 유치원, 초·중·고등학생[object Object]◦ (활동내역) 대학생들이 운영하는 교육기부활동을 청소년들이 직접 체험해봄으로써 학생들이 사고력, 창의력 향상을 도모하고 자신의 꿈을 펼칠 수 있는 장 마련[object Object]◦ (주요성과) 대학생들의 교육기부에 대한 전반적인 이해를 돕고 교육 기부 활동의 우수성 홍보[object Object]□ 청춘누리봉사단과 함께하는 교육기부(쏙쏙캠프, 함성소리)[object Object]◦ (참가대상) 전국의 초·중학생[object Object]◦ (활동내역)[object Object]- 쏙쏙캠프 : 방학을 이용하여 상대적으로 교육기부 혜택이 적은 소외 지역을 방문하여 창의력 체험, 진로체험 등을 제공, 배움의 기회 균등 및 꿈을 찾아주는 활동 전개[object Object]- 함성소리 : 학기중 토요일마다 수도권에 있는 청소년 대상으로 꿈을 설계하고 지원하는 활동 전개[object Object]◦ (주요성과) 소외지역 청소년 대상 배움의 기회를 제공하고 대학생들의 봉사활동을 장려하여 많은 청소년 대상 멘토 활동 전개	개도국에 IT나눔을 실천한 청년들과 아름다운 동행[object Object]□ 미래창조과학부(장관 최문기)와 한국정보화진흥원(원장 장광수)은 12월 18일(수) 오후 2시 10분 과천과학관에서 ｢2013년도 월드프렌즈 IT봉사단 귀국보고대회｣(이하, IT봉사단 귀국보고대회)를 개최하였다.[object Object]o 정부는 2001년부터 현재까지 전 세계 70여개 개도국에 5,158명의 IT봉사단을 파견한 바 있으며, ｢IT봉사단 귀국보고대회｣는 매년 개도국에서 활동하고 온 봉사단원들이 서로의 경험을 공유하고 글로벌 역량을 배양하는 ‘소통'과 ‘협력‘의 장(場)으로 운영되고 있다.[object Object]※ 월드프렌즈(World Frends Korea, WFK) : 우리나라 해외봉사단사업 통합브랜드[object Object]□ 이번 ｢IT봉사단 귀국보고대회｣에는 30개국에 파견되었던 552명의 봉사단원 중 약 300여명의 봉사단원이 참석했으며, 윤종록 제2차관과 주한 외교사절(인도네시아 대사, 코스타리카 대사, 네팔 대사 등)이 참석해 세계의 오지를 누비고 온 봉사단원들을 격려했다.[object Object]o 윤종록 제2차관은 IT봉사단원들에게“귀한경험을 활용하여 대한민국의 이름을 빛내는 사람이 되기를 바란다”는 당부와 함께“정부는 여러분과 같은 젊은이들이 세계를 무대로 능력을 마음껏 발휘할 수 있는 글로벌 플랫폼을 구축하는데 노력할 계획”이라고 덧붙였다.
`Loads sensor filters from an Excel file. Both new style XLSX and oldstyle XLS formats are supported.`	def load_sensor_filters_excel(filename, normalise=False, sheet_names=None):[object Object][object Object] sensor_filters = {}[object Object] with pd.ExcelFile(filename) as excel_file:[object Object] # default is all sheets[object Object] if not sheet_names:[object Object] sheet_names = excel_file.sheet_names[object Object][object Object] for sheet in sheet_names:[object Object] try:[object Object] dataframe = excel_file.parse([object Object] sheet, index_col=0[object Object] ) # the sheet as a DataFrame[object Object] # OK, we have the data frame. Let's process it...[object Object] if not _validate_filter_dataframe(dataframe):[object Object] continue[object Object][object Object] if normalise:[object Object] dataframe = _normalise_dataframe(dataframe)[object Object][object Object] sensor_filters[sheet] = ([object Object] np.array(dataframe.index),[object Object] dataframe.values.transpose(),[object Object] )[object Object][object Object] except xlrd.biffh.XLRDError:[object Object] continue[object Object] # except xlrd.biffh.XLRDError as xlrd_error:[object Object] # TODO: log wa...	def convert_csv(fname):[object Object][object Object] # Make sure this is an Excel file.[object Object] if (not is_excel_file(fname)):[object Object][object Object] # Not Excel, so no sheets.[object Object] return [][object Object][object Object] # Run soffice in listening mode if it is not already running.[object Object] run_soffice()[object Object] [object Object] # TODO: Make sure soffice is running in listening mode.[object Object] # [object Object] [object Object] # Connect to the local LibreOffice server.[object Object] context = connect(Socket(HOST, PORT))[object Object][object Object] # Load the Excel sheet.[object Object] component = get_component(fname, context)[object Object][object Object] # Iterate on all the sheets in the spreadsheet.[object Object] controller = component.getCurrentController()[object Object] sheets = component.getSheets()[object Object] enumeration = sheets.createEnumeration()[object Object] r = [][object Object] pos = 0[object Object] if sheets.getCount() > 0:[object Object] while enumeration.hasMoreElements():[object Object][object Object] # Move to next sheet.[object Object] sheet = enumeration.nextElement()[object Object] name = sheet.getName()[object Object] if (name.count(" ") > 10):[object Object] name = name.replace(" ", "")[object Object] name = fix_file_name(name)[object Object] ...
`Create an additional feature to metadata by counting number of occurrences in data, for a specific element_type`	def create_count_features(metadata, element_type, data, grp_feat, res_feat, feature_suffix):[object Object] feature_name = 'n_'+ element_type + '_modif' + feature_suffix[object Object] newfeature = (data.groupby([grp_feat])[res_feat][object Object] .count()[object Object] .reset_index()[object Object] .fillna(0))[object Object] newfeature.columns = [grp_feat, feature_name][object Object] metadata = pd.merge(metadata, newfeature, on=grp_feat, how="outer").fillna(0)[object Object] return metadata	def test(self):[object Object] count = Counter()[object Object] for example in self.testing_set:[object Object] classification = self.classify(example.attributes)[object Object][object Object] if example.CLASS and classification:[object Object] count['TP'] += 1[object Object] elif not example.CLASS and classification:[object Object] count['FP'] += 1[object Object] elif not example.CLASS and not classification:[object Object] count['TN'] += 1[object Object] elif example.CLASS and not classification:[object Object] count['FN'] += 1[object Object] return count

Loss: CachedGISTEmbedLoss with these parameters:
json
1{'guide': SentenceTransformer( 2 (0): Transformer({'max_seq_length': 40960, 'do_lower_case': False}) with Transformer model: Qwen3Model 3 (1): Pooling({'word_embedding_dimension': 4096, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True}) 4 (2): Normalize() 5), 'temperature': 0.01}

Training Hyperparameters

Non-Default Hyperparameters

overwrite_output_dir: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
gradient_accumulation_steps: 16
learning_rate: 2e-06
weight_decay: 0.01
adam_beta2: 0.99
adam_epsilon: 1e-07
max_grad_norm: 0.3
num_train_epochs: 1.0
warmup_ratio: 0.1
dataloader_num_workers: 16
hub_model_id: CocoRoF/POLAR-Qwen3-0.6b-linq-gist
prompts: ({'query': 'Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:', 'document': ''},)
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: True
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 32
per_device_eval_batch_size: 32
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 16
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-06
weight_decay: 0.01
adam_beta1: 0.9
adam_beta2: 0.99
adam_epsilon: 1e-07
max_grad_norm: 0.3
num_train_epochs: 1.0
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: True
dataloader_num_workers: 16
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
tp_size: 0
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: CocoRoF/POLAR-Qwen3-0.6b-linq-gist
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: ({'query': 'Instruct: Given a web search query, retrieve relevant passages that answer the query\nQuery:', 'document': ''},)
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional

Training Logs

Click to expand

Epoch	Step	Training Loss
0.0082	1	2.0699
0.0164	2	1.7826
0.0246	3	1.9799
0.0328	4	8.1569
0.0410	5	4.641
0.0492	6	4.847
0.0573	7	8.2247
0.0655	8	8.9525
0.0737	9	4.2975
0.0819	10	6.3088
0.0901	11	5.6983
0.0983	12	4.3867
0.1065	13	6.1817
0.1147	14	6.0226
0.1229	15	15.2869
0.1311	16	11.8965
0.1393	17	9.4219
0.1475	18	5.9216
0.1557	19	6.5436
0.1639	20	5.4599
0.1720	21	4.6468
0.1802	22	4.9366
0.1884	23	4.5267
0.1966	24	4.9044
0.2048	25	4.9682
0.2130	26	4.1537
0.2212	27	4.0729
0.2294	28	3.9093
0.2376	29	3.3863
0.2458	30	3.9228
0.2540	31	2.8689
0.2622	32	3.3243
0.2704	33	2.7494
0.2785	34	3.108
0.2867	35	3.1585
0.2949	36	3.2985
0.3031	37	2.7137
0.3113	38	2.8327
0.3195	39	2.7932
0.3277	40	3.038
0.3359	41	2.769
0.3441	42	2.7036
0.3523	43	3.1873
0.3605	44	2.5984
0.3687	45	2.6836
0.3769	46	3.0616
0.3850	47	2.87
0.3932	48	2.5225
0.4014	49	2.3775
0.4096	50	2.3407
0.4178	51	2.6313
0.4260	52	2.6966
0.4342	53	2.3673
0.4424	54	2.4391
0.4506	55	2.5654
0.4588	56	2.2967
0.4670	57	2.4656
0.4752	58	2.2497
0.4834	59	2.3793
0.4916	60	2.4427
0.4997	61	2.2327
0.5079	62	2.04
0.5161	63	2.2881
0.5243	64	2.0218
0.5325	65	2.3258
0.5407	66	2.1217
0.5489	67	1.9639
0.5571	68	2.1681
0.5653	69	2.1941
0.5735	70	2.1217
0.5817	71	2.1097
0.5899	72	2.1242
0.5981	73	1.9071
0.6062	74	1.8552
0.6144	75	1.8398
0.6226	76	1.9429
0.6308	77	1.6457
0.6390	78	1.656
0.6472	79	1.6597
0.6554	80	1.8188
0.6636	81	2.0348
0.6718	82	1.9511
0.6800	83	1.8009
0.6882	84	1.8279
0.6964	85	1.7993
0.7046	86	1.782
0.7127	87	1.6168
0.7209	88	1.7357
0.7291	89	1.5588
0.7373	90	1.6574
0.7455	91	1.7124
0.7537	92	1.7205
0.7619	93	1.7439
0.7701	94	1.4042
0.7783	95	1.547
0.7865	96	1.5815
0.7947	97	1.4141
0.8029	98	1.3568
0.8111	99	1.5084
0.8193	100	1.4027
0.8274	101	1.4902
0.8356	102	1.317
0.8438	103	1.8041
0.8520	104	1.4397
0.8602	105	1.3406
0.8684	106	1.5127
0.8766	107	1.2449
0.8848	108	1.4508
0.8930	109	1.4171
0.9012	110	1.626
0.9094	111	1.285
0.9176	112	1.2682
0.9258	113	1.5178
0.9339	114	1.3686
0.9421	115	1.227
0.9503	116	1.3685
0.9585	117	1.3253
0.9667	118	1.0893
0.9749	119	1.1753
0.9831	120	1.252
0.9913	121	1.2304
0.9995	122	1.1111

Framework Versions

Python: 3.11.11
Sentence Transformers: 3.4.1
Transformers: 4.51.0
PyTorch: 2.6.0+cu124
Accelerate: 1.6.0
Datasets: 3.3.2
Tokenizers: 0.21.1

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

	query	positive	negative
type	string	string	string
details	min: 6 tokens mean: 22.57 tokens max: 67 tokens	min: 8 tokens mean: 132.85 tokens max: 1160 tokens	min: 4 tokens mean: 122.89 tokens max: 1758 tokens

CocoRoF/POLAR-Q3-0.6b-gist

SentenceTransformer based on Qwen/Qwen3-Embedding-0.6B