This is a sentence-transformers model finetuned from Qwen/Qwen3-Embedding-0.6B. It maps sentences & paragraphs to a 1024-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: Qwen/Qwen3-Embedding-0.6B
Maximum Sequence Length: 512 tokens
Output Dimensionality: 1024 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False, 'architecture': 'Qwen3Model'})
  (1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': True, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("sentence_transformers_model_id")
# Run inference
queries = [
    "Instruct: Given a question about a video transcript, retrieve the most relevant transcript passage.\nQuery: How do you maintain an open throat and a lifted soft palate in opera as you ascend in pitch?",
]
documents = [
    "Eeeeee! Too narrow in the throat. Too open, I can't make an E. Eeeeee! And then again. ZE FERE LTE LOSINGIARI DE VOLLO ZE FERE FE-E-E-I In the middle. ZE FERE LTE LOSINGIARI And then later it has E GYDITE. So... Eeeeee! Eeeeee! Eeeeee! Eeeeee! Eeeeee! Eeeeee! Eeeeee! Now I'm at an F sharp. Now comes the caveat. As you are actually getting higher, those closed vowels, now they do slightly open in the mouth.",
    "As artists, we use lines a lot. We use them as placeholders in the early stages of our drawings to measure, indicate perspective, contour, and gesture. And we use them in the final stages of our drawings when we're shading tone or as a powerful visual element that can guide the eye. So bad line quality can quickly ruin a drawing and give off a really bad impression of your skill. You want to avoid building habits of poor line quality. The longer you practice bad lines, the stronger those habits will become and the harder it will be to break out of those bad habits. So I want to get you on the path to develop the good habits. And if you liked this video, make sure to check out the full drawing basics course at proko.com slash drawing. It's designed to give you the foundation you need to draw from observation or from imagination. Number one, avoid the common mistakes. Two very common mistakes are short scratchy lines and chaotic searching lines.",
    "This whole area asserts Ottoman wealth and by the way guys I wore this blue shirt for the Blue Mosque today. Thinking about it now we probably would recommend you do a tour for all of this. To dive deeper into the history there's so much to learn. Day four. Since we did so much the day before we decided to do some mild sightseeing and lunch in the quaint area of Balant. It's wonderfully colourful and reminds us a lot of Bo Carp in Cape Town. Oh my word, it's so cute already. For that evening we booked a Bosphorus boat cruise through Get Your Guide once again. You can't come to Istanbul without cruising down the Bosphorus Strait. Tonight we're on a yacht to eat delicious Turkish food and watch live Turkish performances. I'm so excited and by the way guys look at the view behind me. Oh this is gonna be great. Honestly though it wasn't quite our cup of tea. It was just a little bit too touristy for us. But you might like it. The views were undoubtedly spectacular though. And that was basically the end of our time in Istanbul. There's so much to do in tea in this city. Like you could just spend the whole day counting cats if you wanted to. There's almost half a million strays here. But here's a list of other things that look great too.",
]
query_embeddings = model.encode_query(queries)
document_embeddings = model.encode_document(documents)
print(query_embeddings.shape, document_embeddings.shape)
# [1, 1024] [3, 1024]

# Get the similarity scores for the embeddings
similarities = model.similarity(query_embeddings, document_embeddings)
print(similarities)
# tensor([[0.4922, 0.2266, 0.1797]], dtype=torch.bfloat16)

Evaluation

Metrics

Information Retrieval

Dataset: val_ir
Evaluated with InformationRetrievalEvaluator

Metric	Value
cosine_accuracy@1	0.5238
cosine_accuracy@3	0.746
cosine_accuracy@5	0.836
cosine_accuracy@10	0.9683
cosine_precision@1	0.5238
cosine_precision@3	0.3404
cosine_precision@5	0.2772
cosine_precision@10	0.1857
cosine_recall@1	0.2904
cosine_recall@3	0.4897
cosine_recall@5	0.6495
cosine_recall@10	0.8344
cosine_ndcg@10	0.6498
cosine_mrr@10	0.6582
cosine_map@100	0.5579

Training Details

Training Dataset

Unnamed Dataset

Size: 3,548 training samples
Columns: anchor and positive
Approximate statistics based on the first 1000 samples:
anchor positive
type string string
details
min: 29 tokens
mean: 39.33 tokens
max: 84 tokens
min: 20 tokens
mean: 223.09 tokens
max: 380 tokens

Samples:

anchor	positive
`Instruct: Given a question about a video transcript, retrieve the most relevant transcript passage.[object Object]Query: How to use Hawaii’s local transportation?`	rap for sometimes being a bit too crowded, but sunset, the vibe is great. There's often live music. Everybody's just enjoying watching that sun go down. So it's a great place to start your vacation. You can even hop in one of the catamarans that has beach loading and go out sailing and watch the sunset that way. That's a great way to start. There you go. Gosh, we're good. Oh my gosh. You're lucky you found this channel. Day number two. And guess what? You are up early, my friend, because you have jet lag. Real bad. Real bad. Especially if you have kids with you. Wow. Good luck. So we are going to take advantage of that today and you are going to head up to the North Shore to go snorkeling at Kualima Cove or just enjoy a nice beach day up there. Kualima Cove is at Turtle Bay Resort, which is like five-star luxury resort that's been remodeled, but all beaches in Hawaii are public. And this is a great place to go snorkeling. It's well protected. And yeah, there's public parking. So enjoy ...
`Instruct: Given a question about a video transcript, retrieve the most relevant transcript passage.[object Object]Query: How to use Hawaii’s local transportation?`	So are you going to Oahu, Hawaii and you're either too cheap or broke to get a rental car for your time there? There's no judgment in that question because that was my mentality and you definitely can have an amazing time in Oahu, Hawaii even for like a week or so without any rental car whatsoever. I was there for a month and mostly got by without a rental car so I'm definitely very qualified to give you an amazing seven-day one-week itinerary for Oahu, Hawaii with no rental car. That being said let's start off with day number zero or arrival day. So unless you're a very strong swimmer or taking a very fast speedboat you'll be arriving at the Daniel K. Inouye International Airport. Probably a little bit jet lagged. It's definitely a little bit tired so let's just get to your hotel and check in. I do recommend staying in Waikiki which people might kind of roll their eyes or scoff at because it's very touristic. It's like a little bit of a concrete jungle in the middle of Hawaii built on...
`Instruct: Given a question about a video transcript, retrieve the most relevant transcript passage.[object Object]Query: How to use Hawaii’s local transportation?`	do you want to walk there and burn out your legs a little bit? It's up to you, depends on how much energy you have to spare. Either way the hike isn't so difficult. It costs about five dollars to enter, walking that is, which you'll be doing because you don't have a car of course, and I believe nowadays you have to either reserve a time slot or something like that. It's also very busy so the earlier you go the better. The hike itself is really cool, it's not too difficult like I said, you go through a little bit of a tunnel and at the end you will get some amazing views of Waikiki, Waikiki Beach, and some other parts of the island which you're probably super excited to explore which you'll definitely do on other days besides today. So once you finish hiking Diamond Head, return back down the way you came, it's an out and back trail, and head to the nearby neighborhood called Kaimuki. Now Kaimuki is a really cool and trendy sort of hip and sort of like a college town feel neighborhood w...

Loss: MultipleNegativesRankingLoss with these parameters:
json
1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false, 5 "directions": [ 6 "query_to_doc" 7 ], 8 "partition_mode": "joint", 9 "hardness_mode": null, 10 "hardness_strength": 0.0 11}

Training Hyperparameters

Non-Default Hyperparameters

num_train_epochs: 10
learning_rate: 0.0002
warmup_steps: 0.1
gradient_accumulation_steps: 4
eval_strategy: steps
per_device_eval_batch_size: 32
load_best_model_at_end: True
remove_unused_columns: False

All Hyperparameters

Click to expand

per_device_train_batch_size: 8
num_train_epochs: 10
max_steps: -1
learning_rate: 0.0002
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_steps: 0.1
optim: adamw_torch_fused
optim_args: None
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
optim_target_modules: None
gradient_accumulation_steps: 4
average_tokens_across_devices: True
max_grad_norm: 1.0
label_smoothing_factor: 0.0
bf16: False
fp16: False
bf16_full_eval: False
fp16_full_eval: False
tf32: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
use_liger_kernel: False
liger_kernel_config: None
use_cache: False
neftune_noise_alpha: None
torch_empty_cache_steps: None
auto_find_batch_size: False
log_on_each_node: True
logging_nan_inf_filter: True
include_num_input_tokens_seen: no
log_level: passive
log_level_replica: warning
disable_tqdm: False
project: huggingface
trackio_space_id: trackio
eval_strategy: steps
per_device_eval_batch_size: 32
prediction_loss_only: True
eval_on_start: False
eval_do_concat_batches: True
eval_use_gather_object: False
eval_accumulation_steps: None
include_for_metrics: []
batch_eval_metrics: False
save_only_model: False
save_on_each_node: False
enable_jit_checkpoint: False
push_to_hub: False
hub_private_repo: None
hub_model_id: None
hub_strategy: every_save
hub_always_push: False
hub_revision: None
load_best_model_at_end: True
ignore_data_skip: False
restore_callback_states_from_checkpoint: False
full_determinism: False
seed: 42
data_seed: None
use_cpu: False
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_pin_memory: True
dataloader_persistent_workers: False
dataloader_prefetch_factor: None
remove_unused_columns: False
label_names: None
train_sampling_strategy: random
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
ddp_backend: None
ddp_timeout: 1800
fsdp: []
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
deepspeed: None
debug: []
skip_memory_metrics: True
do_predict: False
resume_from_checkpoint: None
warmup_ratio: None
local_rank: -1
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	val_ir_cosine_ndcg@10
0.3604	20	0.9475	-
0.7207	40	0.8095	-
1.0721	60	0.5994	-
1.4324	80	0.4689	-
1.7928	100	0.5015	0.6131
2.1441	120	0.4094	-
2.5045	140	0.3802	-
2.8649	160	0.3823	-
3.2162	180	0.3307	-
3.5766	200	0.3340	0.6115
3.9369	220	0.3041	-
4.2883	240	0.2975	-
4.6486	260	0.3046	-
5.0	280	0.2837	-
5.3604	300	0.2595	0.6423
5.7207	320	0.2558	-
6.0721	340	0.2673	-
6.4324	360	0.2468	-
6.7928	380	0.2649	-
7.1441	400	0.2285	0.6564
7.5045	420	0.2203	-
7.8649	440	0.2180	-
8.2162	460	0.1992	-
8.5766	480	0.1888	-
8.9369	500	0.1936	0.6498
9.2883	520	0.2144	-
9.6486	540	0.2128	-
10.0	560	0.2116	-

The bold row denotes the saved checkpoint.

Framework Versions

Python: 3.12.12
Sentence Transformers: 5.3.0
Transformers: 5.2.0
PyTorch: 2.9.0+cu126
Accelerate: 1.12.0
Datasets: 4.0.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

MultipleNegativesRankingLoss