This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/all-MiniLM-L6-v2
Maximum Sequence Length: 256 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 256, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("CatkinChen/sentence-transformers_all-MiniLM-L6-v2_retrieval_finetuned_2025-03-29_23-45-57")
# Run inference
sentences = [
    'What is the name of the spell that removes light?',
    'Book: 3, Chapter: 17\nPassage: He raised his eyebrows at her; she nodded again and let go. Quietly as they could, they crept out into the hall and up the crumbling staircase. Everything was covered in a thick layer of dust except the floor, where a wide shiny stripe had been made by something being dragged upstairs. They reached the dark landing. "Nox," they whispered together, and the lights at the end of their wands went out. Only one door was open. As they crept toward it, they heard movement from behind it; a low moan, and then a deep, loud purring. They exchanged a last look, a last nod. Wand held tightly before him, Harry kicked the door wide open.',
    'Book: 5, Chapter: 15\nPassage: "Uh ... first year - you saved the Stone from You-Know-Who." "But that was luck," said Harry, "that wasn\'t skill -"\n"Second year," Ron interrupted, "you killed the basilisk and destroyed Riddle." "Yeah, but if Fawkes hadn\'t turned up I -"\n"Third year," said Ron, louder still, "you fought off about a hundred dementors at once -"\n"You know that was a fluke, if the Time-Turner hadn\'t -"\n"Last year," Ron said, almost shouting now, "you fought off You-Know-Who again -"\n"Listen to me!" said Harry, almost angrily, because Ron and Hermione were both smirking now. "Just listen to me, all right? It sounds great when you say it like that, but all that stuff was luck - I didn\'t know what I was doing half the time, I didn\'t plan any of it, I just did whatever I could think of, and I nearly always had help -"\nRon and Hermione were still smirking and Harry felt his temper rise; he wasn\'t even sure why he was feeling so angry. "Don\'t sit there grinning like you know better than I do, I was there, wasn\'t I?" he said heatedly. "I know what went on, all right? And I didn\'t get through any of that because I was brilliant at Defense Against the Dark Arts, I got through it all because - because help came at the right time, or because I guessed right - but I just blundered through it all, I didn\'t have a clue what I was doing - STOP LAUGHING!" The bowl of murtlap essence fell to the floor and smashed. He became aware that he was on his feet, though he couldn\'t remember standing up. Crookshanks streaked away under a sofa; Ron and Hermione\'s smiles had vanished. "You don\'t know what it\'s like!',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Triplet

Dataset: test-eval
Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.8857

Training Details

Training Dataset

Unnamed Dataset

Size: 309 training samples
Columns: sentence_0, sentence_1, and sentence_2
Approximate statistics based on the first 309 samples:
sentence_0 sentence_1 sentence_2
type string string string
details
min: 9 tokens
mean: 17.74 tokens
max: 53 tokens
min: 46 tokens
mean: 187.05 tokens
max: 256 tokens
min: 37 tokens
mean: 165.52 tokens
max: 256 tokens

Samples:

sentence_0	sentence_1	sentence_2
`Who is the supposed criminal believed to be after Harry?`	Book: 3, Chapter: 5[object Object]Passage: "He looks like one good hex would finish him off, doesn't he? Anyway ..." He turned to Harry. "What were you going to tell us?" Harry explained all about Mr. and Mrs. Weasley's argument and the warning Mr. Weasley had just given him. When he'd finished, Ron looked thunderstruck, and Hermione had her hands over her mouth. She finally lowered them to say, "Sirius Black escaped to come after you? Oh, Harry ... you'll have to be really, really careful. Don't go looking for trouble, Harry -"[object Object]"I don't go looking for trouble," said Harry, nettled. "Trouble usually finds me." "How thick would Harry have to be, to go looking for a nutter who wants to kill him?"	`Book: 6, Chapter: 9[object Object]Passage: "I do not take cheek from anyone, Potter ... not even 'the Chosen One.' "[object Object]"That was brilliant, Harry!" chortled Ron, once they were safely on their way to break a short while later. "You really shouldn't have said it," said Hermione, frowning at Ron. "What made you?"`
`In the sixth book, what does Harry see in the Pensieve about Voldemort's mother, and how does it connect to the second book?`	Book: 6, Chapter: 10[object Object]Passage: "Voldemort's grandfather, yes," said Dumbledore. "Marvolo, his son, Morfin, and his daughter, Merope, were the last of the Gaunts, a very ancient Wizarding family noted for a vein of instability and violence that flourished through the generations due to their habit of marrying their own cousins. Lack of sense coupled with a great liking for grandeur meant that the family gold was squandered several generations before Marvolo was born. He, as you saw, was left in squalor and poverty, with a very nasty temper, a fantastic amount of arrogance and pride, and a couple of family heirlooms that he treasured just as much as his son, and rather more than his daughter." "So Merope," said Harry, leaning forward in his chair and staring at Dumbledore, "so Merope was ... Sir, does that mean she was ... Voldemort's mother?" "It does," said Dumbledore. "And it so happens that we also had a glimpse of Voldemort's father. I wonder whether you noticed?" "The Muggle Morfin ...	`Book: 5, Chapter: 35[object Object]Passage: Why?" "Why?" Malfoy sounded incredulously delighted. "Because the only people who are permitted to retrieve a prophecy from the Department of Mysteries, Potter, are those about whom it was made, as the Dark Lord discovered when he attempted to use others to steal it for him." "And why did he want to steal a prophecy about me?" "About both of you, Potter, about both of you ... Haven't you ever wondered why the Dark Lord tried to kill you as a baby?"`
`Who is the charms master who helps with the Triwizard Tournament?`	Book: 7, Chapter: 30[object Object]Passage: "Professor," Harry said, approaching the little Charms master, "Professor, I'm sorry to interrupt, but this is important. Have you got any idea where the diadem of Ravenclaw is?" "- Protego Horribilis - the diadem of Ravenclaw?" squeaked Flitwick. "A little extra wisdom never goes amiss, Potter, but I hardly think it would be much use in this situation!" "I only meant - do you know where it is? Have you ever seen it?" "Seen it? Nobody has seen it in living memory! Long since lost, boy!" Harry felt a mixture of desperate disappointment and panic. What, then, was the Horcrux? "We shall meet you and your Ravenclaws in the Great Hall, Filius!"	`Book: 5, Chapter: 6[object Object]Passage: "What do you want anyway?" George asked. Kreacher's huge eyes darted onto George. "Kreacher is cleaning," he said evasively. "A likely story," said a voice behind Harry.`

Loss: TripletLoss with these parameters:
json
1{ 2 "distance_metric": "TripletDistanceMetric.COSINE", 3 "triplet_margin": 0.5 4}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
num_train_epochs: 10
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 10
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
tp_size: 0
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
dispatch_batches: None
split_batches: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	test-eval_cosine_accuracy
0.5	10	0.8000
1.0	20	0.8071
1.5	30	0.8500
2.0	40	0.8429
2.5	50	0.8714
3.0	60	0.9000
3.5	70	0.8929
4.0	80	0.8929
4.5	90	0.8929
5.0	100	0.9000
5.5	110	0.9000
6.0	120	0.9000
6.5	130	0.8929
7.0	140	0.8857
7.5	150	0.8929
8.0	160	0.8929
8.5	170	0.8857
9.0	180	0.8857
9.5	190	0.8857
10.0	200	0.8857
-1	-1	0.8857

Framework Versions

Python: 3.12.2
Sentence Transformers: 3.4.1
Transformers: 4.50.0
PyTorch: 2.4.1
Accelerate: 1.4.0
Datasets: 2.19.2
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

bibtex

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

	sentence_0	sentence_1	sentence_2
type	string	string	string
details	min: 9 tokens mean: 17.74 tokens max: 53 tokens	min: 46 tokens mean: 187.05 tokens max: 256 tokens	min: 37 tokens mean: 165.52 tokens max: 256 tokens