This is a sentence-transformers model finetuned from google/bert_uncased_L-2_H-128_A-2. It maps sentences & paragraphs to a 128-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: google/bert_uncased_L-2_H-128_A-2
Maximum Sequence Length: 128 tokens
Output Dimensionality: 128 dimensions
Similarity Function: Cosine Similarity
Supported Modality: Text

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 128, 'pooling_mode': 'mean', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("swardiantara/bert-tiny-yelp-k1-fixed-euclidean")
# Run inference
sentences = [
    "What a great time!  This place was awesome!  I wish I lived right down the road, because I could see myself being a regular here.\\n\\nHonestly, these guys have got some talent, like real talent.  They can play the piano, sing, play guitar, and the drums.  Did I mention that they are all pretty hilarious, too.  I'd easily pay the $7 cover to get in here any day because it really is worth it.  \\n\\nI am sure I will be back in no time at all.  Next time, I am bringing a big group of friends.",
    "While we were out in Tempe, the husband and I were at a complete loss as to where we should eat.  I did a little yelping and sheepishly mentioned this place.  Now, we like country music and all, but I steer very clear of Toby Keith stuff.  I don't care for the in your face American Pride type stuff.  Anyway, we made out way to the restaurant, with entrances in front and back, and smelled the delicious bbq aroma floating in the air.\\n\\nThis place is HUGE!  I noticed that the capacity is just about 1000 people.  That is nuts.  there is a huge bar running down the middle and a ton of TV's spaced no more than 4 feet apart running all along the bar and all along the roof.  It's a good place to watch a game or the music videos.  I feel like I should also add that there is apparently a dress code for the servers.  All the girls are supposed to wear jingly black cowboy boots, super short denim shorts or skirts, skimpy tops that sometimes show too much skin on a less that perfectly built body, and a super sparkly rhinestoned out belt with a big ol buckle.\\n\\nAs far as the menu goes, the layout and look of it kind of suck.  I heard from our waitress that they are re-doing it, so let's hope that it will be for the better.  Almost everything on the menu actually looks like something I would order.  It's all pretty simple BBQ style stuff with some sandwiches and salads and some pricier entrees.  The appetizer menu looked delicious all around.  I laughed out loud when I read that there were deep fried twinkies on the dessert menu.  Classic.  I ordered the Oklahoma BBQ Beef sandwich with coleslaw as my side.  The sandwich was very filling and tasty.  The meat was juicy and the sauce was sweet and tangy.  The bun could have been more substantial but it was more than made up for with the yummy fried onion strings.  The coleslaw was just blah but I sampled the husband's potato salad and it was delicious!  Simple and tasty.\\n\\nOverall experience was pretty good.  Our waitress was nice and gave an honest opinion when we asked for one.  The atmosphere was kind of dull but that would probably be because we were there at such an odd time.  In case you were wondering, in the 30-45 minutes we were there, I counted 6 Toby Keith songs.  If I were into guns and in your face patriotism I would probably give this place 4 stars.",
    "First Watch on Black Canyon Hwy in Phoenix has amazing service and very good food! They're worth the drive. You won't be disappointed. Coincidentally, I went there b/c it was right next door to the Courtyard Marriott. I have been back several times and continue to receive excellent customer service and delicious food. \\n\\nMy favorite breakfast is their Tri-Athlete omelette; which I hesitatingly tried b/c it was a healthy choice but fell in love with it! Wonderfully roasted vegetables in an egg-white only omelette topped with salsa...muahhh, magnifico!!\\n\\nFor lunch I had their Pecan Dijon chicken salad...wow, another amazing dish full of flavors...good to the last drop!\\n\\nAnd lastly, their customer service...always SPOT ON! Once we had only 15 minutes. Informed the always-welcoming staff and VOILA, an excellent breakfast in exactly 15 minutes. I'll be back soon! \\n\\nThank you First Watch on Black Canyon Hwy!! You're awesome :)",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 128]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.9926, 0.9997],
#         [0.9926, 1.0000, 0.9910],
#         [0.9997, 0.9910, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 2,925,010 training samples
Columns: text_a, text_b, and label
Approximate statistics based on the first 100 samples:
text_a text_b label
type string string list
modality text text
details
min: 6 tokens
mean: 95.69 tokens
max: 128 tokens
min: 128 tokens
mean: 128.0 tokens
max: 128 tokens
size: 2 elements

Samples:

text_a	text_b	label
dr. goldberg offers everything i look for in a general practitioner. he's nice and easy to talk to without being patronizing; he's always on time in seeing his patients; he's affiliated with a top-notch hospital (nyu) which my parents have explained to me is very important in case something happens and you need surgery; and you can get referrals to see specialists without having to see him first. really, what more do you need? i'm sitting here trying to think of any complaints i have about him, but i'm really drawing a blank.	First Watch on Black Canyon Hwy in Phoenix has amazing service and very good food! They're worth the drive. You won't be disappointed. Coincidentally, I went there b/c it was right next door to the Courtyard Marriott. I have been back several times and continue to receive excellent customer service and delicious food. \n\nMy favorite breakfast is their Tri-Athlete omelette; which I hesitatingly tried b/c it was a healthy choice but fell in love with it! Wonderfully roasted vegetables in an egg-white only omelette topped with salsa...muahhh, magnifico!!\n\nFor lunch I had their Pecan Dijon chicken salad...wow, another amazing dish full of flavors...good to the last drop!\n\nAnd lastly, their customer service...always SPOT ON! Once we had only 15 minutes. Informed the always-welcoming staff and VOILA, an excellent breakfast in exactly 15 minutes. I'll be back soon! \n\nThank you First Watch on Black Canyon Hwy!! You're awesome :)	`[1.0, 0.0]`
dr. goldberg offers everything i look for in a general practitioner. he's nice and easy to talk to without being patronizing; he's always on time in seeing his patients; he's affiliated with a top-notch hospital (nyu) which my parents have explained to me is very important in case something happens and you need surgery; and you can get referrals to see specialists without having to see him first. really, what more do you need? i'm sitting here trying to think of any complaints i have about him, but i'm really drawing a blank.	No, it's not worth it! Bellagio overcharged me and their accounting department never got back to me (I asked them for an itemized receipt 2 weeks ago). When I called them, I was (twice!) put on this music-less "dead" hold for 30-40 minutes. They're so full of themselves, they don't provide good customer service. Their wi-fi needed fixing when I stayed there. And by the way, the woman at the buffet was super rude to me. Finally, the famous waterworks never happened while I was standing outside in the cold waiting (though i asked the staff and they said they're up and running). No, don't waste your money! I won't and won't let my family and friends do that either.	`[0.0, 1.0]`
dr. goldberg offers everything i look for in a general practitioner. he's nice and easy to talk to without being patronizing; he's always on time in seeing his patients; he's affiliated with a top-notch hospital (nyu) which my parents have explained to me is very important in case something happens and you need surgery; and you can get referrals to see specialists without having to see him first. really, what more do you need? i'm sitting here trying to think of any complaints i have about him, but i'm really drawing a blank.	Really, I should know better...\n\nIt was late at night, we didn't feel like fast food burger places, and I have driven past this place millions of times. \n\nIt's in a dark scary lot too close for comfort to Greenway square. Probably not the best place for a woman to go by herself late at night, but luckily I had the husband with me.\n\nAfter looking at the extensive menu, I ordered the chili relleno plate and my husband ordered a chicken burrito, beef taco, and a horchata. \n\nI couldn't finish my chili relleno plate. (Quick lesson in Mexican cooking- when preparing anything with chilis, its a good idea to remove the majority of the seeds. Not just because of the heat, but because its NOT fun to have a mouthful of seeds.) Besides the hundreds of chewy seeds in my mouth, the chili rellenos were lukewarm. The whole plate was pretty bland, and with every bite I took I knew my stomach would be paying for it later. So I only ate half, then popped 3 tums and hoped for the best. I ended not...	`[0.0, 0.75]`

Loss: [object Object].OrdinalProxyContrastiveLoss

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 1024
num_train_epochs: 10
learning_rate: 2e-05
load_best_model_at_end: True

All Hyperparameters

Click to expand

per_device_train_batch_size: 1024
num_train_epochs: 10
max_steps: -1
learning_rate: 2e-05
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_steps: 0
optim: adamw_torch
optim_args: None
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
optim_target_modules: None
gradient_accumulation_steps: 1
average_tokens_across_devices: True
max_grad_norm: 1.0
label_smoothing_factor: 0.0
bf16: False
fp16: False
bf16_full_eval: False
fp16_full_eval: False
tf32: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
use_liger_kernel: False
liger_kernel_config: None
use_cache: False
neftune_noise_alpha: None
torch_empty_cache_steps: None
auto_find_batch_size: False
log_on_each_node: True
logging_nan_inf_filter: True
include_num_input_tokens_seen: no
log_level: passive
log_level_replica: warning
disable_tqdm: False
project: huggingface
trackio_space_id: None
trackio_bucket_id: None
trackio_static_space_id: None
per_device_eval_batch_size: 8
prediction_loss_only: True
eval_on_start: False
eval_do_concat_batches: True
eval_use_gather_object: False
eval_accumulation_steps: None
include_for_metrics: []
batch_eval_metrics: False
save_only_model: False
save_on_each_node: False
enable_jit_checkpoint: False
push_to_hub: False
hub_private_repo: None
hub_model_id: None
hub_strategy: every_save
hub_always_push: False
hub_revision: None
load_best_model_at_end: True
ignore_data_skip: False
restore_callback_states_from_checkpoint: False
full_determinism: False
seed: 42
data_seed: None
use_cpu: False
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_pin_memory: True
dataloader_persistent_workers: False
dataloader_prefetch_factor: None
remove_unused_columns: True
label_names: None
train_sampling_strategy: random
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
ddp_static_graph: None
ddp_backend: None
ddp_timeout: 1800
fsdp: None
fsdp_config: None
deepspeed: None
debug: []
skip_memory_metrics: True
do_predict: False
resume_from_checkpoint: None
warmup_ratio: None
local_rank: -1
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
0.1750	500	0.3914
0.3500	1000	0.1428
0.5250	1500	0.1338
0.7000	2000	0.1296
0.8750	2500	0.1263
1.0	2857	-
1.0501	3000	0.1249
1.2251	3500	0.1228
1.4001	4000	0.1219
1.5751	4500	0.1213
1.7501	5000	0.1202
1.9251	5500	0.1194
2.0	5714	-
2.1001	6000	0.1184
2.2751	6500	0.1182
2.4501	7000	0.1177
2.6251	7500	0.1170
2.8001	8000	0.1169
2.9751	8500	0.1164
3.0	8571	-
3.1502	9000	0.1156
3.3252	9500	0.1150
3.5002	10000	0.1151
3.6752	10500	0.1150
3.8502	11000	0.1150
4.0	11428	-
4.0252	11500	0.1144
4.2002	12000	0.1139
4.3752	12500	0.1139
4.5502	13000	0.1133
4.7252	13500	0.1132
4.9002	14000	0.1131
5.0	14285	-
5.0753	14500	0.1126
5.2503	15000	0.1125
5.4253	15500	0.1123
5.6003	16000	0.1119
5.7753	16500	0.1125
5.9503	17000	0.1119
6.0	17142	-
6.1253	17500	0.1112
6.3003	18000	0.1113
6.4753	18500	0.1111
6.6503	19000	0.1115
6.8253	19500	0.1111
7.0	19999	-
7.0004	20000	0.1110
7.1754	20500	0.1106
7.3504	21000	0.1103
7.5254	21500	0.1104
7.7004	22000	0.1107
7.8754	22500	0.1107
8.0	22856	-
8.0504	23000	0.1103
8.2254	23500	0.1099
8.4004	24000	0.1098
8.5754	24500	0.1101
8.7504	25000	0.1102
8.9254	25500	0.1098
9.0	25713	-
9.1005	26000	0.1101
9.2755	26500	0.1097
9.4505	27000	0.1097
9.6255	27500	0.1100
9.8005	28000	0.1099
9.9755	28500	0.1096
10.0	28570	-

The bold row denotes the saved checkpoint.

Training Time

Training: 2.2 hours
Evaluation: 2.3 minutes
Total: 2.3 hours

Framework Versions

Python: 3.12.4
Sentence Transformers: 5.5.1
Transformers: 5.11.0
PyTorch: 2.5.1+cu121
Accelerate: 1.13.0
Datasets: 2.21.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

	text_a	text_b	label
type	string	string	list
modality	text	text
details	min: 6 tokens mean: 95.69 tokens max: 128 tokens	min: 128 tokens mean: 128.0 tokens max: 128 tokens	size: 2 elements