This is a sentence-transformers model finetuned from BAAI/bge-base-en. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: BAAI/bge-base-en
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True, 'architecture': 'BertModel'})
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("aaa961/finetuned-bge-base-en-firefox-bugs-pairs")
# Run inference
sentences = [
    "Loss of bookmarks state after OS hibernation User Agent: Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:88.0) Gecko/20100101 Firefox/88.0\n\nSteps to reproduce:\n\nKept Firefox 88.0 open for several days, using hibernate on Windows inbetween.\nCreated several bookmarks using Ctrl-D and Shift-Ctrl-D both for private as well as non-private sites.\nAlso created some bookmarks by dragging from address bar into subfolders of bookmark window.\nI was wondering, that on some windows star icon an address bar did not become blue.\nVerified, that i already had bookmarked this site by pressing Ctrl-D. The appropriate bookmarks subfolder was shown. Star still did not become true.\nAnything else seem ok.\nDid quit Firefox using File->Quit menu command, saving session with all open tabs.\nDid start Firefox with restoring previous session using menu command.\n\n\nActual results:\n\nSome of the bookmarks created above are lost. Can't remember most of them.\nHowever i remember a few sites, which i definitely had created a bookmark for.\nNo bookmark exists.\nAlso there is no entry in history for these sites. Missing entry in history might be due to using private window. (Can't remember if i was using a private window for opening this site.)\n\nI repeatedly had lost both bookmarks and history entries some month ago and then created a new profile. Did not import anything from previous profiles. \n\n\nExpected results:\n\nBookmarks should have been saved as expected.",
    'Messages with triggers cause trigger listeners to initialize multiple times [Currently](https://searchfox.org/mozilla-central/rev/07342ce09126c513540c1c343476e026cfa907bf/browser/components/newtab/lib/ASRouter.jsm#827) we call the `.init` function of every trigger (when defined).\nThis makes sense for `openURL` where we accumulate more URLs but for others it is a bug\n* moments trigger will add multiple setIntervals\n* whatsNew will evaluate multiple times which is wasteful).',
    'Urlbar docs: In index.rst, the Architecture Overview link in the Where to Start section is broken In index.rst, the Architecture Overview link in the Where to Start section links to an unrelated page in the devtools docs.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[ 1.0000, -0.7628, -0.7601],
#         [-0.7628,  1.0000,  0.9988],
#         [-0.7601,  0.9988,  1.0000]])

Evaluation

Metrics

Triplet

Datasets: bge-base-en-eval and bge-base-en-train
Evaluated with TripletEvaluator

Metric	bge-base-en-eval	bge-base-en-train
cosine_accuracy	0.5193	0.5367

Training Details

Training Dataset

Unnamed Dataset

Size: 5,424 training samples
Columns: texts and label
Approximate statistics based on the first 1000 samples:
texts label
type string int
details
min: 17 tokens
mean: 211.84 tokens
max: 511 tokens
1: ~26.20%
2: ~21.70%
3: ~42.90%
4: ~0.80%
5: ~8.40%

Samples:

texts	label
The "X" button of the opt-in modal is only read as "button" using a screen reader [object Object][object Object]- Firefox Beta 97.0b2 (Build ID: 20220111185943)[object Object]- Firefox Nightly 98.a1 (Build ID: 20220111093827)[object Object][object Object][object Object][object Object]- Windows 10 x64[object Object]- Ubuntu 20.04 x64[object Object]- macOS 10.15.7[object Object][object Object][object Object][object Object]- Have Firefox Beta 97.0b2 downloaded on your computer.[object Object]- Have the "browser.search.region" set to "US".[object Object]- Have one of the [object Object] on your computer.[object Object]- Make sure there is no other modal displayed when starting the browser (browser default window, onboarding for new users etc).[object Object]- Have a screen reader application opened.[object Object][object Object][object Object][object Object]1. Open Firefox Beta 97.0b2.[object Object]2. Navigate to the “about:support” page and paste the user.js file into the Profile folder.[object Object]3. Restart the browser.[object Object]4. Focus on the "X" button of the modal.[object Object]5. Listen to what the screen reader application reads.[object Object][object Object]**[Exp...	`1`
Text and radio buttons are overlapping on PDF Tested with:[object Object]Nightly 91.0a1 (2021-06-23)[object Object][object Object]Tested on:[object Object]Win 10[object Object][object Object]Preconditions:[object Object]In about:config, set pdfjs.enableXfa = true[object Object][object Object]Steps:[object Object][object Object]1. Launch Firefox[object Object]2. Open the attached pdf.[object Object]3. Go to "Taille de l'entreprise"[object Object][object Object]Actual result:[object Object]Radio buttons and text are overlapping.[object Object][object Object]Expected result:[object Object]Text and radio buttons should be properly displayed	`1`
opening a new private window shows the old private window start page for a moment with browser.privatebrowsing.felt-privacy-v1, causing a bad perceived performance STR:[object Object][object Object]1. Set browser.privatebrowsing.felt-privacy-v1 to true[object Object]2. Open a private window[object Object][object Object]Actual:[object Object][object Object]The content of the old private window start page is visible during the window creation. This causes a very visible jump of the content and the window creation feels slow. Please see the following screencast:[object Object][object Object][object Object][object Object][object Object]You can change the speed to 0,25 on YouTube to see it even better.[object Object][object Object]Expected:[object Object][object Object]No felt performance issue.	`3`

Loss: BatchSemiHardTripletLoss

Evaluation Dataset

Unnamed Dataset

Size: 1,162 evaluation samples
Columns: texts and label
Approximate statistics based on the first 1000 samples:
texts label
type string int
details
min: 16 tokens
mean: 209.86 tokens
max: 508 tokens
1: ~26.30%
2: ~21.00%
3: ~42.50%
4: ~0.70%
5: ~9.50%

Samples:

texts	label
(Proton) (a11y) Selected tabs almost not to see STR:[object Object][object Object]1. select multiple tabs[object Object][object Object]Expected:[object Object][object Object]You have no difficulties to see your selected tabs.[object Object][object Object]Actual:[object Object][object Object]It's almost not possible (at least to me) to see the selected tabs due to a insufficient contrast between the tab bar color and the tab color.[object Object][object Object]Marking as regression since it has a11y implications compared to pre Proton.	`2`
The Not Now button from the Fakespot Onboarding sidebar is missing the Clicked State [object Object][object Object]* Nightly 118.0a1 (2023-08-18)[object Object][object Object][object Object][object Object]* Nightly 118.0a1 (2023-08-18)[object Object][object Object][object Object][object Object]* ALL[object Object][object Object][object Object][object Object]Set the browser.shopping.experience2023.enabled - TRUE[object Object]Set the toolkit.shopping.useOHTTP - TRUE[object Object][object Object][object Object][object Object]1. Reach about:preferences and turn off feature recommendations.[object Object]2. Reach the Amazon [object Object] link [object Object]3. Click and Hold the Not Now button from the Onboarding Shopping sidebar.[object Object][object Object][object Object][object Object]* The Not now button from the Onboarding Shopping sidebar should change its state when Clicked.[object Object][object Object][object Object][object Object]* The Not now button from the Onboarding Shopping sidebar is missing the Clicked state.[object Object][object Object][object Object][object Object]Not Applicable	`1`
Missing data from table Tested with:[object Object]Nightly 91.0a1 (2021-06-22)[object Object][object Object]Tested on:[object Object]Win 10[object Object][object Object]Preconditions:[object Object]In about:config, set pdfjs.enableXfa = true[object Object][object Object]Steps:[object Object][object Object]1. Launch firefox[object Object]2. Open the attached PDF[object Object][object Object]Actual result:[object Object]No data in table is displayed[object Object][object Object]Expected result:[object Object]A table with data should be displayed	`1`

Loss: BatchSemiHardTripletLoss

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
per_device_train_batch_size: 16
per_device_eval_batch_size: 32
gradient_accumulation_steps: 8
learning_rate: 2e-05
num_train_epochs: 7
warmup_ratio: 0.1
fp16: True
batch_sampler: no_duplicates

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 16
per_device_eval_batch_size: 32
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 8
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 2e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1.0
num_train_epochs: 7
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.1
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: True
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: no_duplicates
multi_dataset_batch_sampler: proportional
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss	Validation Loss	bge-base-en-eval_cosine_accuracy	bge-base-en-train_cosine_accuracy
-1	-1	-	-	0.5073	-
2.3304	100	4.9515	4.9080	-	0.5302
4.6608	200	4.8737	4.9078	-	0.5382
6.9912	300	4.8542	4.9142	-	0.5367
-1	-1	-	-	0.5193	-

Framework Versions

Python: 3.10.10
Sentence Transformers: 5.1.0
Transformers: 4.56.0
PyTorch: 2.7.1+cu128
Accelerate: 1.10.1
Datasets: 4.0.0
Tokenizers: 0.22.0

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

BatchSemiHardTripletLoss

bibtex

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

	texts	label
type	string	int
details	min: 17 tokens mean: 211.84 tokens max: 511 tokens	1: ~26.20% 2: ~21.70% 3: ~42.90% 4: ~0.80% 5: ~8.40%