This is a sentence-transformers model finetuned from sentence-transformers/all-MiniLM-L6-v2. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for retrieval.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: sentence-transformers/all-MiniLM-L6-v2
Maximum Sequence Length: 256 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity
Supported Modality: Text

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 384, 'pooling_mode': 'mean', 'include_prompt': True})
  (2): Normalize({})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ronit01/final_golden_rag_tuned_minilm_contrastive_100epoch")
# Run inference
sentences = [
    'How does RFPromptManager select few-shot examples for each input query?',
    'Stop\n----\n\nThis IC Op earmarks a run to be stopped at the end of its current chunk. \nIt will still be alive but it will not use any GPU resources from the next chunk. \nYou will still see its minibatch-level plots advancing for the current chunk. \nYou cannot stop an already stopped or deleted run. \n\n\n.. raw:: html\n\n    <img src="_static/icop-stop2.png" alt="IC Op Stop" \n         style="cursor: zoom-in; max-width: 100%;" onclick="this.requestFullscreen()">\n\n    <img src="_static/icop-stop.png" alt="IC Op Stop" \n         style="cursor: zoom-in; max-width: 100%;" onclick="this.requestFullscreen()">\n',
    'Eval Compute Metrics Function\n-------------------------\n\nMandatory user-provided function to compute eval metrics on a given batch of (postprocessed) examples, \nwhich is injected by the system. \nIt should return metrics computed over the batch as a whole. \n\nIt is invoked for each batch during the evaluation process after generation and postprocessing (if applicable). \nPass it directly to the :code:`compute_metrics_fn` key in your eval config dictionary.\n\n\n.. py:function:: eval.compute_metrics_fn(batch: dict[str, list]) -> dict[str, dict[str, Any]]\n\n   :param batch: Dictionary containing a batch of examples, including all preprocessed fields, generated outputs, and any postprocessed fields\n   :type batch: dict[str, list]\n\n   :return: Dictionary with a metric\'s name as key and a dictionary as value inside which a reserved key :code:`"value"` must exist with that corresponding metric\'s value over this batch of examples.\n   :rtype: dict[str, dict[str, Any]]\n\n\n**Example:**\n\n.. code-block:: python\n\n    # Example 1 from FiQA use case: Metrics on retrieval accuracy\n    def sample_compute_metrics_fn(batch: Dict[str, list]) -> Dict[str, Dict[str, Any]]:\n\t\t"""Function to compute all eval metrics based on retrievals and/or generations"""\n\n\t\ttrue_positives, precisions, recalls, f1_scores, ndcgs, rrs = 0, [], [], [], [], []\n\t\ttotal_queries = len(batch["query"])\n\n\t\tfor pred, gt in zip(batch["retrieved_documents"], batch["ground_truth_documents"]):\n\t\t\texpected_set = set(gt)\n\t\t\tretrieved_set = set(pred)\n\n\t\t\ttrue_positives = len(expected_set.intersection(retrieved_set))\n\t\t\tprecision = true_positives / len(retrieved_set) if len(retrieved_set) > 0 else 0\n\t\t\trecall = true_positives / len(expected_set) if len(expected_set) > 0 else 0\n\t\t\tf1 = (\n\t\t\t\t2 * precision * recall / (precision + recall)\n\t\t\t\tif (precision + recall) > 0\n\t\t\t\telse 0\n\t\t\t)\n\n\t\t\tprecisions.append(precision)\n\t\t\trecalls.append(recall)\n\t\t\tf1_scores.append(f1)\n\t\t\tndcgs.append(compute_ndcg_at_k(retrieved_set, expected_set, k=5))\n\t\t\trrs.append(compute_rr(retrieved_set, expected_set))\n\n\t\treturn {\n\t\t\t"Total": {"value": total_queries},\n\t\t\t"Precision": {"value": sum(precisions) / total_queries},\n\t\t\t"Recall": {"value": sum(recalls) / total_queries},\n\t\t\t"F1 Score": {"value": sum(f1_scores) / total_queries},\n\t\t\t"NDCG@5": {"value": sum(ndcgs) / total_queries},\n\t\t\t"MRR": {"value": sum(rrs) / total_queries},\n\t\t}\n\n.. code-block:: python\n\n    # Example 2 from GSM8K use case: Direct answer correctness check\n    def sample_compute_metrics_fn(batch: dict[str, list]) -> dict[str, dict[str, Any]]:\n        """Function to compute all eval metrics based on retrievals and/or generations"""\n\n        correct = sum(\n            1\n            for pred, gt in zip(batch["model_answer"], batch["ground_truth"])\n            if pred == gt\n        )\n        total = len(batch["model_answer"])\n        return {\n            "Correct": {"value": correct},\n            "Total": {"value": total},\n        }',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities)
# tensor([[1.0000, 0.4287, 0.3777],
#         [0.4287, 1.0000, 0.3455],
#         [0.3777, 0.3455, 1.0000]])

Training Details

Training Dataset

Unnamed Dataset

Size: 276 training samples
Columns: sentence_0, sentence_1, and label
Approximate statistics based on the first 276 samples:
sentence_0 sentence_1 label
type string string float
details
min: 11 tokens
mean: 30.57 tokens
max: 48 tokens
min: 64 tokens
mean: 231.63 tokens
max: 256 tokens
min: 0.0
mean: 0.17
max: 1.0
Samples:

sentence_0 sentence_1 label
How do you install and initialize RapidFire AI for fine-tuning workflows, and what steps are required to access gated Hugging Face models? Eval Accumulate Metrics Function

Optional user-provided function to aggregate algebraic eval metrics across all batches of the data. If this function is not provided, all metrics returned by :func:eval.compute_metrics_fn() will be assumed to be distributive (i.e., summed across batches) by default. Use this function when metrics require (weighted) averaging or other custom dataset-wide aggregation logic.

It is invoked once at the very end of the evaluation process after all batches have been processed. Pass it directly to the :code:accumulate_metrics_fn key in your eval config dictionary.

.. py:function:: eval.accumulate_metrics_fn(aggregated_metrics: dict[str, list[dict[str, Any]]]) -> dict[str, dict[str, Any]]

:param aggregated_metrics: Dictionary with a metric's name as key and a list of per-batch metric dictionaries as values from across all data batches. Inside each value dictionary, at least the reserved key :code:"value" will exist t... | 0.0 | | What are all the parameters accepted by the RFOpenAIAPIModelConfig class, and what does each one configure? | Clone-Modify
```
[object Object]
[object Object]
```
```
You can also [object Object] a clone using its parent's weights if you'd like.
Warm-started clones inherit their parent's learning behavior so far and thus, they can reach better
eval metrics faster.
Note that warm starting is only allowed if t...
```
| 0.0 | | What rate limiting parameters does RFOpenAIAPIModelConfig provide, and why are they needed? |
```
RapidFire AI transforms the status quo by adapting the powerful idea of [object Object]
from database systems research to LLM evals.
Our adaptive execution engine, :doc:[object Object], automatically
shards the data and processes multiple configs in parallel, one shard at a time, with
efficient swapping techniques.
```
```
[object Object]
[object Object]
[object Object]
```
```
.. list-table::
:widths: 50 50
:class...
```
| 0.0 |
Loss: ContrastiveLoss with these parameters:

json
1{ 2 "distance_metric": "SiameseDistanceMetric.COSINE_DISTANCE", 3 "margin": 0.5, 4 "size_average": true 5}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 16
per_device_eval_batch_size: 16
num_train_epochs: 100
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

do_predict: False
prediction_loss_only: True
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 100
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: None
warmup_ratio: None
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
enable_jit_checkpoint: False
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
use_cpu: False
seed: 42
data_seed: None
bf16: False
fp16: False
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: -1
ddp_backend: None
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
parallelism_config: None
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch_fused
optim_args: None
group_by_length: False
length_column_name: length
project: huggingface
trackio_space_id: trackio
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
hub_revision: None
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_for_metrics: []
eval_do_concat_batches: True
auto_find_batch_size: False
full_determinism: False
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_num_input_tokens_seen: no
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
liger_kernel_config: None
eval_use_gather_object: False
average_tokens_across_devices: True
use_cache: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin
router_mapping: {}
learning_rate_mapping: {}

Training Logs

Epoch	Step	Training Loss
27.7778	500	0.0025
55.5556	1000	0.0008
83.3333	1500	0.0007

Training Time

Training: 5.0 minutes

Framework Versions

Python: 3.12.13
Sentence Transformers: 5.4.1
Transformers: 5.0.0
PyTorch: 2.10.0+cu128
Accelerate: 1.13.0
Datasets: 4.0.0
Tokenizers: 0.22.2

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

ContrastiveLoss

bibtex

@inproceedings{hadsell2006dimensionality,
    author={Hadsell, R. and Chopra, S. and LeCun, Y.},
    booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
    title={Dimensionality Reduction by Learning an Invariant Mapping},
    year={2006},
    volume={2},
    number={},
    pages={1735-1742},
    doi={10.1109/CVPR.2006.100}
}

	sentence_0	sentence_1	label
type	string	string	float
details	min: 11 tokens mean: 30.57 tokens max: 48 tokens	min: 64 tokens mean: 231.63 tokens max: 256 tokens	min: 0.0 mean: 0.17 max: 1.0