1from sentence_transformers import SentenceTransformer
23# Download from the 🤗 Hub4model = SentenceTransformer("ronit01/rag_tuned_minilm_10")5# Run inference6sentences =[7'What are the two knob set generators currently supported by RapidFire AI for creating multi-config specifications?',8'We currently support two common knob set generators: :func:`List()` for a discrete \nset of values and :func:`Range()` for sampling from a continuous value interval.\n\n\n.. py:function:: List(values: List[Any])\n\n\t:param values: List of discrete values for a knob; all values must be the same python data type.\n\t:type values: List[Any]\n\n\n.. py:function:: Range(start: int | float, end: int | float, dtype: str = "int" | "float")\n\n\t:param start: Lower bound of range interval.\n\t:type start: int | float\n\n\t:param end: Upper bound of range interval.\n\t:type end: int | float\n\n\t:param dtype: Data type of value to be sampled, either :code:`"int"` or :code:`"float"`.\n\t:type dtype: str\n\n\n**Notes:**\n\nAs of this writing, :func:`Range()` performs uniform sampling within the given interval. \nWe plan to continue expanding this API and add more functionality on this front based on feedback.',9'Note that if you plan to use only OpenAI APIs and not self-hosted models (for embedding or generation), you do NOT need GPUs on your machine. \nBut you must provide a valid OpenAI API key via a config argument as shown in the GSM8K and SciFact tutorial notebooks.\n\n\nStep 1: Install dependencies and package\n-----------------------\n\nObtain the RapidFire AI OSS package from pypi (includes all dependencies) and ensure it is installed correctly.\n\n.. important::\n\n Requires Python 3.12+. Ensure that ``python3`` resolves to Python 3.12 before creating the venv.\n\n.. code-block:: bash\n\n python3 --version # must be 3.12.x\n python3 -m venv .venv\n source .venv/bin/activate\n\n pip install rapidfireai\n\n rapidfireai --version\n # Verify it prints the following:\n # RapidFire AI 0..14.0\n\n # Due to current issue: https://github.com/huggingface/xet-core/issues/527\n pip uninstall -y hf-xet\n\n\nThe tutorial notebooks for RAG evals do not use any gated models from Hugging Face.\nIf you want to access gated models, provide your Hugging Face account token.\nFor more details on that, :doc:`see Step 1 here</walkthroughft>`.\n\n\nStep 2: Initialize and start RapidFire AI server\n------------\n\nRun the following commands to initialize rapidfireai to use the correct dependencies for RAG evals:\n\n.. code-block:: bash\n\n rapidfireai init --evals\n # It will install specific dependencies and initialize rapidfireai for RAG evals\n\n\n.. note::\n You need to run init **only once** for a new venv or when switching GPU(s) on your machine. You do NOT need to run it after a reboot or for a new terminal tab.\n\n\nNext start RapidFire AI services: the frontend with the ML metrics dashboard and the API server. \nThe frontend URL shown below can be opened on your local browser.\n\n.. code-block:: bash\n\n rapidfireai start\n # It should print about 50 lines, including the following:\n # ...\n # RapidFire Frontend is ready\n # Open your browser and navigate to: http://0.0.0.0:8853\n # ...\n # Press Ctrl+C to stop all services\n\n.. important::\n\n Do NOT proceed until the start is successful with "Available endpoints" printed as above. Leave this terminal running while you work through the tutorial notebooks. \n\n\nIf you close the terminal in which you started rapidfireai or if you rebooted your machine, \njust start rapidfireai again with the above command.\n\nIf the start command fails for whatever reason, wait for half a minute and rerun it.\nFor diagnostics and common fixes (including Linux/macOS and Windows steps), see :doc:`Troubleshooting </troubleshooting>`.\n\n.. note::\n For RAG/context engineering experiments with :func:`run_evals()`, starting the server is **optional** and only needed if you want to see results on the ML metrics dashboard too. Just as results are shown in an in-notebook table too, IC Ops panel can be displayed in the notebook too, as illustrated below (Steps 5 and 6).',10]11embeddings = model.encode(sentences)12print(embeddings.shape)13# [3, 384]1415# Get the similarity scores for the embeddings16similarities = model.similarity(embeddings, embeddings)17print(similarities)18# tensor([[1.0000, 0.8854, 0.4689],19# [0.8854, 1.0000, 0.3173],20# [0.4689, 0.3173, 1.0000]])
Training Details
Training Dataset
Unnamed Dataset
Size: 208 training samples
Columns: sentence_0, sentence_1, and label
Approximate statistics based on the first 208 samples:
sentence_0
sentence_1
label
type
string
string
float
details
min: 11 tokens
mean: 24.87 tokens
max: 34 tokens
min: 31 tokens
mean: 222.9 tokens
max: 256 tokens
min: 0.0
mean: 0.25
max: 1.0
Samples:
sentence_0
sentence_1
label
What arguments does the RFModelConfig class accept for defining a model configuration in RapidFire AI?
Recovering Storage Space[object Object]-------[object Object][object Object]If you run out of storage space on your machine due to experimenting with lots of LLMs, we [object Object]recommend clearing out the ".cache" folder on your home directory that is created by [object Object]Hugging Face to import the base models. [object Object]One experiment's imported models are not needed for another; so, it is safe to delete them.[object Object][object Object]If you want to reclaim even more space, look at the artifacts from your experiments and [object Object]either delete some of the files or move them to other/remote storage. [object Object]Note that when you use LoRA adapters, RapidFire AI saves only the trained adapters in the [object Object]checkpoints of the runs, not the base models.
0.0
How do reward functions work in RapidFire AI for GRPO training, and what arguments does TRL inject into them?
Multi-GPU Model Partitioning with FSDP
RapidFire AI supports automated large model partitioning across GPUs (on the same machine) via PyTorch's native FSDP.
Provide the relevant FSDP deatils in a config knob, optionally along with the number of GPUs to use for that run.
The following notebooks showcase the use of FSDP for SFT with the corresponding LLMs:
FSDP Lite with base model TinyLlama-1.1B-Chat-v1.0. Needs at least 2x A10 GPUs or equivalent (48 GB total HBM) to work. View on GitHub <https://github.com/RapidFireAI/rapidfireai/blob/main/tutorial_notebooks/fine-tuning/rf-tutorial-sft-chatqa-fsdp-lite.ipynb>__
FSDP Regular with base model Qwen3-32B. Needs at least 4x A10 GPUs or equivalent (96 GB total HBM) to work. View on GitHub <https://github.com/RapidFireAI/rapidfireai/blob/main/tutorial_notebooks/fine-tuning/rf-tutorial-sft-chatqa.ipynb>__
FSDP Large with base model Llama-3-70B-Instruct. Needs at least 8x A10 GPUs or equivalent (192 GB total HBM) to work.... | 0.0 |
| What are the three use case tutorials provided for RAG and context engineering, and what type of workflow does each demonstrate? | This use case notebook features an all-closed model API workflow, with Open AI calls used for both embedding for generation. So, you do not need a GPU to run this notebook. | 1.0 |
1@inproceedings{reimers-2019-sentence-bert,
2 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
3 author = "Reimers, Nils and Gurevych, Iryna",
4 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
5 month = "11",
6 year = "2019",
7 publisher = "Association for Computational Linguistics",
8 url = "https://arxiv.org/abs/1908.10084",
9}
ContrastiveLoss
bibtex
1@inproceedings{hadsell2006dimensionality,
2 author={Hadsell, R. and Chopra, S. and LeCun, Y.},
3 booktitle={2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)},
4 title={Dimensionality Reduction by Learning an Invariant Mapping},
5 year={2006},
6 volume={2},
7 number={},
8 pages={1735-1742},
9 doi={10.1109/CVPR.2006.100}
10}
