Views
No views yet
SentenceTransformer(
(0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel
(1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)pip install -U sentence-transformers1from sentence_transformers import SentenceTransformer
2
3# Download from the 🤗 Hub
4model = SentenceTransformer("anishareddyalla/bge-base-matryoshka-aws-casestudies")
5# Run inference
6sentences = [
7 'Alternatively, you can run the inference via code. Here is one example written in Python, using the requests library: import requests url = "https://<YOUR_API_GATEWAY_ENDPOINT_ID>. execute-api. <YOUR_ENDPOINT_REGION>. amazonaws. com/prod/question?question=\\"What is the color of my car now?\\"&context=\\"My car used to be blue but I painted red\\"" response = requests. request("GET", url, headers=headers, data=payload) print(response. text) The code outputs a string similar to the following: \'{"score":0. 6947233080863953,"start":38,"end":41,"answer":"red"}\' If you are interested in knowing more about deploying Generative AI and large language models on AWS, check out here: Deploy Serverless Generative AI on AWS Lambda with OpenLLaMa Deploy large language models on AWS Inferentia2 using large model inference containers Clean up Inside the root directory of your repository, run the following code to clean up your resources: make destroy Conclusion In this post, we introduced how you can use Lambda to deploy your trained ML model using your preferred web application framework, such as FastAPI. We provided a detailed code repository that you can deploy, and you retain the flexibility of switching to whichever trained model artifacts you process. The performance can depend on how you implement and deploy the model. You are welcome to try it out yourself, and we’re excited to hear your feedback! About the Authors Tingyi Li is an Enterprise Solutions Architect from AWS based out in Stockholm, Sweden supporting the Nordics customers. She enjoys helping customers with the architecture, design, and development of cloud-optimized infrastructure solutions. She is specialized in AI and Machine Learning and is interested in empowering customers with intelligence in their AI/ML applications. In her spare time, she is also a part-time illustrator who writes novels and plays the piano. Demir Catovic is a Machine Learning Engineer from AWS based in Zurich, Switzerland. He engages with customers and helps them implement scalable and fully-functional ML applications. He is passionate about building and productionizing machine learning applications for customers and is always keen to explore around new trends and cutting-edge technologies in the AI/ML world. TAGS: Generative AI , Natural Language Processing Comments View Comments Resources Getting Started What\'s New Blog Topics Amazon Comprehend Amazon Kendra Amazon Lex Amazon Polly Amazon Rekognition Amazon SageMaker Amazon Textract Follow Twitter Facebook LinkedIn Twitch Email Updates.',
8 'How can Lambda be used to deploy trained ML models using a preferred web application framework?',
9 'How has PayEye been able to accelerate their development process and enter the production phase within a few months using AWS services, and what impact has this had on their recruitment efforts and team focus?',
10]
11embeddings = model.encode(sentences)
12print(embeddings.shape)
13# [3, 768]
14
15# Get the similarity scores for the embeddings
16similarities = model.similarity(embeddings, embeddings)
17print(similarities.shape)
18# [3, 3]dim_768InformationRetrievalEvaluator| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.5121 |
| cosine_accuracy@3 | 0.8266 |
| cosine_accuracy@5 | 0.9234 |
| cosine_accuracy@10 | 0.9637 |
| cosine_precision@1 | 0.5121 |
| cosine_precision@3 | 0.2755 |
| cosine_precision@5 | 0.1847 |
| cosine_precision@10 | 0.0964 |
| cosine_recall@1 | 0.5121 |
| cosine_recall@3 | 0.8266 |
| cosine_recall@5 | 0.9234 |
| cosine_recall@10 | 0.9637 |
| cosine_ndcg@10 | 0.7539 |
| cosine_mrr@10 | 0.6844 |
| cosine_map@100 | 0.6859 |
dim_512InformationRetrievalEvaluator| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.5323 |
| cosine_accuracy@3 | 0.8226 |
| cosine_accuracy@5 | 0.9194 |
| cosine_accuracy@10 | 0.9677 |
| cosine_precision@1 | 0.5323 |
| cosine_precision@3 | 0.2742 |
| cosine_precision@5 | 0.1839 |
| cosine_precision@10 | 0.0968 |
| cosine_recall@1 | 0.5323 |
| cosine_recall@3 | 0.8226 |
| cosine_recall@5 | 0.9194 |
| cosine_recall@10 | 0.9677 |
| cosine_ndcg@10 | 0.7597 |
| cosine_mrr@10 | 0.6913 |
| cosine_map@100 | 0.6924 |
dim_256InformationRetrievalEvaluator| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.5242 |
| cosine_accuracy@3 | 0.8226 |
| cosine_accuracy@5 | 0.9194 |
| cosine_accuracy@10 | 0.9597 |
| cosine_precision@1 | 0.5242 |
| cosine_precision@3 | 0.2742 |
| cosine_precision@5 | 0.1839 |
| cosine_precision@10 | 0.096 |
| cosine_recall@1 | 0.5242 |
| cosine_recall@3 | 0.8226 |
| cosine_recall@5 | 0.9194 |
| cosine_recall@10 | 0.9597 |
| cosine_ndcg@10 | 0.7528 |
| cosine_mrr@10 | 0.6846 |
| cosine_map@100 | 0.6863 |
dim_128InformationRetrievalEvaluator| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.496 |
| cosine_accuracy@3 | 0.7903 |
| cosine_accuracy@5 | 0.8911 |
| cosine_accuracy@10 | 0.9556 |
| cosine_precision@1 | 0.496 |
| cosine_precision@3 | 0.2634 |
| cosine_precision@5 | 0.1782 |
| cosine_precision@10 | 0.0956 |
| cosine_recall@1 | 0.496 |
| cosine_recall@3 | 0.7903 |
| cosine_recall@5 | 0.8911 |
| cosine_recall@10 | 0.9556 |
| cosine_ndcg@10 | 0.7338 |
| cosine_mrr@10 | 0.6613 |
| cosine_map@100 | 0.6631 |
dim_64InformationRetrievalEvaluator| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.4476 |
| cosine_accuracy@3 | 0.7661 |
| cosine_accuracy@5 | 0.879 |
| cosine_accuracy@10 | 0.9476 |
| cosine_precision@1 | 0.4476 |
| cosine_precision@3 | 0.2554 |
| cosine_precision@5 | 0.1758 |
| cosine_precision@10 | 0.0948 |
| cosine_recall@1 | 0.4476 |
| cosine_recall@3 | 0.7661 |
| cosine_recall@5 | 0.879 |
| cosine_recall@10 | 0.9476 |
| cosine_ndcg@10 | 0.7053 |
| cosine_mrr@10 | 0.6261 |
| cosine_map@100 | 0.6277 |
positive and anchor| positive | anchor | |
|---|---|---|
| type | string | string |
| details |
|
|
| positive | anchor |
|---|---|
TCSG is helping students enter a competitive workforce as educated cloud professionals and providing opportunities for success. TCSG built its Cloud Academy using AWS Academy, which provides higher education institutions with a free, ready-to-teach cloud computing curriculum that prepares students to pursue industry-recognized certifications and in-demand cloud jobs. TCSG launched the TCSG Cloud Academy in two forms: one as a specialization within an existing associate’s degree and the second as a stand-alone technical certificate of credit. For the technical certificate of credit, students who have existing degrees can enter the curriculum to focus on cloud computing and participate in hands-on cloud experiences using AWS services. Tiếng Việt Italiano ไทย The Technical College System of Georgia is the state government agency that supervises workforce development of more than 294,000 students across 22 technical colleges, 88 campuses, and more than 600 programs. Using the AWS curriculum and technology as the foundation for its courses, TCSG is preparing students to earn industry-recognized AWS Certifications to increase employability while improving accessibility to cloud education by offering the academy virtually and remotely. Learn more » TCSG is the state of Georgia government agency that supervises workforce development of hundreds of thousands of students across 22 technical colleges, 88 campuses, and more than 600 programs. The agency aims to run a system of technical education using the latest technology that’s accessible to all adults and corporate citizens in the state. To develop and deploy its new cloud-focused curriculum, it worked with AWS Education Programs, which helps TCSG institutions develop initiatives that align education to careers in the cloud and promote student employability, preparing diverse learners for in-demand cloud roles around the world. In 2020, the organization officially launched the TCSG Cloud Academy—a virtual program for students pursuing expertise and certifications in cloud computing—on its eCampus virtual learning system. Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today. Português. | How has the use of AWS Academy by TCSG helped prepare students for pursuing industry-recognized certifications and in-demand cloud jobs in Georgia's workforce? |
This prompt is then provided to the LLM for generating an answer to the user question. @router. post("/rag") async def rag_handler(req: Request) -> Dict[str, Any]: # dump the received request for debugging purposes logger. info(f"req={req}") # initialize vector db and SageMaker Endpoint _init(req) # Use the vector db to find similar documents to the query # the vector db call would automatically convert the query text # into embeddings docs = _vector_db. similarity_search(req. q, k=req. max_matching_docs) logger. info(f"here are the {req. max_matching_docs} closest matching docs to the query="{req. q}"") for d in docs: logger. info(f"---------") logger. info(d) logger. info(f"---------") # now that we have the matching docs, lets pack them as a context # into the prompt and ask the LLM to generate a response prompt_template = """Answer based on context:\n\n{context}\n\n{question}""" prompt = PromptTemplate( template=prompt_template, input_variables=["context", "question"] ) logger. info(f"prompt sent to llm = "{prompt}"") chain = load_qa_chain(llm=_sm_llm, prompt=prompt) answer = chain({"input_documents": docs, "question": req. q}, return_only_outputs=True)['output_text'] logger. info(f"answer received from llm,\nquestion: "{req. q}"\nanswer: "{answer}"") resp = {'question': req. q, 'answer': answer} if req. verbose is True: resp['docs'] = docs return resp Clean up To avoid incurring future charges, delete the resources. You can do this by deleting the CloudFormation stack as shown in the following screenshot. | What resources need to be deleted to avoid future charges, and how can they be deleted? |
append(input_1_s3_location) async_response = base_model_predictor. predict_async(input_path=input_1_s3_location) output_locations. append(async_response. output_path) if i > max_images: break This may take up to 30 minutes or more depending on how much data you have uploaded for asynchronous inference. You can visualize one of these inferences as follows: plot_response('data/single. out') Convert the asynchronous inference output to a Ground Truth input manifest In this step, we create an input manifest for a bounding box verification job on Ground Truth. We upload the Ground Truth UI template and label categories file, and create the verification job. The notebook linked to this post uses a private workforce to perform the labeling; you can change this if you’re using other types of workforces. For more details, refer to the full code in the notebook. Verify labels from the auto-labeling process in Ground Truth In this step, we complete the verification by accessing the labeling portal. For more details, refer to here. When you access the portal as a workforce member, you will be able to see the bounding boxes created by the JumpStart model and make adjustments as required. You can use this template to repeat auto-labeling with many task-specific models, potentially merge labels, and use the resulting labeled dataset in downstream tasks. Clean up In this step, we clean up by deleting the endpoint and the model created in previous steps: # Delete the SageMaker endpoint base_model_predictor. delete_model() base_model_predictor. delete_endpoint() Conclusion In this post, we walked through an auto-labeling process involving JumpStart and asynchronous inference. We used the results of the auto-labeling process to convert and visualize labeled data on a real-world dataset. You can use the solution to perform auto-labeling with many task-specific models, potentially merge labels, and use the resulting labeled dataset in downstream tasks. You can also explore using tools like the Segment Anything Model for generating segment masks as part of the auto-labeling process. In future posts in this series, we will cover the perception module and segmentation. | How can you visualize the inferences generated by the asynchronous inference process using the provided solution? |
MatryoshkaLoss with these parameters:
1{
2 "loss": "MultipleNegativesRankingLoss",
3 "matryoshka_dims": [
4 768,
5 512,
6 256,
7 128,
8 64
9 ],
10 "matryoshka_weights": [
11 1,
12 1,
13 1,
14 1,
15 1
16 ],
17 "n_dims_per_step": -1
18}eval_strategy: epochper_device_train_batch_size: 32per_device_eval_batch_size: 16gradient_accumulation_steps: 16learning_rate: 2e-05num_train_epochs: 4lr_scheduler_type: cosinewarmup_ratio: 0.1bf16: Truetf32: Trueload_best_model_at_end: Trueoptim: adamw_torch_fusedbatch_sampler: no_duplicatesoverwrite_output_dir: Falsedo_predict: Falseeval_strategy: epochprediction_loss_only: Trueper_device_train_batch_size: 32per_device_eval_batch_size: 16per_gpu_train_batch_size: Noneper_gpu_eval_batch_size: Nonegradient_accumulation_steps: 16eval_accumulation_steps: Nonelearning_rate: 2e-05weight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08max_grad_norm: 1.0num_train_epochs: 4max_steps: -1lr_scheduler_type: cosinelr_scheduler_kwargs: {}warmup_ratio: 0.1warmup_steps: 0log_level: passivelog_level_replica: warninglog_on_each_node: Truelogging_nan_inf_filter: Truesave_safetensors: Truesave_on_each_node: Falsesave_only_model: Falserestore_callback_states_from_checkpoint: Falseno_cuda: Falseuse_cpu: Falseuse_mps_device: Falseseed: 42data_seed: Nonejit_mode_eval: Falseuse_ipex: Falsebf16: Truefp16: Falsefp16_opt_level: O1half_precision_backend: autobf16_full_eval: Falsefp16_full_eval: Falsetf32: Truelocal_rank: 0ddp_backend: Nonetpu_num_cores: Nonetpu_metrics_debug: Falsedebug: []dataloader_drop_last: Falsedataloader_num_workers: 0dataloader_prefetch_factor: Nonepast_index: -1disable_tqdm: Falseremove_unused_columns: Truelabel_names: Noneload_best_model_at_end: Trueignore_data_skip: Falsefsdp: []fsdp_min_num_params: 0fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}fsdp_transformer_layer_cls_to_wrap: Noneaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}deepspeed: Nonelabel_smoothing_factor: 0.0optim: adamw_torch_fusedoptim_args: Noneadafactor: Falsegroup_by_length: Falselength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falsedataloader_pin_memory: Truedataloader_persistent_workers: Falseskip_memory_metrics: Trueuse_legacy_prediction_loop: Falsepush_to_hub: Falseresume_from_checkpoint: Nonehub_model_id: Nonehub_strategy: every_savehub_private_repo: Falsehub_always_push: Falsegradient_checkpointing: Falsegradient_checkpointing_kwargs: Noneinclude_inputs_for_metrics: Falseeval_do_concat_batches: Truefp16_backend: autopush_to_hub_model_id: Nonepush_to_hub_organization: Nonemp_parameters:auto_find_batch_size: Falsefull_determinism: Falsetorchdynamo: Noneray_scope: lastddp_timeout: 1800torch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Nonedispatch_batches: Nonesplit_batches: Noneinclude_tokens_per_second: Falseinclude_num_input_tokens_seen: Falseneftune_noise_alpha: Noneoptim_target_modules: Nonebatch_eval_metrics: Falseeval_on_start: Falsebatch_sampler: no_duplicatesmulti_dataset_batch_sampler: proportional| Epoch | Step | Training Loss | dim_128_cosine_map@100 | dim_256_cosine_map@100 | dim_512_cosine_map@100 | dim_64_cosine_map@100 | dim_768_cosine_map@100 |
|---|---|---|---|---|---|---|---|
| 0.9143 | 4 | - | 0.6663 | 0.6851 | 0.7027 | 0.6120 | 0.6998 |
| 1.8286 | 8 | - | 0.6758 | 0.6822 | 0.6966 | 0.6311 | 0.6941 |
| 2.2857 | 10 | 1.883 | - | - | - | - | - |
| 2.9714 | 13 | - | 0.6631 | 0.6881 | 0.6904 | 0.6245 | 0.6873 |
| 3.6571 | 16 | - | 0.6631 | 0.6863 | 0.6924 | 0.6277 | 0.6859 |
1@inproceedings{reimers-2019-sentence-bert,
2 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
3 author = "Reimers, Nils and Gurevych, Iryna",
4 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
5 month = "11",
6 year = "2019",
7 publisher = "Association for Computational Linguistics",
8 url = "https://arxiv.org/abs/1908.10084",
9}1@misc{kusupati2024matryoshka,
2 title={Matryoshka Representation Learning},
3 author={Aditya Kusupati and Gantavya Bhatt and Aniket Rege and Matthew Wallingford and Aditya Sinha and Vivek Ramanujan and William Howard-Snyder and Kaifeng Chen and Sham Kakade and Prateek Jain and Ali Farhadi},
4 year={2024},
5 eprint={2205.13147},
6 archivePrefix={arXiv},
7 primaryClass={cs.LG}
8}1@misc{henderson2017efficient,
2 title={Efficient Natural Language Response Suggestion for Smart Reply},
3 author={Matthew Henderson and Rami Al-Rfou and Brian Strope and Yun-hsuan Sung and Laszlo Lukacs and Ruiqi Guo and Sanjiv Kumar and Balint Miklos and Ray Kurzweil},
4 year={2017},
5 eprint={1705.00652},
6 archivePrefix={arXiv},
7 primaryClass={cs.CL}
8}