Views
No views yet
SentenceTransformer(
(0): Transformer({'max_seq_length': 256, 'do_lower_case': False, 'architecture': 'XLMRobertaModel'})
(1): Pooling({'word_embedding_dimension': 1024, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
(2): Normalize()
)pip install -U sentence-transformers1from sentence_transformers import SentenceTransformer
2
3# Download from the 🤗 Hub
4model = SentenceTransformer("songphucn7/me5-checkthat-task1-v1")
5# Run inference
6sentences = [
7 'query: Labor in the Era of Coronavirus via @user #COVID19 #washthehate #AsianAmerican "Apart from the threat of illness and death, fear is perhaps the most virulent part of COVID-19."',
8 'passage: title: Work in the Time of Coronavirus abstract: It was a meditation on life, love, old age, death: ideas that had often fluttered around her head like nocturnal birds but dissolved into a trickle of feathers when she tried to catch hold of them.\n—Gabriel García Márquez, Love in the Time of Cholera As surgeons, we pride ourselves on our sterility.\nWe have made a home of the aseptic operating room, where don\'t touch your face is a lesson learned on day 1.\nWe don gown and gloves and communicate behind mask and shield, all to keep ourselves and our patients safe.\nHowever, with the advent of the 2019 coronavirus disease (COVID-19) pandemic, we now find ourselves unable to scrub out at the end of the case.\nSterile technique has leaked into the public domain, marked not only by how expensive hand sanitizer has become or how frequently we are reminded to wash our hands, but also by the ways we have come to treat one another.\nIt is undeniable that the social distancing movement is critical in quelling the exponential spread of this virus.\nUnfortunately, for Asians, this social sterility has left in its wake a sense of isolation and blame.\nDuring the last few months, there have been countless stories of racism and xenophobia toward individuals of Asian descent.\nAlthough these accounts have ranged from derogatory comments ("Get your coronavirus out of my country!',
9 'passage: title: Wolves make roadways safer, generating large economic returns to predator conservation abstract: Significance Measuring the economic benefits conveyed by predators is difficult—often, effects are indirect and operate through complex ecological changes.\nAs a result, debates about the expansion of predators have pit salient costs against more speculative estimates of benefits that might be dismissed as unreliable or ideologically motivated.\nWe quantify the indirect benefits of wolves ( Canis lupus ) to human lives and property through reductions in deer-vehicle collisions.\nMoreover, we decompose the effect into two components: changes in prey behavior versus prey abundance.\nThis decomposition is important when effective policy depends on whether hunters can replicate the effects of predators.\nIn the case of wolves, we conclude that human deer hunters cannot.',
10]
11embeddings = model.encode(sentences)
12print(embeddings.shape)
13# [3, 1024]
14
15# Get the similarity scores for the embeddings
16similarities = model.similarity(embeddings, embeddings)
17print(similarities)
18# tensor([[1.0000, 0.6389, 0.1379],
19# [0.6389, 1.0000, 0.1470],
20# [0.1379, 0.1470, 1.0000]])10-percent-dev-splitInformationRetrievalEvaluator| Metric | Value |
|---|---|
| cosine_accuracy@1 | 0.5096 |
| cosine_accuracy@3 | 0.7106 |
| cosine_accuracy@5 | 0.7761 |
| cosine_accuracy@10 | 0.827 |
| cosine_precision@1 | 0.5096 |
| cosine_precision@3 | 0.2369 |
| cosine_precision@5 | 0.1552 |
| cosine_precision@10 | 0.0827 |
| cosine_recall@1 | 0.5096 |
| cosine_recall@3 | 0.7106 |
| cosine_recall@5 | 0.7761 |
| cosine_recall@10 | 0.827 |
| cosine_ndcg@10 | 0.6724 |
| cosine_mrr@10 | 0.6224 |
| cosine_map@100 | 0.628 |
sentence_0 and sentence_1| sentence_0 | sentence_1 | |
|---|---|---|
| type | string | string |
| details |
|
|
| sentence_0 | sentence_1 |
|---|---|
query: Group 100 patients convalesced #COVID19 CMR showed heart involvement 78 patients & persistent myocardial inflammation 60 patients, independent underlying conditions, intensity & full course of the sudden illness, & time from the initial diagnosis | passage: Native T1 and T2 were the measures with the best discriminatory ability to detect COVID-19-related myocardial pathology.[object Object][object Object]In this study of a cohort of German patients recently recovered from COVID-19 infection, CMR revealed cardiac involvement in 78 patients (78%) and ongoing myocardial inflammation in 60 patients (60%), independent of preexisting conditions, severity and overall course of the acute illness, and time from the original diagnosis.[object Object]These findings indicate the need for ongoing investigation of the long-term cardiovascular consequences of COVID-19. |
query: Preprint zu den Verlaufsdaten unserer #Charite Post Covid Studie, die bedauerlicherweise zeigt, dass die Mehrheit #PCS #MECFS dauerhaft schwer krank sind. | passage: title: Symptom persistence and biomarkers in post-COVID-19/chronic fatigue syndrome – results from a prospective observational cohort abstract: Abstract Introduction Post-COVID-19 syndrome (PCS) is characterized by a wide range of symptoms, predominantly fatigue and exertional intolerance.[object Object]While disease courses during the first year post infection have been repeatedly described, little is known about long-term health consequences.[object Object]Methods We assessed symptom severity and various biomarkers at three time points post infection (3-8 months (mo), 9-16mo, 17-20mo) in 106 PCS patients with moderate to severe fatigue and exertional intolerance.[object Object]A subset of patients fulfilled diagnostic criteria of myalgic encephalomyelitis/chronic fatigue syndrome (PCS-ME/CFS) based on the Canadian Consensus Criteria.[object Object]Results While PCS-ME/CFS patients showed persisting symptom severity and disability up to 20mo post infection, PCS patients reported an overall health improvement.[object Object]Inflammatory biomarke... |
query: Fresh Peer-reviewed: “SARSCoV2–Reactive Mucosal Bcells in the Upper Respiratory Tract of Uninfected Individuals” “Pre-existing SARSCoV2–reactive Bcells are found in tonsillar tissue.. contribute to prevent manifestations of symptomatic or severe disease” | passage: title: SARS-CoV-2–Reactive Mucosal B Cells in the Upper Respiratory Tract of Uninfected Individuals abstract: SARS-CoV-2 is a respiratory pathogen that can cause severe disease in at-risk populations but results in asymptomatic infections or a mild course of disease in the majority of cases.[object Object]We report the identification of SARS-CoV-2-reactive B cells in human tonsillar tissue obtained from children who were negative for coronavirus disease 2019 prior to the pandemic and the generation of mAbs recognizing the SARS-CoV-2 Spike protein from these B cells.[object Object]These Abs showed reduced binding to Spike proteins of SARS-CoV-2 variants and did not recognize Spike proteins of endemic coronaviruses, but subsets reacted with commensal microbiota and exhibited SARS-CoV-2-neutralizing potential.[object Object]Our study demonstrates pre-existing SARS-CoV-2-reactive Abs in various B cell populations in the upper respiratory tract lymphoid tissue that may lead to the rapid engagement of the pathogen and contr... |
MultipleNegativesRankingLoss with these parameters:
1{
2 "scale": 20.0,
3 "similarity_fct": "cos_sim",
4 "gather_across_devices": false,
5 "directions": [
6 "query_to_doc"
7 ],
8 "partition_mode": "joint",
9 "hardness_mode": null,
10 "hardness_strength": 0.0
11}per_device_train_batch_size: 32num_train_epochs: 1eval_strategy: stepsper_device_eval_batch_size: 32multi_dataset_batch_sampler: round_robinper_device_train_batch_size: 32num_train_epochs: 1max_steps: -1learning_rate: 5e-05lr_scheduler_type: linearlr_scheduler_kwargs: Nonewarmup_steps: 0optim: adamw_torch_fusedoptim_args: Noneweight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08optim_target_modules: Nonegradient_accumulation_steps: 1average_tokens_across_devices: Truemax_grad_norm: 1label_smoothing_factor: 0.0bf16: Falsefp16: Falsebf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonegradient_checkpointing: Falsegradient_checkpointing_kwargs: Nonetorch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneuse_liger_kernel: Falseliger_kernel_config: Noneuse_cache: Falseneftune_noise_alpha: Nonetorch_empty_cache_steps: Noneauto_find_batch_size: Falselog_on_each_node: Truelogging_nan_inf_filter: Trueinclude_num_input_tokens_seen: nolog_level: passivelog_level_replica: warningdisable_tqdm: Falseproject: huggingfacetrackio_space_id: trackioeval_strategy: stepsper_device_eval_batch_size: 32prediction_loss_only: Trueeval_on_start: Falseeval_do_concat_batches: Trueeval_use_gather_object: Falseeval_accumulation_steps: Noneinclude_for_metrics: []batch_eval_metrics: Falsesave_only_model: Falsesave_on_each_node: Falseenable_jit_checkpoint: Falsepush_to_hub: Falsehub_private_repo: Nonehub_model_id: Nonehub_strategy: every_savehub_always_push: Falsehub_revision: Noneload_best_model_at_end: Falseignore_data_skip: Falserestore_callback_states_from_checkpoint: Falsefull_determinism: Falseseed: 42data_seed: Noneuse_cpu: Falseaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedataloader_drop_last: Falsedataloader_num_workers: 0dataloader_pin_memory: Truedataloader_persistent_workers: Falsedataloader_prefetch_factor: Noneremove_unused_columns: Truelabel_names: Nonetrain_sampling_strategy: randomlength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falseddp_backend: Noneddp_timeout: 1800fsdp: []fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}deepspeed: Nonedebug: []skip_memory_metrics: Truedo_predict: Falseresume_from_checkpoint: Nonewarmup_ratio: Nonelocal_rank: -1prompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: round_robinrouter_mapping: {}learning_rate_mapping: {}| Epoch | Step | Training Loss | 10-percent-dev-split_cosine_ndcg@10 |
|---|---|---|---|
| 0.1845 | 100 | - | 0.6457 |
| 0.3690 | 200 | - | 0.6645 |
| 0.5535 | 300 | - | 0.6712 |
| 0.7380 | 400 | - | 0.6721 |
| 0.9225 | 500 | 0.6100 | 0.6720 |
| 1.0 | 542 | - | 0.6724 |
1@inproceedings{reimers-2019-sentence-bert,
2 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
3 author = "Reimers, Nils and Gurevych, Iryna",
4 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
5 month = "11",
6 year = "2019",
7 publisher = "Association for Computational Linguistics",
8 url = "https://arxiv.org/abs/1908.10084",
9}1@misc{oord2019representationlearningcontrastivepredictive,
2 title={Representation Learning with Contrastive Predictive Coding},
3 author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
4 year={2019},
5 eprint={1807.03748},
6 archivePrefix={arXiv},
7 primaryClass={cs.LG},
8 url={https://arxiv.org/abs/1807.03748},
9}