Views
No views yet
SentenceTransformer(
(0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'MPNetModel'})
(1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'mean', 'include_prompt': True})
(2): Normalize({})
)pip install -U sentence-transformers1from sentence_transformers import SentenceTransformer
2
3# Download from the 🤗 Hub
4model = SentenceTransformer("sindsub/cybersec-mpnet-finetuned-round3")
5# Run inference
6sentences = [
7 'Ave Maria is a remote access trojan, written in C++, that is capable of process and file system manipulation, command shell access, webcam control , keylogging, password theft, and remote desktop access.',
8 "An adversary can leverage a computer's peripheral devices (e.g., integrated cameras or webcams) or applications (e.g., video call services) to capture video recordings for the purpose of gathering information. Images may also be captured from devices or applications, potentially in specified intervals, in lieu of video files.\n\nMalware or scripts may be used to interact with the devices through an available API provided by the operating system or an application to capture video or images. Video or image files may be written to disk and exfiltrated later. This technique differs from [Screen Capture](https://attack.mitre.org/techniques/T1113) due to use of specific devices or applications for video recording rather than capturing the victim's screen.\n\nIn macOS, there are a few different malware samples that record the user's webcam such as FruitFly and Proton. (Citation: objective-see 2017 review)",
9 'Network Traffic Content from packet inspection',
10]
11embeddings = model.encode(sentences)
12print(embeddings.shape)
13# [3, 768]
14
15# Get the similarity scores for the embeddings
16similarities = model.similarity(embeddings, embeddings)
17print(similarities)
18# tensor([[ 1.0000, 0.5280, -0.0770],
19# [ 0.5280, 1.0000, 0.0186],
20# [-0.0770, 0.0186, 1.0000]])anchor and positive| anchor | positive | |
|---|---|---|
| type | string | string |
| modality | text | text |
| details |
|
|
| anchor | positive |
|---|---|
! [object Object] | Adversaries may employ various means to detect and avoid virtualization and analysis environments. This may include changing behaviors based on the results of checks for the presence of artifacts indicative of a virtual machine environment (VME) or sandbox. If the adversary detects a VME, they may alter their malware to disengage from the victim or conceal the core functions of the implant. They may also search for VME artifacts before dropping secondary or additional payloads. Adversaries may use the information learned from [object Object] during automated discovery to shape follow-on behaviors.(Citation: Deloitte Environment Awareness)[object Object][object Object]Adversaries may use several methods to accomplish [object Object] such as checking for security monitoring tools (e.g., Sysinternals, Wireshark, etc.) or other system artifacts associated with analysis or virtualization. Adversaries... |
[object Object] | Adversaries may configure system settings to automatically execute a program during system boot or logon to maintain persistence or gain higher-level privileges on compromised systems. Operating systems may have mechanisms for automatically running a program on system boot or account logon.(Citation: Microsoft Run Key)(Citation: MSDN Authentication Packages)(Citation: Microsoft TimeProvider)(Citation: Cylance Reg Persistence Sept 2013)(Citation: Linux Kernel Programming) These mechanisms may include automatically executing programs that are placed in specially designated directories or are referenced by repositories that store configuration information, such as the Windows Registry. An adversary may achieve the same goal by modifying or extending features of the kernel.[object Object][object Object]Since some boot or logon autostart programs run with higher privileges, an adversary may leverage these to elevate privileges. |
[object Object] | Adversaries may enumerate files and directories or may search in specific locations of a host or network share for certain information within a file system. Adversaries may use the information from [object Object] during automated discovery to shape follow-on behaviors, including whether or not the adversary fully infects the target and/or attempts specific actions.[object Object][object Object]Many command shell utilities can be used to obtain this information. Examples include [object Object], [object Object], [object Object], [object Object], and [object Object].(Citation: Windows Commands JPCERT) Custom tools may also be used to gather file and directory information and interact with the [object Object]. Adversaries may also leverage a [object Object] on network devices to gather file and directory information (e.g. [object Object], [object Object] |
MultipleNegativesRankingLoss with these parameters:
1{
2 "scale": 20.0,
3 "similarity_fct": "cos_sim",
4 "gather_across_devices": false,
5 "directions": [
6 "query_to_doc"
7 ],
8 "partition_mode": "joint",
9 "hardness_mode": null,
10 "hardness_strength": 0.0
11}per_device_train_batch_size: 16num_train_epochs: 4learning_rate: 5e-06warmup_steps: 70per_device_train_batch_size: 16num_train_epochs: 4max_steps: -1learning_rate: 5e-06lr_scheduler_type: linearlr_scheduler_kwargs: Nonewarmup_steps: 70optim: adamw_torch_fusedoptim_args: Noneweight_decay: 0.0adam_beta1: 0.9adam_beta2: 0.999adam_epsilon: 1e-08optim_target_modules: Nonegradient_accumulation_steps: 1average_tokens_across_devices: Truemax_grad_norm: 1.0label_smoothing_factor: 0.0bf16: Falsefp16: Falsebf16_full_eval: Falsefp16_full_eval: Falsetf32: Nonegradient_checkpointing: Falsegradient_checkpointing_kwargs: Nonetorch_compile: Falsetorch_compile_backend: Nonetorch_compile_mode: Noneuse_liger_kernel: Falseliger_kernel_config: Noneuse_cache: Falseneftune_noise_alpha: Nonetorch_empty_cache_steps: Noneauto_find_batch_size: Falselog_on_each_node: Truelogging_nan_inf_filter: Trueinclude_num_input_tokens_seen: nolog_level: passivelog_level_replica: warningdisable_tqdm: Falseproject: huggingfacetrackio_space_id: Nonetrackio_bucket_id: Nonetrackio_static_space_id: Noneper_device_eval_batch_size: 8prediction_loss_only: Trueeval_on_start: Falseeval_do_concat_batches: Trueeval_use_gather_object: Falseeval_accumulation_steps: Noneinclude_for_metrics: []batch_eval_metrics: Falsesave_only_model: Falsesave_on_each_node: Falseenable_jit_checkpoint: Falsepush_to_hub: Falsehub_private_repo: Nonehub_model_id: Nonehub_strategy: every_savehub_always_push: Falsehub_revision: Noneload_best_model_at_end: Falseignore_data_skip: Falserestore_callback_states_from_checkpoint: Falsefull_determinism: Falseseed: 42data_seed: Noneuse_cpu: Falseaccelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}parallelism_config: Nonedataloader_drop_last: Falsedataloader_num_workers: 0dataloader_pin_memory: Truedataloader_persistent_workers: Falsedataloader_prefetch_factor: Noneremove_unused_columns: Truelabel_names: Nonetrain_sampling_strategy: randomlength_column_name: lengthddp_find_unused_parameters: Noneddp_bucket_cap_mb: Noneddp_broadcast_buffers: Falseddp_static_graph: Noneddp_backend: Noneddp_timeout: 1800fsdp: Nonefsdp_config: Nonedeepspeed: Nonedebug: []skip_memory_metrics: Truedo_predict: Falseresume_from_checkpoint: Nonewarmup_ratio: Nonelocal_rank: -1prompts: Nonebatch_sampler: batch_samplermulti_dataset_batch_sampler: proportionalrouter_mapping: {}learning_rate_mapping: {}| Epoch | Step | Training Loss |
|---|---|---|
| 0.1437 | 25 | 0.2841 |
| 0.2874 | 50 | 0.2983 |
| 0.4310 | 75 | 0.3229 |
| 0.5747 | 100 | 0.3010 |
| 0.7184 | 125 | 0.2719 |
| 0.8621 | 150 | 0.2619 |
| 1.0057 | 175 | 0.3647 |
| 1.1494 | 200 | 0.2494 |
| 1.2931 | 225 | 0.2454 |
| 1.4368 | 250 | 0.2100 |
| 1.5805 | 275 | 0.2931 |
| 1.7241 | 300 | 0.3017 |
| 1.8678 | 325 | 0.2776 |
| 2.0115 | 350 | 0.2925 |
| 2.1552 | 375 | 0.2372 |
| 2.2989 | 400 | 0.2606 |
| 2.4425 | 425 | 0.2645 |
| 2.5862 | 450 | 0.2166 |
| 2.7299 | 475 | 0.2441 |
| 2.8736 | 500 | 0.2659 |
| 3.0172 | 525 | 0.2414 |
| 3.1609 | 550 | 0.2348 |
| 3.3046 | 575 | 0.2598 |
| 3.4483 | 600 | 0.2984 |
| 3.5920 | 625 | 0.2190 |
| 3.7356 | 650 | 0.2224 |
| 3.8793 | 675 | 0.2564 |
1@inproceedings{reimers-2019-sentence-bert,
2 title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
3 author = "Reimers, Nils and Gurevych, Iryna",
4 booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
5 month = "11",
6 year = "2019",
7 publisher = "Association for Computational Linguistics",
8 url = "https://arxiv.org/abs/1908.10084",
9}1@misc{oord2019representationlearningcontrastivepredictive,
2 title={Representation Learning with Contrastive Predictive Coding},
3 author={Aaron van den Oord and Yazhe Li and Oriol Vinyals},
4 year={2019},
5 eprint={1807.03748},
6 archivePrefix={arXiv},
7 primaryClass={cs.LG},
8 url={https://arxiv.org/abs/1807.03748},
9}