This is a sentence-transformers model finetuned from jiwonyou0420/MNLP_M2_document_encoder. It maps sentences & paragraphs to a 384-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: jiwonyou0420/MNLP_M2_document_encoder
Maximum Sequence Length: 512 tokens
Output Dimensionality: 384 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': True}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 384, 'pooling_mode_cls_token': True, 'pooling_mode_mean_tokens': False, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
  (2): Normalize()
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("jiwonyou0420/encoder-qa-finetuned-v2")
# Run inference
sentences = [
    'How can the renormalization technique be used to address the issue of infinity in the quantum electrodynamics calculation of the self-energy of an electron? Specifically, how can the divergent integrals be reorganized and regularized to yield a finite value for the self-energy?',
    'The prevalence of ALG6-CDG is unknown, but it is thought to be the second most common type of congenital disorder of glycosylation. More than 30 cases of ALG6-CDG have been described in the scientific literature.',
    'Superconductivity and superfluidity are two distinct quantum phenomena that share some similarities. Both phenomena involve the emergence of macroscopic quantum coherence, leading to the disappearance of electrical resistance or viscosity, respectively. They are both observed in materials at very low temperatures, where quantum effects become more pronounced.\n\nSuperconductivity is a phenomenon observed in certain materials, usually metals and alloys, where the electrical resistance drops to zero below a critical temperature. This allows for the flow of electric current without any energy loss. Superconductivity is explained by the BCS (Bardeen-Cooper-Schrieffer) theory, which states that electrons in a superconductor form Cooper pairs, which can move through the material without resistance due to their quantum mechanical nature.\n\nSuperfluidity, on the other hand, is a phenomenon observed in certain liquids, such as liquid helium, where the viscosity drops to zero below a critical temperature. This allows the liquid to flow without any resistance, leading to some unusual properties, such as the ability to climb the walls of a container or flow through extremely narrow channels. Superfluidity in liquid helium is explained by the Bose-Einstein condensation of helium atoms, which form a coherent quantum state that allows them to flow without resistance.\n\nWhile superconductivity and superfluidity are distinct phenomena, they share some similarities in their underlying mechanisms. Both involve the formation of a macroscopic quantum state, where particles (electrons in superconductors or atoms in superfluids) form pairs or condensates that can move without resistance. In this sense, superconductivity can be thought of as a type of superfluidity for charged particles.\n\nIn the case of liquid helium, superconductivity does not directly contribute to its superfluidity, as the two phenomena involve different particles (electrons for superconductivity and helium atoms for superfluidity). However, the study of superconductivity has provided valuable insights into the understanding of superfluidity, as both phenomena share some common underlying principles related to quantum coherence and the behavior of particles at very low temperatures.',
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 384]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Training Details

Training Dataset

Unnamed Dataset

Size: 72,812 training samples
Columns: sentence_0, sentence_1, and label
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1 label
type string string float
details
min: 7 tokens
mean: 33.24 tokens
max: 148 tokens
min: 12 tokens
mean: 343.73 tokens
max: 512 tokens
min: 0.0
mean: 0.5
max: 1.0

Samples:

sentence_0	sentence_1	label
`What is (are) Multicentric Castleman Disease ?`	Multicentric Castleman disease (MCD) is a rare condition that affects the lymph nodes and related tissues. It is a form of Castleman disease that is "systemic" and affects multiple sets of lymph nodes and other tissues throughout the body (as opposed to unicentric Castleman disease which has more "localized" effects). The signs and symptoms of MCD are often nonspecific and blamed on other, more common conditions. They can vary but may include fever; weight loss; fatigue; night sweats; enlarged lymph nodes; nausea and vomiting; and an enlarged liver or spleen. The eact underlying cause is unknown. Treatment may involve immunotherapy, chemotherapy, corticosteroid medications and/or anti-viral drugs.	`1.0`
`What are the treatments for multiple sclerosis ?`	The rotation period of the Milky Way galaxy can be estimated based on the observed velocities of stars in the outer regions of the galaxy. The Milky Way has a diameter of about 100,000 light-years, and the Sun is located about 27,000 light-years from the galactic center. The Sun orbits the galactic center at a speed of approximately 220 km/s.[object Object][object Object]To estimate the rotation period, we can use the formula for the circumference of a circle (C = 2πr) and divide it by the orbital speed of the Sun. The radius of the Sun's orbit is about 27,000 light-years, which is equivalent to 2.54 x 10^20 meters. Using this value, we can calculate the circumference of the Sun's orbit:[object Object][object Object]C = 2π(2.54 x 10^20 m) ≈ 1.6 x 10^21 meters[object Object][object Object]Now, we can divide the circumference by the Sun's orbital speed to find the time it takes for the Sun to complete one orbit around the Milky Way:[object Object][object Object]T = C / v = (1.6 x 10^21 m) / (220 km/s) ≈ 7.3 x 10^15 seconds[object Object][object Object]Converting this to years, we get:[object Object][object Object]T ≈ 7.3 x 10^15 s * (1 year / 3.15 x 10...	`0.0`
`"How do black holes affect the large-scale structure of the cosmic web, specifically in terms of dark matter distribution and the formation of galaxy clusters?"`	Black holes, especially supermassive black holes (SMBHs) found at the centers of galaxies, play a significant role in the large-scale structure of the cosmic web, which is a complex network of dark matter, gas, and galaxies that spans the universe. The cosmic web is organized into filaments, nodes, and voids, with galaxy clusters typically forming at the intersections of these filaments. The influence of black holes on the cosmic web can be understood in terms of their impact on dark matter distribution and the formation of galaxy clusters.[object Object][object Object]1. Dark matter distribution: Dark matter is a key component of the cosmic web, as it provides the gravitational scaffolding for the formation of galaxies and galaxy clusters. Black holes, particularly SMBHs, can influence the distribution of dark matter in several ways. For instance, when black holes merge, they release gravitational waves that can potentially redistribute dark matter in their vicinity. Additionally, the accretion of matter onto bl...	`1.0`

Loss: CosineSimilarityLoss with these parameters:
json
1{ 2 "loss_fct": "torch.nn.modules.loss.MSELoss" 3}

Training Hyperparameters

Non-Default Hyperparameters

per_device_train_batch_size: 16
per_device_eval_batch_size: 16
num_train_epochs: 1
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: no
prediction_loss_only: True
per_device_train_batch_size: 16
per_device_eval_batch_size: 16
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
tp_size: 0
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss
0.1099	500	0.0476
0.2197	1000	0.0277
0.3296	1500	0.0243
0.4395	2000	0.0225
0.5493	2500	0.0207
0.6592	3000	0.0206
0.7691	3500	0.019
0.8789	4000	0.02
0.9888	4500	0.0189

Framework Versions

Python: 3.12.8
Sentence Transformers: 3.4.1
Transformers: 4.51.3
PyTorch: 2.5.1+cu124
Accelerate: 1.3.0
Datasets: 3.2.0
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

	sentence_0	sentence_1	label
type	string	string	float
details	min: 7 tokens mean: 33.24 tokens max: 148 tokens	min: 12 tokens mean: 343.73 tokens max: 512 tokens	min: 0.0 mean: 0.5 max: 1.0