This is a sentence-transformers model finetuned from allenai/scibert_scivocab_uncased. It maps sentences & paragraphs to a 768-dimensional dense vector space and can be used for semantic textual similarity, semantic search, paraphrase mining, text classification, clustering, and more.

Model Details

Model Description

Model Type: Sentence Transformer
Base model: allenai/scibert_scivocab_uncased
Maximum Sequence Length: 512 tokens
Output Dimensionality: 768 dimensions
Similarity Function: Cosine Similarity

Model Sources

Documentation: Sentence Transformers Documentation
Repository: Sentence Transformers on GitHub
Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

First install the Sentence Transformers library:

pip install -U sentence-transformers

Then you can load this model and run inference.

python

from sentence_transformers import SentenceTransformer

# Download from the 🤗 Hub
model = SentenceTransformer("ML5562/MNLP_M3_document_encoder_f32")
# Run inference
sentences = [
    "Freshly graduated from EPFL, you have been hired as contractors for a successful and rapidly growing bank. The bank has been experiencing problems with their money management system recently, which is written in Scala, and so they hired the best and brightest young engineer they could find: you! The system had been working perfectly fine so far, they tell you. In the past days, due to an increased number of customers, they had to switch from a single-threaded sequential execution environment to a multi-threaded concurrent one, in which the threads may perform transactions concurrently. That's when problems started, your manager says...  Here is the code responsible to withdraw money from the account from and transfer it to the account to, within the same bank:  def transfer(from: Account, to: Account, amount: BigInt): Unit = {   require(amount >= 0)    val balanceFrom = from.balance    if (balanceFrom >= amount) {     from.balance = balanceFrom - amount     val balanceTo = to.balance     to.balance = balanceTo + amount   } } For the bank, it is very important that the following two properties hold after any sequence of completed transfer transactions:  The balance of an account never goes below 0. The total sum of money held by the bank is constant.  For each of the proposed implementations of transfer below, check which of the two properties hold. Additionally, check if the system is vulnerable to deadlocks.  Variant 1:  def transfer1(from: Account, to: Account, amount: Long): Unit = {   require(amount >= 0)    val balanceFrom = from.balance   if (balanceFrom >= amount) {     from.synchronized {       from.balance = balanceFrom - amount     }     to.synchronized {       val balanceTo = to.balance       to.balance = balanceTo + amount     }   } } Variant 2:  def transfer2(from: Account, to: Account, amount: Long): Unit = {   require(amount >= 0)    from.synchronized {     val balanceFrom = from.balance     if (balanceFrom >= amount) {       from.balance = balanceFrom - amount       to.synchronized {         val balanceTo = to.balance         to.balance = balanceTo + amount       }     }   } } Variant 3  object lock // Global object def transfer3(from: Account, to: Account, amount: Long): Unit = {   require(amount >= 0)    lock.synchronized {     val balanceFrom = from.balance     if (balanceFrom >= amount) {       from.balance = balanceFrom - amount       val balanceTo = to.balance       to.balance = balanceTo + amount     }   } }",
    "Let's analyze each of the three variants of the `transfer` function in terms of the two properties that must hold after any sequence of completed transfer transactionsâ\x80\x94the balance of an account never goes below 0, and the total sum of money held by the bank is constantâ\x80\x94as well as checking for potential deadlocks.\n\n### Variant 1\n```scala\ndef transfer1(from: Account, to: Account, amount: Long): Unit = {\n  require(amount >= 0)\n  val balanceFrom = from.balance\n  if (balanceFrom >= amount) {\n    from.synchronized {\n      from.balance = balanceFrom - amount\n    }\n    to.synchronized {\n      val balanceTo = to.balance\n      to.balance = balanceTo + amount\n    }\n  }\n}\n```\n\n**Analysis:**\n- **Property 1 (Balance never goes below 0):** This property is maintained because the balance is checked before any modification. However, since `from.synchronized` and `to.synchronized` are called separately, there can be a race condition. If two threads are transferring money between the same accounts, both threads can check the balance before either one updates it, leading to a situation where both might proceed to deduct from the same account.\n  \n- **Property 2 (Total sum of money is constant):** This property can be violated. Consider two threads, T1 and T2, executing the transfer concurrently. If T1 checks the balance and finds it sufficient, it proceeds to deduct the amount while T2 does the same. As a result, both transfers could succeed and the total money held by the bank could increase.\n\n- **Deadlocks:** This implementation is not susceptible to deadlocks because the locks are acquired in a non-nested manner (first on `from`, then on `to`), and there are no circular dependencies.\n\n### Variant 2\n```scala\ndef transfer2(from: Account, to: Account, amount: Long): Unit = {\n  require(amount >= 0)\n  from.synchronized {\n    val balanceFrom = from.balance\n    if (balanceFrom >= amount) {\n      from.balance = balanceFrom - amount\n      to.synchronized {\n        val balanceTo = to.balance\n        to.balance = balanceTo + amount\n      }\n    }\n  }\n}\n```\n\n**Analysis:**\n- **Property 1 (Balance never goes below 0):** This property is maintained. The check for balance occurs inside the synchronized block for `from`, ensuring that no other thread can modify the balance of `from` while the check is made and before the deduction occurs.\n\n- **Property 2 (Total sum of money is constant):** This property is also maintained. The operation is atomic concerning `from`, and since `to` is only modified after confirming the balance of `from`, no money is lost or created.\n\n- **Deadlocks:** This implementation is vulnerable to deadlocks. If one thread locks `from` and another thread locks `to`, and they try to acquire each other's locks, a deadlock can occur.\n\n### Variant 3\n```scala\nobject lock // Global object\ndef transfer3(from: Account, to: Account, amount: Long): Unit = {\n  require(amount >= 0)\n  lock.synchronized {\n    val balanceFrom = from.balance\n    if (balanceFrom >= amount) {\n      from.balance = balanceFrom - amount\n      val balanceTo = to.balance\n      to.balance = balanceTo + amount\n    }\n  }\n}\n```\n\n**Analysis:**\n- **Property 1 (Balance never goes below 0):** This property is maintained. The balance check occurs before any modifications under a synchronized block, preventing any concurrent modifications that could lead to a negative balance.\n\n- **Property 2 (Total sum of money is constant):** This property is also maintained. The total amount of money in the system does not change since the operations are enclosed in a synchronized block.\n\n- **Deadlocks:** This implementation is not susceptible to deadlocks since there is a single global lock that prevents any other thread from entering the critical section until the current operation completes.\n\n### Summary\n- **Variant 1:** Violates property 2; not deadlock-prone.\n- **Variant 2:** Both properties are maintained; deadlock-prone.\n- **Variant 3:** Both properties are maintained; not deadlock-prone but not parallelizable for different accounts.\n\nGiven the requirements of a bank system, **Variant 2** is the most robust in terms of ensuring that the properties hold, but it needs to be modified to avoid deadlocks (for example, by establishing a global order for acquiring locks). **Variant 3** is safe but sacrifices parallel execution efficiency.",
    "Let's analyze each variant of the `transfer` function to determine whether they maintain the two properties (non-negative balance and constant total sum) and whether they are prone to deadlocks.\n\n### Properties Check:\n\n1. **The balance of an account never goes below 0.**\n2. **The total sum of money held by the bank is constant.**\n\n### Variant 1:\n```scala\ndef transfer1(from: Account, to: Account, amount: Long): Unit = {\n  require(amount >= 0)\n  val balanceFrom = from.balance\n  if (balanceFrom >= amount) {\n    from.synchronized {\n      from.balance = balanceFrom - amount\n    }\n    to.synchronized {\n      val balanceTo = to.balance\n      to.balance = balanceTo + amount\n    }\n  }\n}\n```\n\n- **Non-negative Balance:** This implementation checks if `balanceFrom >= amount` before making any changes. It ensures that the withdrawal will not cause the account to go negative.\n- **Total Sum Constant:** The total sum is preserved because the amount is deducted from `from` and added to `to`.\n- **Deadlock Vulnerability:** Yes, this implementation is vulnerable to deadlocks. If two threads try to transfer money between the same two accounts in opposite directions, they can end up waiting on each other indefinitely.\n\n### Variant 2:\n```scala\ndef transfer2(from: Account, to: Account, amount: Long): Unit = {\n  require(amount >= 0)\n  from.synchronized {\n    val balanceFrom = from.balance\n    if (balanceFrom >= amount) {\n      from.balance = balanceFrom - amount\n      to.synchronized {\n        val balanceTo = to.balance\n        to.balance = balanceTo + amount\n      }\n    }\n  }\n}\n```\n\n- **Non-negative Balance:** The balance check is performed before any withdrawal is made, ensuring that the account will not go negative.\n- **Total Sum Constant:** The total amount remains unchanged as the amount is deducted from `from` and added to `to`.\n- **Deadlock Vulnerability:** This implementation is also vulnerable to deadlocks for the same reasons as Variant 1. If two threads try to transfer money between two accounts in opposite directions, they could wait indefinitely.\n\n### Variant 3:\n```scala\nobject lock // Global object\ndef transfer3(from: Account, to: Account, amount: Long): Unit = {\n  require(amount >= 0)\n  lock.synchronized {\n    val balanceFrom = from.balance\n    if (balanceFrom >= amount) {\n      from.balance = balanceFrom - amount\n      val balanceTo = to.balance\n      to.balance = balanceTo + amount\n    }\n  }\n}\n```\n\n- **Non-negative Balance:** Similar to the previous variants, this implementation checks that `balanceFrom >= amount` before any withdrawal, ensuring no account goes negative.\n- **Total Sum Constant:** The total balance remains the same as the amount is properly transferred between accounts.\n- **Deadlock Vulnerability:** This implementation is not prone to deadlocks. Since it uses a single global lock, only one thread can execute a transfer at a time, preventing any circular wait conditions.\n\n### Summary:\n- **Variant 1:** \n  - Non-negative balance: Yes\n  - Total sum constant: Yes\n  - Deadlock: Yes\n\n- **Variant 2:** \n  - Non-negative balance: Yes\n  - Total sum constant: Yes\n  - Deadlock: Yes\n\n- **Variant 3:** \n  - Non-negative balance: Yes\n  - Total sum constant: Yes\n  - Deadlock: No\n\n### Conclusion:\nTo ensure proper functioning in a concurrent environment, **Variant 3** is the only implementation that guarantees no deadlocks while still maintaining the required properties of the banking system. Variants 1 and 2 should be avoided due to their vulnerability to deadlocks.",
]
embeddings = model.encode(sentences)
print(embeddings.shape)
# [3, 768]

# Get the similarity scores for the embeddings
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# [3, 3]

Evaluation

Metrics

Triplet

Dataset: val-eval
Evaluated with TripletEvaluator

Metric	Value
cosine_accuracy	0.3251

Training Details

Training Dataset

Unnamed Dataset

Size: 19,392 training samples
Columns: sentence_0, sentence_1, and sentence_2
Approximate statistics based on the first 1000 samples:
sentence_0 sentence_1 sentence_2
type string string string
details
min: 5 tokens
mean: 97.99 tokens
max: 512 tokens
min: 3 tokens
mean: 355.29 tokens
max: 512 tokens
min: 3 tokens
mean: 326.38 tokens
max: 512 tokens

Samples:

sentence_0	sentence_1	sentence_2
`You are given the following collection of premises:[object Object]If I go to the museum, it either rains or snows.[object Object]I went to the museum on Saturday or I went to the museum on Sunday.[object Object]It did not rain and it did not snow on Saturday.[object Object]It did not rain on Sunday.[object Object]Which conclusions can be drawn from these premises ? (more than one answer can be correct)`	To analyze the premises you've provided and draw valid conclusions, let's break them down systematically.[object Object][object Object]1. [object Object]:[object Object] [object Object] a. [object Object]: "If I go to the museum, it either rains or snows." [object Object] - This is a conditional statement indicating that visiting the museum is accompanied by at least one of two weather conditions: rain or snow.[object Object][object Object] b. [object Object]: "I went to the museum on Saturday or I went to the museum on Sunday."[object Object] - This premise establishes that at least one of these two days must have involved a visit to the museum.[object Object][object Object] c. [object Object]: "It did not rain and it did not snow on Saturday."[object Object] - This explicitly states that the weather on Saturday was clear, meaning neither rain nor snow occurred.[object Object][object Object] d. [object Object]: "It did not rain on Sunday."[object Object] - This indicates that while we know it didn't rain on Sunday, we still lack information about whether it snowed on that day.[object Object][object Object]2. [object Object]:[object Object] [object Object] Now, let's evaluate the possible conclusions ba...	To address the premises presented, one must engage in a logical analysis. The first premise indicates that a visit to the museum necessitates either rain or snow. The second premise posits that I visited the museum either on Saturday or on Sunday. The third premise explicitly states that there was neither rain nor snow on Saturday. The fourth premise similarly confirms the absence of rain on Sunday.[object Object][object Object]Now, if we consider these premises collectively, we can deduce some insights. Since it did not rain or snow on Saturday, it logically follows that I could not have gone to the museum on that day, as the first premise would be violated. Therefore, the only remaining possibility for my visit to the museum must be Sunday.[object Object][object Object]However, since it also did not rain on Sunday, we again confront a contradiction with the first premise which stipulates that a visit to the museum requires either rain or snow. Hence, if I went to the museum on Sunday, it must have snowed, yet the premises state it did not...
`Let $\mathcal C_1$ be a linear code over $\mathbb F_3^n$, and let $\mathcal C_2$ be a linear code over $\mathbb F_2^n$. True or false: $\mathcal C_1 \cap \mathcal C_2$ is necessarily a linear code over $\mathbb F_3^n$.`	To determine whether the intersection (\mathcal C_1 \cap \mathcal C_2) is necessarily a linear code over (\mathbb F_3^n), we will analyze the properties of linear codes and their intersections.[object Object][object Object]1. [object Object]: [object Object] - A linear code over a field (\mathbb F_q) is a subspace of (\mathbb F_q^n). This means that any linear combination of codewords in the code is also a codeword.[object Object][object Object]2. [object Object]:[object Object] - (\mathcal C_1) is a linear code over (\mathbb F_3^n), meaning its codewords are vectors in (\mathbb F_3^n).[object Object] - (\mathcal C_2) is a linear code over (\mathbb F_2^n), meaning its codewords are vectors in (\mathbb F_2^n).[object Object][object Object]3. [object Object]:[object Object] - The intersection (\mathcal C_1 \cap \mathcal C_2) consists of codewords that belong to both (\mathcal C_1) and (\mathcal C_2).[object Object][object Object]4. [object Object]:[object Object] - A codeword in (\mathcal C_1) is represented as a vector in (\mathbb F_3^n), while a c...	The statement is [object Object]. [object Object][object Object]To analyze this, we must consider the nature of the codes involved. The linear code $\mathcal{C}_1$ is defined over the field $\mathbb{F}_3$, meaning that its codewords are formed using coefficients from the field with three elements. On the other hand, $\mathcal{C}_2$ is defined over the field $\mathbb{F}_2$, which consists of two elements.[object Object][object Object]The intersection $\mathcal{C}_1 \cap \mathcal{C}_2$ consists of codewords that are common to both codes. However, the elements of $\mathcal{C}_1$ can take on values from ${0, 1, 2}$ (the elements of $\mathbb{F}_3$), while the elements of $\mathcal{C}_2$ can only take on values from ${0, 1}$ (the elements of $\mathbb{F}_2$). [object Object][object Object]This means that codewords in $\mathcal{C}_1$ that contain the element '2' cannot be part of $\mathcal{C}_2$, as '2' does not exist in $\mathbb{F}_2$. Consequently, the intersection may not retain the linearity properties necessary to be considered a linear code over $\mathbb{F}_3$, as it may ...
Consider the (toy) grammar $G$ consisting of the following rules:[object Object]R1: S --> NP VP[object Object]R2: NP --> NN[object Object]R3: NP --> Det NN[object Object]R4: NN --> N[object Object]R5: NN --> NN NN[object Object]R6: NN --> NN PNP[object Object]R7: PNP --> Prep NP[object Object]R8: VP --> V[object Object]R9: VP --> Adv V[object Object]Precisely define the type of grammar G is corresponding to (for that, consider at least the following aspects: dependency-based vs. constituency-based, position in the Chomsky hierarchy, and CNF).[object Object]Justify your answer for each of the aspects you will be mentioning.	Let’s analyze the grammar ( G ) in detail, considering the specified aspects: dependency-based vs. constituency-based, position in the Chomsky hierarchy, and Chomsky Normal Form (CNF). [object Object][object Object]### 1. Dependency-based vs. Constituency-based[object Object][object Object][object Object][object Object]- [object Object] Focuses on the relationships between words, where the structure is determined by the dependencies and how words depend on each other.[object Object]- [object Object] Emphasizes how words group into larger units (constituents), creating hierarchical structures such as phrases and clauses.[object Object][object Object][object Object][object Object]The grammar ( G ) is [object Object]. This conclusion arises because:[object Object]- The grammar defines phrases (like Noun Phrases (NP) and Verb Phrases (VP)) that can be broken down into smaller components. For example, ( S \to NP VP ) indicates that a sentence (S) is composed of a noun phrase and a verb phrase, reflecting a hierarchical structure.[object Object]- The use of multiple non-terminal symbols (like NN, NP, ...	To classify the grammar ( G ) according to the specified aspects more comprehensively, let's re-evaluate the grammar's characteristics with a clear structure, ensuring that each aspect is thoroughly justified.[object Object][object Object]### 1. Dependency-based vs. Constituency-based[object Object][object Object][object Object]: [object Object][object Object][object Object][object Object]:[object Object]- [object Object]: Constituency-based grammars organize sentences into phrases (constituents) that can be recursively defined, which is a key characteristic of many natural languages.[object Object]- [object Object]: The production rules in grammar ( G ) demonstrate a clear hierarchical structure:[object Object] - The rule ( S \rightarrow NP \ VP ) signifies that a sentence (S) is composed of a noun phrase (NP) and a verb phrase (VP), which can further break down into smaller constituents.[object Object] - The rules for noun phrases (e.g., ( NP \rightarrow NN ) and ( NP \rightarrow Det \ NN )) show how constituents can be formed from other constituents, indicating a constituency-based structure.[object Object]- **Comp...

Loss: TripletLoss with these parameters:
json
1{ 2 "distance_metric": "TripletDistanceMetric.EUCLIDEAN", 3 "triplet_margin": 5 4}

Training Hyperparameters

Non-Default Hyperparameters

eval_strategy: steps
num_train_epochs: 1
multi_dataset_batch_sampler: round_robin

All Hyperparameters

Click to expand

overwrite_output_dir: False
do_predict: False
eval_strategy: steps
prediction_loss_only: True
per_device_train_batch_size: 8
per_device_eval_batch_size: 8
per_gpu_train_batch_size: None
per_gpu_eval_batch_size: None
gradient_accumulation_steps: 1
eval_accumulation_steps: None
torch_empty_cache_steps: None
learning_rate: 5e-05
weight_decay: 0.0
adam_beta1: 0.9
adam_beta2: 0.999
adam_epsilon: 1e-08
max_grad_norm: 1
num_train_epochs: 1
max_steps: -1
lr_scheduler_type: linear
lr_scheduler_kwargs: {}
warmup_ratio: 0.0
warmup_steps: 0
log_level: passive
log_level_replica: warning
log_on_each_node: True
logging_nan_inf_filter: True
save_safetensors: True
save_on_each_node: False
save_only_model: False
restore_callback_states_from_checkpoint: False
no_cuda: False
use_cpu: False
use_mps_device: False
seed: 42
data_seed: None
jit_mode_eval: False
use_ipex: False
bf16: False
fp16: False
fp16_opt_level: O1
half_precision_backend: auto
bf16_full_eval: False
fp16_full_eval: False
tf32: None
local_rank: 0
ddp_backend: None
tpu_num_cores: None
tpu_metrics_debug: False
debug: []
dataloader_drop_last: False
dataloader_num_workers: 0
dataloader_prefetch_factor: None
past_index: -1
disable_tqdm: False
remove_unused_columns: True
label_names: None
load_best_model_at_end: False
ignore_data_skip: False
fsdp: []
fsdp_min_num_params: 0
fsdp_config: {'min_num_params': 0, 'xla': False, 'xla_fsdp_v2': False, 'xla_fsdp_grad_ckpt': False}
tp_size: 0
fsdp_transformer_layer_cls_to_wrap: None
accelerator_config: {'split_batches': False, 'dispatch_batches': None, 'even_batches': True, 'use_seedable_sampler': True, 'non_blocking': False, 'gradient_accumulation_kwargs': None}
deepspeed: None
label_smoothing_factor: 0.0
optim: adamw_torch
optim_args: None
adafactor: False
group_by_length: False
length_column_name: length
ddp_find_unused_parameters: None
ddp_bucket_cap_mb: None
ddp_broadcast_buffers: False
dataloader_pin_memory: True
dataloader_persistent_workers: False
skip_memory_metrics: True
use_legacy_prediction_loop: False
push_to_hub: False
resume_from_checkpoint: None
hub_model_id: None
hub_strategy: every_save
hub_private_repo: None
hub_always_push: False
gradient_checkpointing: False
gradient_checkpointing_kwargs: None
include_inputs_for_metrics: False
include_for_metrics: []
eval_do_concat_batches: True
fp16_backend: auto
push_to_hub_model_id: None
push_to_hub_organization: None
mp_parameters:
auto_find_batch_size: False
full_determinism: False
torchdynamo: None
ray_scope: last
ddp_timeout: 1800
torch_compile: False
torch_compile_backend: None
torch_compile_mode: None
include_tokens_per_second: False
include_num_input_tokens_seen: False
neftune_noise_alpha: None
optim_target_modules: None
batch_eval_metrics: False
eval_on_start: False
use_liger_kernel: False
eval_use_gather_object: False
average_tokens_across_devices: False
prompts: None
batch_sampler: batch_sampler
multi_dataset_batch_sampler: round_robin

Training Logs

Epoch	Step	Training Loss	val-eval_cosine_accuracy
0.2063	500	4.9757	0.3554
0.4125	1000	4.9463	0.3216
0.6188	1500	4.978	0.2935
0.8251	2000	4.9374	0.3061
1.0	2424	-	0.3251

Framework Versions

Python: 3.12.8
Sentence Transformers: 4.1.0
Transformers: 4.51.3
PyTorch: 2.7.1+cu126
Accelerate: 1.7.0
Datasets: 3.2.0
Tokenizers: 0.21.0

Citation

BibTeX

Sentence Transformers

bibtex

@inproceedings{reimers-2019-sentence-bert,
    title = "Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks",
    author = "Reimers, Nils and Gurevych, Iryna",
    booktitle = "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing",
    month = "11",
    year = "2019",
    publisher = "Association for Computational Linguistics",
    url = "https://arxiv.org/abs/1908.10084",
}

TripletLoss

bibtex

@misc{hermans2017defense,
    title={In Defense of the Triplet Loss for Person Re-Identification},
    author={Alexander Hermans and Lucas Beyer and Bastian Leibe},
    year={2017},
    eprint={1703.07737},
    archivePrefix={arXiv},
    primaryClass={cs.CV}
}

	sentence_0	sentence_1	sentence_2
type	string	string	string
details	min: 5 tokens mean: 97.99 tokens max: 512 tokens	min: 3 tokens mean: 355.29 tokens max: 512 tokens	min: 3 tokens mean: 326.38 tokens max: 512 tokens