SentenceTransformer based on lufercho/my-finetuned-bert-mlm

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: lufercho/my-finetuned-bert-mlm
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 5,000 training samples
• Columns: sentence_0 and sentence_1
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1
  type	string	string
  details	min: 4 tokens mean: 13.29 tokens max: 56 tokens	min: 26 tokens mean: 202.49 tokens max: 506 tokens

• Samples:

  sentence_0	sentence_1
  Validation of nonlinear PCA	Linear principal component analysis (PCA) can be extended to a nonlinear PCA[object Object]by using artificial neural networks. But the benefit of curved components[object Object]requires a careful control of the model complexity. Moreover, standard[object Object]techniques for model selection, including cross-validation and more generally[object Object]the use of an independent test set, fail when applied to nonlinear PCA because[object Object]of its inherent unsupervised characteristics. This paper presents a new[object Object]approach for validating the complexity of nonlinear PCA models by using the[object Object]error in missing data estimation as a criterion for model selection. It is[object Object]motivated by the idea that only the model of optimal complexity is able to[object Object]predict missing values with the highest accuracy. While standard test set[object Object]validation usually favours over-fitted nonlinear PCA models, the proposed model[object Object]validation approach correctly selects the optimal model complexity.[object Object]
  Learning Attitudes and Attributes from Multi-Aspect Reviews	The majority of online reviews consist of plain-text feedback together with a[object Object]single numeric score. However, there are multiple dimensions to products and[object Object]opinions, and understanding the `aspects' that contribute to users' ratings may[object Object]help us to better understand their individual preferences. For example, a[object Object]user's impression of an audiobook presumably depends on aspects such as the[object Object]story and the narrator, and knowing their opinions on these aspects may help us[object Object]to recommend better products. In this paper, we build models for rating systems[object Object]in which such dimensions are explicit, in the sense that users leave separate[object Object]ratings for each aspect of a product. By introducing new corpora consisting of[object Object]five million reviews, rated with between three and six aspects, we evaluate our[object Object]models on three prediction tasks: First, we use our model to uncover which[object Object]parts of a review discuss which of the rated aspects. Second, we use our model[object Object]to summarize reviews, which for us means finding the sentences...
  Bayesian Differential Privacy through Posterior Sampling	Differential privacy formalises privacy-preserving mechanisms that provide[object Object]access to a database. We pose the question of whether Bayesian inference itself[object Object]can be used directly to provide private access to data, with no modification.[object Object]The answer is affirmative: under certain conditions on the prior, sampling from[object Object]the posterior distribution can be used to achieve a desired level of privacy[object Object]and utility. To do so, we generalise differential privacy to arbitrary dataset[object Object]metrics, outcome spaces and distribution families. This allows us to also deal[object Object]with non-i.i.d or non-tabular datasets. We prove bounds on the sensitivity of[object Object]the posterior to the data, which gives a measure of robustness. We also show[object Object]how to use posterior sampling to provide differentially private responses to[object Object]queries, within a decision-theoretic framework. Finally, we provide bounds on[object Object]the utility and on the distinguishability of datasets. The latter are[object Object]complemented by a novel use of Le Cam's method to obtain lower bounds....

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim" 4}

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 2
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.46.2
• PyTorch: 2.5.1+cu121
• Accelerate: 1.1.1
• Datasets: 3.1.0
• Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss