SentenceTransformer based on lufercho/my-finetuned-bert-mlm

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: lufercho/my-finetuned-bert-mlm
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'max_seq_length': 512, 'do_lower_case': False}) with Transformer model: BertModel 
  (1): Pooling({'word_embedding_dimension': 768, 'pooling_mode_cls_token': False, 'pooling_mode_mean_tokens': True, 'pooling_mode_max_tokens': False, 'pooling_mode_mean_sqrt_len_tokens': False, 'pooling_mode_weightedmean_tokens': False, 'pooling_mode_lasttoken': False, 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

Training Details

Training Dataset

Unnamed Dataset

• Size: 5,000 training samples
• Columns: sentence_0, sentence_1, and sentence_2
• Approximate statistics based on the first 1000 samples:

  	sentence_0	sentence_1	sentence_2
  type	string	string	string
  details	min: 4 tokens mean: 13.41 tokens max: 38 tokens	min: 37 tokens mean: 201.32 tokens max: 512 tokens	min: 24 tokens mean: 204.09 tokens max: 512 tokens

• Samples:

  sentence_0	sentence_1	sentence_2
  Clustering with Transitive Distance and K-Means Duality	Recent spectral clustering methods are a propular and powerful technique for[object Object]data clustering. These methods need to solve the eigenproblem whose[object Object]computational complexity is $O(n^3)$, where $n$ is the number of data samples.[object Object]In this paper, a non-eigenproblem based clustering method is proposed to deal[object Object]with the clustering problem. Its performance is comparable to the spectral[object Object]clustering algorithms but it is more efficient with computational complexity[object Object]$O(n^2)$. We show that with a transitive distance and an observed property,[object Object]called K-means duality, our algorithm can be used to handle data sets with[object Object]complex cluster shapes, multi-scale clusters, and noise. Moreover, no[object Object]parameters except the number of clusters need to be set in our algorithm.[object Object]	We show that the log-likelihood of several probabilistic graphical models is[object Object]Lipschitz continuous with respect to the lp-norm of the parameters. We discuss[object Object]several implications of Lipschitz parametrization. We present an upper bound of[object Object]the Kullback-Leibler divergence that allows understanding methods that penalize[object Object]the lp-norm of differences of parameters as the minimization of that upper[object Object]bound. The expected log-likelihood is lower bounded by the negative lp-norm,[object Object]which allows understanding the generalization ability of probabilistic models.[object Object]The exponential of the negative lp-norm is involved in the lower bound of the[object Object]Bayes error rate, which shows that it is reasonable to use parameters as[object Object]features in algorithms that rely on metric spaces (e.g. classification,[object Object]dimensionality reduction, clustering). Our results do not rely on specific[object Object]algorithms for learning the structure or parameters. We show preliminary[object Object]results for activity recognition and temporal segmentation.[object Object]
  Clustering Dynamic Web Usage Data	Most classification methods are based on the assumption that data conforms to[object Object]a stationary distribution. The machine learning domain currently suffers from a[object Object]lack of classification techniques that are able to detect the occurrence of a[object Object]change in the underlying data distribution. Ignoring possible changes in the[object Object]underlying concept, also known as concept drift, may degrade the performance of[object Object]the classification model. Often these changes make the model inconsistent and[object Object]regular updatings become necessary. Taking the temporal dimension into account[object Object]during the analysis of Web usage data is a necessity, since the way a site is[object Object]visited may indeed evolve due to modifications in the structure and content of[object Object]the site, or even due to changes in the behavior of certain user groups. One[object Object]solution to this problem, proposed in this article, is to update models using[object Object]summaries obtained by means of an evolutionary approach based on an intelligent[object Object]clustering approach. We carry out various clustering str...	Exponential family extensions of principal component analysis (EPCA) have[object Object]received a considerable amount of attention in recent years, demonstrating the[object Object]growing need for basic modeling tools that do not assume the squared loss or[object Object]Gaussian distribution. We extend the EPCA model toolbox by presenting the first[object Object]exponential family multi-view learning methods of the partial least squares and[object Object]canonical correlation analysis, based on a unified representation of EPCA as[object Object]matrix factorization of the natural parameters of exponential family. The[object Object]models are based on a new family of priors that are generally usable for all[object Object]such factorizations. We also introduce new inference strategies, and[object Object]demonstrate how the methods outperform earlier ones when the Gaussianity[object Object]assumption does not hold.[object Object]
  Trading USDCHF filtered by Gold dynamics via HMM coupling	We devise a USDCHF trading strategy using the dynamics of gold as a filter.[object Object]Our strategy involves modelling both USDCHF and gold using a coupled hidden[object Object]Markov model (CHMM). The observations will be indicators, RSI and CCI, which[object Object]will be used as triggers for our trading signals. Upon decoding the model in[object Object]each iteration, we can get the next most probable state and the next most[object Object]probable observation. Hopefully by taking advantage of intermarket analysis and[object Object]the Markov property implicit in the model, trading with these most probable[object Object]values will produce profitable results.[object Object]	Most existing machine learning classifiers are highly vulnerable to[object Object]adversarial examples. An adversarial example is a sample of input data which[object Object]has been modified very slightly in a way that is intended to cause a machine[object Object]learning classifier to misclassify it. In many cases, these modifications can[object Object]be so subtle that a human observer does not even notice the modification at[object Object]all, yet the classifier still makes a mistake. Adversarial examples pose[object Object]security concerns because they could be used to perform an attack on machine[object Object]learning systems, even if the adversary has no access to the underlying model.[object Object]Up to now, all previous work have assumed a threat model in which the adversary[object Object]can feed data directly into the machine learning classifier. This is not always[object Object]the case for systems operating in the physical world, for example those which[object Object]are using signals from cameras and other sensors as an input. This paper shows[object Object]that even in such physical world scenarios, machine learning systems are[object Object]vul...

• Loss: TripletLoss with these parameters:
  json

  1{ 2 "distance_metric": "TripletDistanceMetric.EUCLIDEAN", 3 "triplet_margin": 5 4}

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• per_device_eval_batch_size: 16
• num_train_epochs: 2
• multi_dataset_batch_sampler: round_robin

All Hyperparameters

Training Logs

Framework Versions

• Python: 3.10.12
• Sentence Transformers: 3.3.1
• Transformers: 4.46.2
• PyTorch: 2.5.1+cu121
• Accelerate: 1.1.1
• Datasets: 3.1.0
• Tokenizers: 0.20.3

Citation

BibTeX

Sentence Transformers

TripletLoss