chowder-embed (SPECTER fine-tuned on the chowder corpus)

Model Details

Model Description

• Model Type: Sentence Transformer
• Base model: sentence-transformers/allenai-specter
• Maximum Sequence Length: 512 tokens
• Output Dimensionality: 768 dimensions
• Similarity Function: Cosine Similarity
• Supported Modality: Text
• Training Dataset:
  • parquet
• Language: en

Model Sources

• Documentation: Sentence Transformers Documentation
• Repository: Sentence Transformers on GitHub
• Hugging Face: Sentence Transformers on Hugging Face

Full Model Architecture

SentenceTransformer(
  (0): Transformer({'transformer_task': 'feature-extraction', 'modality_config': {'text': {'method': 'forward', 'method_output_name': 'last_hidden_state'}}, 'module_output_name': 'token_embeddings', 'architecture': 'BertModel'})
  (1): Pooling({'embedding_dimension': 768, 'pooling_mode': 'cls', 'include_prompt': True})
)

Usage

Direct Usage (Sentence Transformers)

Evaluation

Metrics

Information Retrieval

• Datasets: NanoSCIDOCS, NanoSciFact, NanoNFCorpus and chowder-val
• Evaluated with InformationRetrievalEvaluator

Nano BEIR

• Dataset: NanoBEIR_mean
• Evaluated with NanoBEIREvaluator with these parameters:
  json

  1{ 2 "dataset_names": [ 3 "scidocs", 4 "scifact", 5 "nfcorpus" 6 ], 7 "dataset_id": "sentence-transformers/NanoBEIR-en" 8}

Training Details

Training Dataset

parquet

• Dataset: parquet
• Size: 70,764 training samples
• Columns: anchor and positive
• Approximate statistics based on the first 100 samples:

  	anchor	positive
  type	string	string
  modality	text	text
  details	min: 8 tokens mean: 18.96 tokens max: 51 tokens	min: 54 tokens mean: 276.39 tokens max: 512 tokens

• Samples:

  anchor	positive
  Los-mamba: A low-rank recursive mamba framework for mitigating stationary bias in trajectory prediction	Trajectory prediction plays a key role in autonomous driving and intelligent transportation systems. Mamba performs well in modeling long sequences but struggles with short-term static or local motion features. In this paper, we propose the Lo w-rank Recur s ive Mamba framework ( Los-Mamba ). It addresses prediction bias caused by inadequate modeling of local static features in trajectory prediction tasks. Firstly, Los-Mamba uses hybrid Mamba branches to capture both global motion trends and local features. Secondly, it reduces trajectory feature dimensionality through low-rank approximation, simplifying computational complexity. Thirdly, Los-Mamba also includes a spatio-temporal constraint mechanism. It uses position coding and a multilayer perceptron (MLP) to interpolate static and dynamic trajectories, improving the representation of local features. Experiments show that Los-Mamba outperforms baseline models on datasets like ETH and Hotel. Its average displacement error (ADE) and fi...
  The Universal Language of CSI:Unifying Wireless Sensing Across Devices and Environments	WiFi sensing based on Channel State Information (CSI) promises ubiquitous, device-free perception, yet current research remains trapped in a Tower of Babel - fragmented into isolated silos where models are tailored to specific hardware dialects, fixed environments, and narrow tasks. The primary bottleneck is the Heterogeneity Gap: the disparity in signal dimensions, sampling rates, and semantic labels that prevents cross-system understanding. To bridge this gap, we propose a foundation-model framework that treats CSI not merely as raw signals but as a structured language with a learnable universal grammar. We first curate and standardize a large collection of heterogeneous real-world CSI datasets, establishing a unified infrastructure that allows incompatible signal formats to be treated as a single corpus. Second, we introduce a modular architecture that acts as a universal translator where lightweight dataset-specific adapters tokenize diverse signal inputs into a shared latent vocab...
  Long-term follow-up of the transplant-eligible cohort of the EMN12/HOVON-129 study for primary plasma cell leukemia patients	Primary plasma cell leukemia (pPCL) is a rare and aggressive plasma cell disorder [ 1 , 2 ]. Compared to multiple myeloma (MM), pPCL is characterized by a higher frequency of poor-risk features, such as high-risk cytogenetic abnormalities (HRCAs) and elevated LDH [ 1 , 2 ]. Although MM treatment has improved, the survival of pPCL patients remains poor [ 1 , 2 ]. We therefore initiated the phase 2 EMN12/HOVON-129 study for patients with newly diagnosed pPCL, which showed that carfilzomib and lenalidomide-based therapy in conjunction with autologous stem-cell-transplantation (ASCT) provides efficient disease control [ 3 ]. Here, we present the long-term follow-up of patients aged 18–65 years included in the EMN12/HOVON-129 study. We also aimed to evaluate if patients with HRCAs, elevated LDH, or other high-risk features have a worse clinical outcome. Inclusion criteria were newly diagnosed pPCL (defined as >2 × 10 9 /L circulating plasma cells or >20% plasma cells in peripheral blood) an...

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false, 5 "directions": [ 6 "query_to_doc" 7 ], 8 "partition_mode": "joint", 9 "hardness_mode": null, 10 "hardness_strength": 0.0 11}

Evaluation Dataset

parquet

• Dataset: parquet
• Size: 2,000 evaluation samples
• Columns: anchor and positive
• Approximate statistics based on the first 100 samples:

  	anchor	positive
  type	string	string
  modality	text	text
  details	min: 7 tokens mean: 19.21 tokens max: 37 tokens	min: 68 tokens mean: 278.03 tokens max: 512 tokens

• Samples:

  anchor	positive
  Boolean Algebra -- Driven Sepsis Diagnosis	Sepsis remains a diagnostic challenge due to its heterogeneous molecular signatures and complex immune responses. In this study, we develop a logical data analysis framework based on Boolean polynomial rings. This method constructs an ideal $\mathcal{I}$ of selection criteria that isolate empty subsets of previously analyzed patient data. This approach enables the derivation of interpretable classification rules based on biomarker profiles. We demonstrate that logical data analysis identifies distinct logical patterns for positive and negative sepsis classification. For instance, elevated levels of GLP-1 and MyD88 are associated with septic states in our dataset, whereas high TRAIL and low MyD88 concentrations may suggest a non-septic condition. Importantly, a new way to integrate expert knowledge to filter out potential overfitting or dataset-specific artifacts is shown. Our findings highlight the utility of logics in generating transparent, biologically plausible rules for a data-bas...
  Notes on the bootstrap of four-point conformal integrals	We set up a bootstrap workflow to study four-point conformal integrals in position space, using leading singularities, single-valued multiple polylogarithmic ans"atze and boundary data from expansion by regions. These four-point conformal integrals are general in the sense that they are generated by the four-point projections of all possible $f$-graphs, including all non-planar $f$-graph sectors. For three-loop cases, fourteen of the fifteen inequivalent integrand basis can be directly calculated by \texttt{HyperlogProcedures} and the last one is fixed by Gram identity. Then we concentrate on how far the bootstrap workflow can go for four-loop cases, though it works for three-loop cases as well. We show that integrals with several leading singularities can be made tractable by decomposing them into pieces with simpler cut structure. Some four-loop integrals which can not be calculated or very hard to be calculated by other methods for now are obtained in this way. We also provide a pa...
  Clean2FX: Label-conditioned modeling for clean-to-effect guitar audio transformations	We present Clean2FX, a study and demo of label-conditioned clean-to-effect transformation for electric guitar audio. Given a clean guitar input and a target effect label, the task is to synthesize the corresponding effected signal while preserving the musical content. Training and evaluation pairs are constructed from EGFxSet real, single tone recordings by assembling matched clean/effected chords, melodies, and mixed timelines. This allows for controlled comparison across effects. We evaluate four neural approaches under a common spectrogram-based transformation setting: two variational autoencoders and two U-Net models that differ in whether they operate on linear or log-magnitude representations. Performance is measured using linear-magnitude spectrogram MSE and Fr'echet Audio Distance. The U-Net models outperform the variational autoencoder variants. Per-effect results show that distortion effects are most readily improved, whereas delay and reverb effects exhibit weaker FAD gains...

• Loss: MultipleNegativesRankingLoss with these parameters:
  json

  1{ 2 "scale": 20.0, 3 "similarity_fct": "cos_sim", 4 "gather_across_devices": false, 5 "directions": [ 6 "query_to_doc" 7 ], 8 "partition_mode": "joint", 9 "hardness_mode": null, 10 "hardness_strength": 0.0 11}

Training Hyperparameters

Non-Default Hyperparameters

• per_device_train_batch_size: 16
• learning_rate: 2e-05
• warmup_steps: 0.1
• weight_decay: 0.01
• fp16: True
• per_device_eval_batch_size: 16
• load_best_model_at_end: True
• seed: 12
• batch_sampler: no_duplicates

All Hyperparameters

Training Logs

• The bold row denotes the saved checkpoint.

Training Time

• Training: 44.8 minutes
• Evaluation: 3.9 minutes
• Total: 48.7 minutes

Framework Versions

• Python: 3.14.7
• Sentence Transformers: 5.7.0
• Transformers: 5.15.0
• PyTorch: 2.13.0+cu130
• Accelerate: 1.14.0
• Datasets: 5.0.1
• Tokenizers: 0.22.2

Additional Resources

• Training and Finetuning Embedding Models with Sentence Transformers: the end-to-end guide for training or finetuning Sentence Transformer models.
• Introduction to Matryoshka Embedding Models: variable-size embeddings that can be truncated with minimal quality loss.
• Binary and Scalar Embedding Quantization for Significantly Faster & Cheaper Retrieval: post-training compression of embedding vectors.
• Multimodal Embedding & Reranker Models with Sentence Transformers: use text, image, audio, and video models through the same API.
• Training and Finetuning Multimodal Embedding & Reranker Models with Sentence Transformers: train multimodal embedding models, with a Visual Document Retrieval walkthrough.

Citation

BibTeX

Sentence Transformers

MultipleNegativesRankingLoss