CULTURE: Curve Understanding and Learning Transformer for Unique Replication Estimation
CULTURE (Curve Understanding and Learning Transformer for Unique Replication Estimation) is a custom BERT-based model with a Masked Language Model (MLM) head for processing microbial growth curve data. The model is trained on 1x128 vector inputs representing microbial growth curves and can generate hidden state embeddings for downstream tasks in microbiology and bioinformatics.
Background
Microbial growth curves are time-series data that represent the growth of microorganisms over time. Each 1x128 vector in our dataset represents a single growth curve, with 128 time points measuring microbial population or density. By applying CULTURE to this data, we aim to capture complex patterns and relationships within these growth curves, potentially enabling better analysis and prediction in microbiology research.
Requirements
The project requires Python 3.7+ and the following packages:
PyTorch 1.9.0
Transformers 4.11.3
pandas 1.3.3
numpy 1.21.2
tqdm 4.62.3
You can install the required packages using the provided requirements.txt file:
pip install -r requirements.txt
File Structure
config.py: Contains all configurable parameters for the CULTURE model and training process.
model.py: Defines the CULTURE model architecture.
train.py: Script for training the CULTURE model.
test.py: Script for evaluating the CULTURE model's performance on the MLM task.
get_embeddings.py: Script for generating embeddings using the trained CULTURE model.
requirements.txt: List of required Python packages.
train.csv: Training data file containing microbial growth curves (not included in this repository).
val.csv: Validation data file containing microbial growth curves (not included in this repository).
test.csv: Test data file containing microbial growth curves (not included in this repository).
Usage
Prepare your data:
Ensure you have train.csv, val.csv, and test.csv files in the project directory.
Each file should contain 1x128 vector data points representing microbial growth curves, without headers.
Configure the model:
Open config.py and adjust the hyperparameters as needed.
You can modify the hidden size (hidden_size), number of encoder layers (num_hidden_layers), number of attention heads (num_attention_heads), and masking probability (mlm_probability) among other parameters.
Train the model:
python train.py
This will train the CULTURE model on your microbial growth curve data and save it as culture_model.pth.
Evaluate the model:
python test.py
This will load the trained CULTURE model and evaluate its performance on the MLM task using the test data.
This will load the trained CULTURE model, process the input growth curve data, and save the embeddings as a NumPy file.
Customization
To use different input dimensions (e.g., if your growth curves have a different number of time points), modify the input_dim parameter in config.py.
Adjust learning rate, batch size, and other training parameters in config.py.
For more advanced modifications, you can edit the model architecture in model.py.
Output
After running get_embeddings.py, you'll get a NumPy file containing the hidden state embeddings for your input data. These embeddings can be used for downstream tasks.
Note
CULTURE assumes that all growth curves are sampled at consistent time intervals and have been preprocessed to have the same length (128 time points). If your data differs significantly from this format, you may need to preprocess it or adjust the model architecture accordingly.
