SPOT-RNA

Disclaimer

[!TIP] The MultiMolecule team has confirmed that the provided model and checkpoints are producing the same intermediate representations as the original implementation.

Model Details

• pairwise representation: outer concatenation of canonical nucleotide features into an L x L x 8 feature matrix.
• convolutional blocks: 2D residual convolution blocks with LayerNorm, dropout, and checkpoint-matched ReLU/ELU activations.
• architecture paths: checkpoint-matched 2D-BLSTM or dilated-convolution paths where used by the released predictor.
• training strategy: transfer learning from bpRNA to high-resolution PDB RNA structures.

Model Specification

Links

• Code: multimolecule.spotrna
• Weights: multimolecule/spotrna
• Data: multimolecule/bprna-spot
• Paper: RNA secondary structure prediction using an ensemble of two-dimensional deep neural networks and transfer learning
• Developed by: Jaswinder Singh, Jack Hanson, Kuldip Paliwal, Yaoqi Zhou
• Original Repository: jaswindersingh2/SPOT-RNA

Usage

Direct Use

RNA Secondary Structure Pipeline

PyTorch Inference

Training Details

Training Data

• initial training source: bpRNA-1m (Version 1.0) with 102,348 annotated RNAs.
• initial training filtering: CD-HIT-EST at 80% sequence identity, removal of RNAs with PDB structures, and maximum sequence length of 500 nucleotides.
• initial training corpus: 13,419 RNAs after preprocessing.
• initial training split: TR0 = 10,814, VL0 = 1,300, TS0 = 1,305.
• transfer-learning source: high-resolution PDB RNAs downloaded on March 2, 2019.
• transfer-learning filtering: resolution better than 3.5 A and CD-HIT-EST at 80% sequence identity.
• transfer-learning corpus: 226 nonredundant RNAs after preprocessing.
• transfer-learning split before homology filtering: TR1 = 120, VL1 = 30, TS1 = 76.
• additional TS1 filtering: CD-HIT-EST against the training data at 80% identity, followed by BLAST-N against TR0 and TR1 with e-value cutoff 10.
• final TS1 benchmark: 67 RNAs.
• additional evaluation set: TS2 = 39 NMR-solved RNAs selected from 641 candidates after CD-HIT-EST filtering at 80% identity and BLAST-N filtering against TR0, TR1, and TS1.
• use of TS2: post-training evaluation only.

Training Procedure

Preprocessing

• input representation: one-hot L x 4 matrix following the MultiMolecule tokenizer order.
• missing-value handling: invalid or missing residues encoded as -1 in the original TensorFlow implementation before one-hot conversion.
• pairwise features: outer concatenation from L x 4 to L x L x 8.
• input normalization: standardization to zero mean and unit variance using training-set statistics.
• structure labels: extracted from PDB coordinates with DSSR.
• reference NMR model: model 1.
• pseudoknot and motif definitions: bpRNA definitions from the paper.
• unknown-token handling: N tokens are excluded from the canonical four-base features before pairwise feature construction.

Pre-training

• training split: TR0.
• validation split: VL0.
• optimizer: Adam.
• regularization: 25% dropout before convolution layers and 50% dropout in hidden fully connected layers.
• hyperparameter search over N_A: 16 to 32 residual blocks.
• hyperparameter search over D_RES: 32 to 72 convolution channels.
• hyperparameter search over D_BL: 128 to 256 2D-BLSTM hidden units per direction.
• hyperparameter search over N_B: 0 to 4 fully connected blocks.
• hyperparameter search over D_FC: 256 to 512 fully connected hidden units.
• model selection: validation-performance model selection described in the paper.

Transfer Learning

• initialization: start from the TR0-trained models.
• training split: TR1.
• validation split: VL1.
• frozen layers: none; all weights were updated.
• architecture and optimization settings: same as the TS0-trained models.
• model selection: validation-performance model selection described in the paper.
• decision rule: a single probability threshold chosen to optimize validation performance.

Citation

[!NOTE] The artifacts distributed in this repository are part of the MultiMolecule project. If MultiMolecule supports your research, please cite the MultiMolecule project as follows:

Contact

License