[!TIP]
The MultiMolecule team has confirmed that the provided model and checkpoints are producing the same intermediate representations as the original implementation.
The team releasing SpTransformer did not write this model card for this model so this model card has been written by the MultiMolecule team.
Model Details
SpTransformer (SpliceTransformer) is a deep neural network that predicts tissue-specific splicing from primary pre-mRNA sequence.
It combines two pretrained SpliceAI-style dilated-residual convolutional feature extractors with a trainable input-projection path; the concatenated features are processed by a Sinkhorn transformer attention block with axial positional embeddings.
For each position the network predicts a 3-channel splice-site score (no-splice / acceptor / donor) and a per-position splice-site usage score across 15 human tissues.
The model uses a fixed flanking context of 4,000 nucleotides on each side of every predicted position.
SpTransformer is typically used to estimate the effect of genetic variants on tissue-specific splicing by scoring reference and alternate sequences and taking the difference.
Please refer to the Training Details section for more information on the training process.
The logits tensor reproduces the original SpTransformer output: a 3-channel splice-site score (no-splice / acceptor / donor) and a per-tissue (15 tissues) splice-site usage score for each position.
Downstream Use
Token Prediction
You can fine-tune SpTransformer for per-nucleotide tissue-specific splicing regression with [SpTransformerForTokenPrediction][multimolecule.models.SpTransformerForTokenPrediction], which adds a shared token prediction head on top of the backbone.
Interface
Input length: variable pre-mRNA sequence
Flanking context: fixed 4,000 nt on each side of every predicted position
SpTransformer was trained to predict tissue-specific splicing from primary pre-mRNA sequence.
Training Data
SpTransformer was trained on splicing measurements derived from RNA-seq data across 15 human tissues, using gene annotations from GENCODE, together with multi-species sequence data.
The two convolutional feature extractors were pre-trained as SpliceAI-style splice-site predictors and remain trainable submodules for downstream fine-tuning.
For each predicted nucleotide, a sequence window centered on that nucleotide was used, with the flanking context padded with N (unknown nucleotide) when near transcript ends.
Training Procedure
Pre-training
The model was trained to minimize a combination of cross-entropy loss over splice-site classification and a regression loss over per-tissue splice-site usage, comparing predictions against measurements derived from RNA-seq.
Citation
bibtex
1@article{You2024,
2 author = {You, Ningyuan and Liu, Chang and Gu, Yuxin and Wang, Rong and Jia, Hanying and Zhang, Tianyun and Jiang, Song and Shi, Jinsong and Chen, Ming and Guan, Min-Xin and Sun, Siqi and Pei, Shanshan and Liu, Zhihong and Shen, Ning},
3 title = {{SpliceTransformer predicts tissue-specific splicing linked to human diseases}},
4 journal = {Nature Communications},
5 year = {2024},
6 volume = {15},
7 number = {1},
8 pages = {9129},
9 month = {oct},
10 doi = {10.1038/s41467-024-53088-6},
11 issn = {2041-1723},
12 url = {https://doi.org/10.1038/s41467-024-53088-6}
13}
[!NOTE]
The artifacts distributed in this repository are part of the MultiMolecule project.
If MultiMolecule supports your research, please cite the MultiMolecule project as follows:
bibtex
1@software{chen_2024_12638419,
2 author = {Chen, Zhiyuan and Zhu, Sophia Y.},
3 title = {MultiMolecule},
4 doi = {10.5281/zenodo.12638419},
5 publisher = {Zenodo},
6 url = {https://doi.org/10.5281/zenodo.12638419},
7 year = 2024,
8 month = may,
9 day = 4
10}
Contact
Please use GitHub issues of MultiMolecule for any questions or comments on the model card.
Please contact the authors of the SpliceTransformer paper for questions or comments on the paper/model.
