You can deploy this model and start earning money today!

NEO699700/
ACEP

Views

No views yet

ACEP (Adaptive Convergence by Embedding of Protein)

ACEP (Adaptive Convergence by Embedding of Protein) is used for detection of convergence of high-order protein feature.

Whole process of convergence detection

Permutation with simulation test (Permulation)

Content

Process of convergent signal detection
- Parameters prepare
- Preprocess of raw protein sequences
- Inference of evolutionary parameters and ancestral sequence reconstruction by PAML
- Simulation of null sequences under neutral circumstances
- Protein sequences embedding calculation based on pretrained ESM models
- Statistical test of convergence by pairwise distances

Usage

Environment deployment

The pip install process will cost some time because of the Model weight files are over 7GB.

shell

git clone https://huggingface.co/NEO699700/ACEP
cd ./ACEP/ACEP
conda env create -f environment.yml
conda activate ACEP
cd ..
pip install .

Quick start

The meaning of every parameter used here will be explained in Step by Step part. Users should only specify the paths of fasta file and log file.

python

from ACEP import Args, PreProcessor, Fileprepare, Simulators, Embedding, ConvStat

# specify the MSA fasta file and log file path
workfile = "./ACEP/examples/ENSG00000000003.fasta"
logpath = "./ACEP/examples/ACEP.log"
# specify the foreground species by list
case_sps1 = ['Physeter_catodon','Lipotes_vexillifer','Delphinapterus_leucas','Orcinus_orca','Tursiops_truncatus']
case_sps2 = ['Miniopterus_natalensis','Myotis_davidii','Myotis_brandtii','Myotis_lucifugus','Eptesicus_fuscus','Hipposideros_armiger','Rhinolophus_sinicus','Rousettus_aegyptiacus']
args = Args(workfile, logpath, case_sps1=case_sps1, case_sps2=case_sps2)
# specify the foreground species in ./examples/case_sps.txt
args = Args(workfile, logpath)

# preprocessing of raw MSA file
pp = PreProcessor(args)
pp.Clean()
# prepare the file for paml inference
fp = Fileprepare(args)
fp.Infer()
# simulating the null sequences based on paml inferred files
simer = Simulators(args)
simer.GenerateNull()
# embedding the sequences based on pretrained protein models
embeder = Embedding(args)
embeder.embedding()
# calculate the empirical P value
ConvS = ConvStat(args)
ConvS.Pval()

Asides from the above, users can simply use the Acep_test function from script example.py in package. The Acep_test function can redo the whole process or start from the breakpoint.

Step by step

Parameters prepare

workfile: the path of MSA fasta file
logpath: the path of log file
case_sps: default value, the txt file of foreground species, the first line is group 1 and the second line is group 2, separated by comma
case_sps1: default value, list of foreground species in group 1, specify the case_sps or this
case_sps2: default value, list of foreground species in group 2, specify the case_sps or this
sps_tree: default value, the path of species tree without branch length
ctl_file: default value, the path of control file pattern for paml inference
subs_mat: default value, the type of substitution model for paml inference
gap_prop: default value, the threshold of gap proportion of a deleted sequence
freq_mode: default value, amino acid frequency across gene or site in simulation
process_name: default value, name the process by yourself
sim_repeats: default value, number of simulation repeats
pretrained_esm: choose esm-msa-1b or esm-2, default value if esm-msa-1b
device: default value is cuda:0
batch: specify the number of sequence send into model in one batch
encoder: specify the type of bottleneck encoder, choose mammal or plant
onlyForeground: default value is True, only calculate the embeddings of foreground species
redo: if use the exmaple.py for ACEP test, redo means delete all cache file then redo the whole process\

[!WARNING]Important note: Must specify the onlyForeground parameter as False when using Permulation test.

python

from ACEP import Args

# specify the MSA fasta file and log file path
workfile = "ACEP/ACEP/examples/ENSG00000000003.fasta"
logpath = "ACEP/ACEP/examples/ACEP.log"
# specify the foreground species by list
case_sps1 = ['Physeter_catodon','Lipotes_vexillifer','Delphinapterus_leucas','Orcinus_orca','Tursiops_truncatus']
case_sps2 = ['Miniopterus_natalensis','Myotis_davidii','Myotis_brandtii','Myotis_lucifugus','Eptesicus_fuscus','Hipposideros_armiger','Rhinolophus_sinicus','Rousettus_aegyptiacus']
args = Args(workfile, logpath, case_sps1=case_sps1, case_sps2=case_sps2)
# specify the foreground species in ./examples/case_sps.txt
args = Args(workfile, logpath)

Preprocess of raw protein sequences and codon sequences

If the sequence length is greater than 1024 after gaps cleaning, then the fasta file will be split into multiple pieces.
Output will be multiple directories named by id of genes each contains a fasta file with suffix .clean.

python

from ACEP import PreProcessor
pp = PreProcessor(args)
pp.Clean()

Inference of evolutionary parameters and ancestral sequence reconstruction by PAML

Evolutionary rates and ancestral sequences are crucial parameters for simulating null sequences which represent the case that protein sequences evolve under neutral circumstances.
Output will be directory with suffix pamlparas containing control file, partial tree file and specified control file in each gene's corresponding directory.

python

from ACEP import Fileprepare
fp = Fileprepare(args)
fp.Infer()

We will obtain constructed ancestral sequence and tree topology with inferred branch lengths in rst file. In addition, every site's evolutionary rate is deposited in rates file. These parameters will be transferred for simulating null sequences in next step.

Simulation of null sequences under neutral circumstance

This step will simulate null protein sequences from constructed ancestral protein sequence in root node under neutral circumstance along the fixed tree topology with inferred branch length by PAML in former step.

There will be a new directory within each gene's directory ending with inferred_tree which contains tree nwk file with inferred branch lengths. And the simulated sequences is stored in seqid_sim100.fasta with default 100 times of simulation.

python

from ACEP import Simulators
simer = Simulators(args)
simer.GenerateNull()

Protein sequences embedding calculation based on ESM-MSA-1b and bottleneck

The script of this step intends to calculate the embeddings of protein sequences based on pretrained protein model and bottleneck full linear layer with parameters we obtain after training.

After this step, there will be four new files in each gene's directory. One with suffix embedding_real.npy save the original protein sequences' embeddings, another with suffix embedding_sim.npy contains all simulated sequences' corresponding embeddings. File with suffix embindex_real.json save the label of corresponding sequence in embedding_real.npy. File with suffix embindex_sim.json is the same.

python

from ACEP import Embedding
embeder = Embedding(args)
embeder.embedding()

Two statistical methods for convergence test

1. Statistical test of convergence by distances comparison

This script will calculate the distribution of distances of two focus species groups pair by pair using simulated MSAs and real MSA. And the species pairs are stored in file with suffix PairIndices.json.

Finally, file with suffix pvalues.json contains empirical P values of cosine distance and Euclidean distance.

python

from ACEP import ConvStat
ConvS = ConvStat(args)
ConvS.Pval()

The P value will be presented in the following form.

{"cos_pvalue": 0.32, "euc_pvalue": 0.39}

2. Statistical test of convergence by permutation with trait simulation (Permulation)

Permulation test, which based on RERconverge, will permutate binary traits in phylogenetic tree by macroevolution simulation, two foreground groups will be generated during each simulation. Mean / Minimum distance between two groups will be used as null distribution for empirical P value calculation.

Details of permulation test script is Permtest_example.sh in PermTest directory. Meanings of each parameter are show below.

shell

# Permulation
Rscript Perm.R Seqid 
        Times of permulation
        Path of species tree 
        Output path
        Foreground species file
        Inferred gene tree path
        Whether maintain the permulation result overlapped with original one

Rscript Perm.R ENSG00000000003_0 
        100 
        ./SpeciesAged.nwk 
        ./ 
        ./case_sps.txt 
        ../ACEP/examples/ENSG00000000003/ENSG00000000003_0/ENSG00000000003_0_inferred_tree/tree_inferred_branch.nwk
        TRUE

shell

# Statistical test based on permulation results
python PermTest.py Seqid 
        Times of permulation
        Path of permulation result
        Path of species tree 
        Foreground species file
        Inferred gene tree path
        The path of directory which contain original sequence embedding
        Output path
        Calculate the mean or minimum distance between foreground groups

python PermTest.py ENSG00000000003_0
                   100
                   ./ENSG00000000003_0/ENSG00000000003_0_100Permulated.txt
                   ./case_sps.txt 
                   ../ACEP/examples/ENSG00000000003/ENSG00000000003_0/ENSG00000000003_0_inferred_tree/tree_inferred_branch.nwk 
                   ../ACEP/examples/ENSG00000000003/ENSG00000000003_0/ENSG00000000003_0_alter_Embedding 
                   ./ 
                   Min

The empirical P value result will be stored in output path in json file and the P value will be presented in the following form.

{"cos_pvalue": 0.67, "euc_pvalue": 0.67}