GENATATOR-PIPELINE

Hugging Face pipeline usage

Basic example

Example with all main supported parameters defined

Parameter reference

Pipeline setup arguments

• task — Hugging Face task name for this custom pipeline. Use "genatator-pipeline".
• model — Hugging Face repository or local directory containing the GENATATOR pipeline wrapper. For the published version, use "AIRI-Institute/genatator-pipeline".
• trust_remote_code — Must be True because the pipeline uses custom Python code from the model repository. Without it, Transformers will not load the custom pipeline class.
• device — CUDA GPU index used for inference, for example 0 for the first GPU. CPU execution with device=-1 is not supported currently.
• dtype — Tensor dtype used when loading the stage models. Only "float32" is supported currently.

Model repositories

• edge_model_path — Repository or local path for the edge model. This model predicts transcript boundary signals, namely TSS and PolyA signals on both strands.
• region_model_path — Repository or local path for the region model. This model predicts strand-specific intragenic signal that is used to remove weak candidate transcript intervals.
• transcript_type_model_path — Repository or local path for the transcript-type classifier. This model labels each retained candidate interval as mRNA or lnc_RNA.
• segmentation_model_path — Repository or local path for the segmentation model. This model predicts the internal exon, intron, and CDS structure of each retained interval.

Context-length and chunking parameters

• edge_context_length — Token length of each edge-model input window, including tokenizer system tokens. Larger values give the edge model more context, but also increase memory use.
• region_context_length — Token length of each region-model input window, including tokenizer system tokens. The default is 8192 tokens, matching the gene-finding benchmark and manuscript configuration.
• transcript_type_context_length — Maximum token length passed to the transcript-type model for each candidate interval. Only the leading prefix up to this context is evaluated, so sequence beyond this limit is ignored by the classifier.
• segmentation_context_length — Nucleotide length of each segmentation-model block inside a retained interval. Segmentation blocks are processed consecutively and without overlap.
• edge_average_token_length — Estimated average number of nucleotides represented by one edge-model tokenizer token. The pipeline uses this value to convert token context length into genomic window length before tokenization.
• region_average_token_length — Estimated average number of nucleotides represented by one region-model tokenizer token. The pipeline uses this value to convert token context length into genomic window length before tokenization.
• edge_max_genomic_chunk_ratio — Maximum expansion ratio for edge-model genomic extraction before tokenizer truncation. It gives the tokenizer extra nucleotide sequence so the final tokenized window can be filled reliably.
• region_max_genomic_chunk_ratio — Maximum expansion ratio for region-model genomic extraction before tokenizer truncation. It plays the same role as edge_max_genomic_chunk_ratio, but for the region model.
• edge_context_fraction — Fractional overlap between consecutive edge-model genomic windows. Higher overlap can smooth boundary predictions, but it increases the number of model calls.
• region_context_fraction — Fractional overlap between consecutive region-model genomic windows. Higher overlap can make intragenic masks more stable, but it increases computation time.
• edge_drop_last — If True, the final incomplete edge-model window is omitted. The default is False, which keeps the final window so the end of the sequence is still processed.
• region_drop_last — If True, the final incomplete region-model window is omitted. The default is False, which keeps the final window so the end of the sequence is still processed.
• edge_gap_token_id — Token ID used to correct edge-model offset mappings for gap tokens. Most users should keep the default unless they change the tokenizer.
• region_gap_token_id — Token ID used to correct region-model offset mappings for gap tokens. Most users should keep the default unless they change the tokenizer.
• gene_finding_global_chunk_size — Maximum nucleotide length of each global chunk used by the edge model only. For each global chunk, the pipeline computes edge predictions, runs FFT smoothing and peak calling inside that chunk, keeps only sparse peak coordinates, discards the raw edge predictions, and then moves to the next chunk if the DNA sequence is longer.

Interval-discovery parameters

• lp_frac — Fraction of the Fourier spectrum retained by the low-pass smoother before peak detection. Smaller values produce smoother boundary tracks and can remove local noise.
• pk_prom — Minimum peak prominence used during TSS and PolyA boundary detection. Higher values make peak calling more conservative.
• pk_dist — Minimum nucleotide distance between neighboring peaks of the same boundary class. This helps avoid calling several nearby peaks for one broad signal.
• pk_height — Optional minimum peak height after smoothing. Use None to disable this extra height filter.
• interval_window_size — Maximum distance allowed when pairing a TSS peak with a PolyA peak on the same strand. Candidate transcript intervals longer than this pairing window are not created.
• max_pairs_per_seed — Maximum number of nearest PolyA partners retained for each TSS seed. Larger values create more candidate intervals and can increase downstream computation.
• prob_threshold — Threshold used to convert region-model intragenic signal into a binary mask. A base is considered intragenic only when the model signal is above this threshold.
• zero_fraction_drop_threshold — Maximum tolerated fraction of non-intragenic bases inside a candidate interval. Intervals with a larger fraction below prob_threshold are discarded.

Reverse-complement options

• gene_finding_use_reverse_complement — Enables reverse-complement averaging for the edge and region models. This can improve strand-aware interval discovery, but it roughly doubles gene-finding model calls.
• transcript_type_use_reverse_complement — Enables reverse-complement averaging for transcript-type classification. The forward and reverse-complement scores are averaged before the final mRNA or lnc_RNA decision.
• segmentation_use_reverse_complement — Enables reverse-complement averaging for segmentation. This can stabilize structure prediction, but it increases segmentation compute cost.

Activation options

• edge_apply_sigmoid — Applies an additional sigmoid to edge-model output channels before token-to-nucleotide projection. The default is False, because the published edge model outputs are already expected in the correct scale.
• region_apply_sigmoid — Applies an additional sigmoid to region-model output channels before token-to-nucleotide projection. The default is False, because the published region model outputs are already expected in the correct scale.
• transcript_type_apply_sigmoid — Applies sigmoid to single-logit transcript-type outputs before thresholding. For multi-logit outputs, the pipeline uses softmax instead.
• segmentation_apply_sigmoid — Applies an additional sigmoid to segmentation-model output channels before structural decoding. The default is True for the published segmentation setup.

Transcript and segmentation parameters

• transcript_type_threshold — Threshold applied to the predicted lnc_RNA probability. Intervals at or above this value are labeled lnc_RNA, and intervals below it are labeled mRNA.
• splice_filter — Enables splice-motif filtering and terminal splice-boundary correction for exon and CDS segments. This post-processing step can remove or adjust segments that disagree with expected splice signals.
• deduplicate — Removes duplicate final transcript predictions. This is applied near the end of GFF generation to avoid repeated identical transcripts.
• intronic_filtering — Drops transcript predictions whose segmentation starts or ends with the intron class. This removes predictions that appear to begin or end inside an intron.
• keep_longest_terminal_variant — For overlapping transcripts with the same internal structure, keeps the longest terminal variant. This reduces redundant terminal variants that differ mainly by transcript ends.
• predict_internal_structure — Controls whether the pipeline continues past interval discovery into transcript-type classification, segmentation, and GFF generation. Keep it as True for normal annotation output.
• use_cds_heuristic — Replaces predicted CDS segments with the exon-derived CDS heuristic used in the accompanying benchmark code. This affects mRNA transcripts only, and no CDS is emitted for lnc_RNA transcripts.
• transcript_coloring_thresholds — Controls transcript color bins in the output GFF. Use "auto" to split the observed segmentation-confidence range into four bins, or provide a custom list of exactly four thresholds.

• Lowest bin, #66cc66, light green.
• Second bin, #006400, dark green.
• Third bin, #dcdcff, light blue.
• Top bin, #0c0c78, dark blue.

Intermediate-output, logging, and coordinate parameters

• save_intermediate_files — If True, writes gene-finding intermediate artifacts for each FASTA record. In the memory-efficient path, these are compact edge peak .npz files, compact intragenic-mask .npz files, .bed interval files, and a compressed .h5 debug dump when h5py is installed.
• intermediate_output_dir — Output directory for intermediate artifacts. If omitted, intermediate files are written next to the input FASTA file.
• pairing_progress_every — Logging interval, measured in TSS seeds, during candidate interval construction. Increase it for less frequent logs.
• chunk_log_every — Logging interval, measured in genomic chunks, during edge and region inference. Increase it for less frequent logs on large genomes.
• shift — Coordinate offset applied to final GFF coordinates. Use an integer offset directly, or use "UCSC" to infer the offset from FASTA headers of the form chrom:start-end.
• output_gff_path — Path of the GFF file written by the pipeline call. If you do not provide it, the pipeline writes a default GFF path derived from the input FASTA path.

What the pipeline does

1. Interval discovery

• The edge model detects transcription start site and polyadenylation signals, abbreviated as TSS and PolyA.
• The region model predicts intragenic signal, which is used to filter candidate intervals.

2. Transcript-type assignment

3. Segmentation

4. GFF generation

• gene
• mRNA or lnc_RNA
• exon
• CDS, for mRNA transcripts only

Default model repositories

• edge_model_path, AIRI-Institute/genatator-moderngena-base-multispecies-edge-model
• region_model_path, AIRI-Institute/genatator-moderngena-base-multispecies-region-model
• transcript_type_model_path, AIRI-Institute/genatator-caduceus-ps-multispecies-transcript-type
• segmentation_model_path, AIRI-Institute/genatator-caduceus-ps-multispecies-segmentation

Input and output

• Path to a FASTA file.
• The FASTA file may contain one record or multiple records.

• A single Python string, the path to the written .gff file.
• The file contents follow the GFF3 specification.

Dependencies

Output annotation

Docker deployment

• POST /api/genatator-pipeline/upload
• Input, multipart file containing FASTA, or form field dna.
• Output JSON fields, fasta_file, fai_file, gff_file, and archive.