scCAFM is a foundation model for large-scale single-cell RNA-seq analysis that jointly learns context-specific gene-regulatory structure and transferable gene and cell representations. Its two-stage design combines a Structure Foundation Module (SFM) for causal regulatory modeling with an Embedding Foundation Module (EFM) guided by the structure learned by SFM.
Model repository:kaichenxu/scCAFM Source code:Catchxu/scCAFM
Overview
Many single-cell foundation models primarily capture associative gene relationships or summarize regulation at the dataset or cell-type level. scCAFM is designed to model regulatory context at cellular resolution while learning representations that can transfer to downstream biological analyses.
Overview of the scCAFM framework
Highlights
Context-specific regulatory modeling: represents gene-regulatory structure in a compact latent factor space.
Cell- and gene-level outputs: supports downstream analyses that require transferable embeddings and regulatory structure.
Cross-species resources: includes shared assets for Homo sapiens and Mus musculus workflows.
Model architecture
Structure Foundation Module (SFM)
SFM learns context-aware structural representations of gene regulation. A Mixture-of-Experts architecture captures distinct regulatory contexts without fitting an independent causal model for every cell. The resulting structural factors and causal gene ordering guide EFM pretraining.
Embedding Foundation Module (EFM)
EFM learns gene and cell embeddings using the causal ordering produced by a frozen SFM. These representations are intended for downstream supervised and unsupervised analyses, including perturbation-response modeling, trajectory and lineage analysis, and phenotype-associated prediction tasks.
GPU-specific packages, including FlashAttention, must be selected for the local CUDA, PyTorch, compiler, and GPU environment. See the source repository for FA2/FA4 configuration and validation instructions.
Download the model assets
Install or update the Hugging Face CLI, then download this repository into the package's assets directory:
