A foundation model over paired proteomic and transcriptomic expression. Attention between measured features yields a protein-protein association network without task-specific training.

All but one are mapped onto a shared UniProt accession vocabulary of 20,272 accessions. sc_transcriptomics_ensembl is the exception: it is trained directly on 61,497 ENSEMBL gene IDs, with no gene-to-protein mapping applied. It exists so the cost of that mapping can be measured. Mapping genes to proteins drops features that fail to map, and drops ground-truth edges between them; comparing this checkpoint against sc_transcriptomics isolates that effect, because the two are identical in architecture and training and differ only in vocabulary.

Feature ID embedding. The learned variants train an embedding table over the vocabulary from scratch. The ESM-C variants instead look each feature up in a frozen table of precomputed ESM-C sequence embeddings and pass it through a small trained projection, so identity is grounded in protein sequence rather than learned from co-expression alone.

Parameter counts are the trained model excluding the feature ID embedding, which scales with vocabulary rather than model capacity and would otherwise dominate. The transformer is identical across all seven at 4.94 M parameters; the ESM-C variants add 0.81 M for the projection. Excluded on top of that: the 5.19 M learned embedding table (learned variants) and the 23.35 M frozen ESM-C lookup (ESM-C variants), which is not trained at all. sc_transcriptomics_ensembl carries a larger 15.74 M table, since its vocabulary is three times the size -- which is why the exclusion matters: it is the same model, not a bigger one.

Each checkpoint folder holds best_model.ckpt and a config.yaml recording the architecture, hyperparameters, training epoch and validation loss, readable without loading the checkpoint.

esmc_human_cache.pt (95 MB) sits at the repository root and is required by the three _esmc checkpoints, which use it as a frozen lookup table instead of learning their feature embeddings. It holds one 1152-dimensional vector per UniProt accession: ESM-C 600M run over the canonical human proteome, mean-pooled per sequence. It can be recomputed, but that means downloading a 600M-parameter model and running inference over 20,000 sequences. Shipping it keeps the embeddings bit-identical to what the models saw during training, and lets anyone run inference with the _esmc checkpoints straight away. The learned checkpoints do not need it.

The ESM-C variants additionally need the canonical human proteome FASTA and the precomputed embedding cache; see the repository README.

Publication in preparation.