CLSS (Contrastive Learning Sequence–Structure)
CLSS is a self-supervised, two-tower contrastive model that co-embeds protein sequences and protein structures into a shared latent space, enabling unified analysis of protein space across modalities.
Links
Model description
Architecture (high level)
CLSS follows a two-tower architecture:
- Sequence tower: a trainable ESM2-like sequence encoder
- Structure tower: a frozen ESM3 structure encoder
- Each tower is followed by a lightweight linear projection head mapping into a shared embedding space, with L2-normalized outputs
The result is a pair of embeddings (sequence and structure) that live in the same latent space, making cosine similarity directly comparable across modalities.
The paper’s primary configuration uses 32-dimensional embeddings, but multiple embedding sizes are provided in this repository.
Training objective
CLSS is trained with a CLIP-style contrastive objective, aligning:
- Random sequence segments
- With their corresponding full-domain protein structures
No hierarchical labels (e.g. ECOD or CATH) are used during training; structural and evolutionary organization emerges implicitly.
Files in this repository
This Hugging Face repository contains multiple PyTorch Lightning checkpoints, differing only in embedding dimensionality:
CLSS-full.lckpt → Trained exclusively on full sequencesCLSS-sub.lckpt → Trained with subsequences
How to use CLSS
CLSS is intended to be used via the clss-model Python library, which provides:
- Model loading from Lightning checkpoints
- End-to-end inference examples
- Scripts used for generating interactive protein space maps
License
The CLSS codebase is released under the Apache 2.0 License.
Please consult the repository for details on third-party model dependencies.
Citation
If you use CLSS, please cite:
1@article{Yanai2026CLSS,
2 title={Contrastive learning unites sequence and structure in a global representation of protein space},
3 author={Yanai, Guy and Axel, Gabriel and Longo, Liam M. and Ben-Tal, Nir and Kolodny, Rachel},
4 journal={Proceedings of the National Academy of Sciences},
5 volume={123},
6 number={32},
7 pages={e2532702123},
8 year={2026},
9 doi={10.1073/pnas.2532702123},
10 url={https://www.pnas.org/doi/10.1073/pnas.2532702123}
11}
