from transformers import EsmModel, AutoTokenizer
import torch

# Load model and tokenizer
model = EsmModel.from_pretrained("NOC-Lab/AbCDR-ESM2")
tokenizer = AutoTokenizer.from_pretrained("NOC-Lab/AbCDR-ESM2")
model.eval()

Extract Embeddings

python

# Prepare paired sequence
SEP_TOKEN = "-" 
heavy_chain = (
    "EVQLVESGGGLVQPGGSLRLSCAASGFTFSSYAMSWVRQAPGKGLEWVAVISYDGSNKYYADSVKGRF"
    "TISADTSKNTAYLQMNSLRAEDTAVYYCAREGYYGSSYWYFDYWGQGTLVTVSS"
)
light_chain = (
    "DIQMTQSPSSLSASVGDRVTITCRASQSISSYLNWYQQKPGKAPKLLIYAASSLQSGVPSRFSGSGS"
    "GTDFTLTISSLQPEDFATYYCQQSYSTPLTFGGGTKVEIK"
)
paired_sequence = f"{heavy_chain}{SEP_TOKEN}{light_chain}"

# Tokenize
inputs = tokenizer(paired_sequence, return_tensors="pt", add_special_tokens=True)

# Extract embeddings
with torch.no_grad():
    outputs = model(**inputs)
    embeddings = outputs.last_hidden_state
    
# Mean pooling
mask = inputs["attention_mask"].unsqueeze(-1)
pooled = (embeddings * mask).sum(1) / mask.sum(1)

print(f"Embedding shape: {pooled.shape}")  # (1, 2560)

Input Format

Required Format: HEAVY_CHAIN-LIGHT_CHAIN

Heavy and light chains must be separated by hyphen (-)
Use standard single-letter amino acid codes
No spaces in sequence
Uncommon residues should be replaced with X

Example:

sequence = "EVQLVESGGGLVQPGGSLRLSCAASGFTFSSYAMS...-DIQMTQSPSSLSASVGDRVTITCRASQSISS..."

Output

Embedding dimension: 2560
Sequence length: Variable (up to ~1024 tokens including special tokens)
Format: PyTorch tensor

Publication

Preferential CDR masking in paired antibody language models improves binding affinity prediction

Mahtab Talaei, Kenji C. Walker, Boran Hao, Eliot Jolley, Yeping Jin, Dima Kozakov, John Misasi, Sandor Vajda, Ioannis Ch. Paschalidis, Diane Joseph-McCarthy.

Communications AI & Computing 1, 7 (2026).

DOI: https://doi.org/10.1038/s44488-026-00010-2

Published: August 13, 2026

Read the published article

Citation

If you use this model, please cite the published paper:

bibtex

@article{Talaei2026,
  author = {Talaei, Mahtab and Walker, Kenji C. and Hao, Boran and Jolley, Eliot and Jin, Yeping and Kozakov, Dima and Misasi, John and Vajda, Sandor and Paschalidis, Ioannis Ch. and Joseph-McCarthy, Diane},
  title = {Preferential {CDR} masking in paired antibody language models improves binding affinity prediction},
  journal = {Communications AI \& Computing},
  volume = {1},
  pages = {7},
  year = {2026},
  doi = {10.1038/s44488-026-00010-2}
}

Contact

Maintainer: Network Optimization & Control (NOC) Lab
Email: mtalaei@bu.edu
GitHub: https://github.com/noc-lab/AbCDR-ESM
Paper: Communications AI & Computing

License

This model is released under the MIT License.

Acknowledgments

Base model: ESM2 by Meta AI
Data: OAS database

Note: For private repositories, you'll need to authenticate:

bash

# Option 1: CLI login
huggingface-cli login

# Option 2: Environment variable
export HF_TOKEN="your_token_here"