Computational prediction of blood-brain barrier (BBB) permeability has emerged as a vital alternative to traditional experimental assays, which are often resource-intensive and low-throughput to meet the demands of early-stage drug discovery. While early machine learning approaches have shown promise, integration of traditional chemical descriptors with deep learning embeddings remains an underexplored frontier. In this paper, we introduce TITAN-BBB, a multi-modal deep-learning architecture that utilizes tabular, image, and text-based features and combines them using attention mechanisms. To evaluate, we aggregated multiple literature sources to create the largest BBB permeability dataset to date, enabling robust training for both classification and regression tasks. Our results demonstrate that TITAN-BBB achieves 86.5% of balanced accuracy on classification tasks and 0.436 of mean absolute error for regression, outperforming the state-of-the-art by 3.1 percentage points in balanced accuracy and reducing the regression error by 20%. Our approach also outperforms state-of-the-art models in both classification and regression performance, demonstrating the benefits of combining deep and domain-specific representations.
Model Details
TITAN-BBB is a multi-modal method designed for molecular property (BBB) prediction. This architecture effectively combines three sources of information: embeddings from a pre-trained language model (ChemBERTa-100M-MLM), images representation extracted from convolutional neural networks (ResNet50), and classical molecular descriptors (RDKit).
TITAN-BBB consists of three stages: multi-modal feature projection, attention-based fusion, and prediction.
Model
Model Usage
Note: The model is only available using AutoModelForSequenceClassification.
Note: This model uses a custom architecture (Transformer + CNN + RDKit) defined in the source repository. Therefore, you must set trust_remote_code=True when loading both the model and the tokenizer.
Classification
Use the code below to score (between 0 and 1) if a molecule can cross the BBB.
Both classification and regression models return for each input:
logits: the raw output scores. For classification, please apply sigmoid to get the score between 0 and 1. For regression, use it as prediction.
hidden_states: the attention-based aggregation of tabular, image, and text representations.
attentions: the attention weights considering tabular, image, and text features for each input.
Requirements
huggingface_hub
rdkit
torch
torchvision
Citation
The paper is under review. As soon as it is accepted, we will update this section.
License
This model and associated code are released under the CC-BY-NC-ND 4.0 license and may only be used for non-commercial, academic research purposes with proper attribution. Any commercial use, sale, or other monetization of this model and its derivatives, which include models trained on outputs from the model or datasets created from the model, is prohibited and requires prior approval. Downloading the model requires prior registration on Hugging Face and agreeing to the terms of use. By downloading this model, you agree not to distribute, publish or reproduce a copy of the model. If another user within your organization wishes to use the model, they must register as an individual user and agree to comply with the terms of use. Users may not attempt to re-identify the deidentified data used to develop the underlying model. If you are a commercial entity, please contact the corresponding author.
Contact
For any additional questions or comments, contact Fahad Saeed (fsaeed@fiu.edu).
