You can deploy this model and start earning money today!

sagawa/
CompoundT5

Name: CompoundT5
Rating: 2.5 (362 reviews)

sagawa/ZINC-canonicalized

mit

model-index

text-generation-inference

endpoints_compatible

Views

No views yet

CompoundT5

This model is a re-pretrained version of google/t5-v1_1-base on the sagawa/ZINC-canonicalized dataset. It achieves the following results on the evaluation set:

Loss: 0.1202
Accuracy: 0.9497

Model description

We trained t5 on SMILES from ZINC using masked-language modeling (MLM). Its tokenizer is also trained on ZINC.

Intended uses & limitations

This model can be used to predict molecules' properties, reactions, or interactions with proteins by changing the way of finetuning. As an example, We finetuned this model to predict products. The model is here, and you can use the demo here. Using its encoder, we trained a regression model to predict a reaction yield. You can use this demo here.

Training and evaluation data

We downloaded ZINC data and canonicalized them using RDKit. Then, we dropped duplicates. The total number of data is 22992522, and they were randomly split into train:validation=10:1.

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 5e-03
train_batch_size: 30
eval_batch_size: 32
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
num_epochs: 30.0

Training results

Training Loss	Step	Accuracy	Validation Loss
0.2471	25000	0.9843	0.2226
0.1871	50000	0.9314	0.1783
0.1791	75000	0.9371	0.1619
0.1596	100000	0.9401	0.1520
0.1522	125000	0.9422	0.1449
0.1435	150000	0.9436	0.1404
0.1421	175000	0.9447	0.1368
0.1398	200000	0.9459	0.1322
0.1297	225000	0.9466	0.1299
0.1324	250000	0.9473	0.1268
0.1257	275000	0.9483	0.1244
0.1266	300000	0.9491	0.1216
0.1301	325000	0.9497	0.1204

Citation

@article{Sagawa2025,
  title   = {ReactionT5: a pre-trained transformer model for accurate chemical reaction prediction with limited data},
  author  = {Sagawa, Tatsuya and Kojima, Ryosuke},
  journal = {Journal of Cheminformatics},
  year    = {2025},
  volume  = {17},
  number  = {1},
  pages   = {126},
  doi     = {10.1186/s13321-025-01075-4},
  url     = {https://doi.org/10.1186/s13321-025-01075-4}
}

sagawa/CompoundT5