SapBERT-biomedical-clinical model for Spanish
Table of contents
Click to expand
Model description
SapBERT model in Spanish trained with a procedure similar to that described by
Liu et al. (2020). The model has been trained with the Spanish data from
UMLS 2023AA, using
PlanTL-GOB-ES/roberta-base-biomedical-clinical-es as the base model.
Intended uses and limitations
The model is prepared to provide a numerical representation of biomedical concepts in UMLS. This allows using the embeddings generated by the model for semantic similarity tasks of biomedical concepts or entity linking tasks, among others.
How to use
The following script taken and adapted from the
original SapBERT model converts a list of strings (entity names) into embeddings.
1import numpy as np
2import torch
3from tqdm.auto import tqdm
4from transformers import AutoTokenizer, AutoModel
5
6tokenizer = AutoTokenizer.from_pretrained("BSC-NLP4BIA/SapBERT-parents-from-roberta-base-biomedical-clinical-es")
7model = AutoModel.from_pretrained("BSC-NLP4BIA/SapBERT-parents-from-roberta-base-biomedical-clinical-es").cuda()
8
9# replace with your own list of entity names in spanish
10all_names = ["cancer de pulmón", "fiebre", "cirugía torácica"]
11
12bs = 128 # batch size during inference
13all_embs = []
14for i in tqdm(np.arange(0, len(all_names), bs)):
15 toks = tokenizer.batch_encode_plus(all_names[i:i+bs],
16 padding="max_length",
17 max_length=25,
18 truncation=True,
19 return_tensors="pt")
20 toks_cuda = {}
21 for k,v in toks.items():
22 toks_cuda[k] = v.cuda()
23 cls_rep = model(**toks_cuda)[0][:,0,:] # use CLS representation as the embedding
24 all_embs.append(cls_rep.cpu().detach().numpy())
25
26all_embs = np.concatenate(all_embs, axis=0)
For more details about training and eval, see SapBERT
github repo.
Training
The training was performed using the
original SapBERT training repository. As training data, the Spanish entries in UMLS were used, as well as the commercial names of the drugs (although they are in English), transformed to lowercase. To train the model, a set of 15 pairs of synonymous terms has been generated for each UMLS concept, we have considered as synonyms the lexical entries of each concept.
Evaluation
Evaluation of the results of using this model are in:
Gallego, F., López-García, G., Gasco-Sánchez, L., Krallinger, M., & Veredas, F. J. (2024, June). Clinlinker: Medical entity linking of clinical concept mentions in spanish. In International Conference on Computational Science (pp. 266-280). Cham: Springer Nature Switzerland.
Additional information
Author
NLP4BIA at the Barcelona Supercomputing Center
Licensing information
Citation information
1@inproceedings{gallego2024clinlinker,
2 title={Clinlinker: Medical entity linking of clinical concept mentions in spanish},
3 author={Gallego, Fernando and L{\'o}pez-Garc{\'\i}a, Guillermo and Gasco-S{\'a}nchez, Luis and Krallinger, Martin and Veredas, Francisco J},
4 booktitle={International Conference on Computational Science},
5 pages={266--280},
6 year={2024},
7 organization={Springer}
8}
Disclaimer
Click to expand
The models published in this repository are intended for a generalist purpose and are available to third parties. These models may have bias and/or any other undesirable distortions.
When third parties, deploy or provide systems and/or services to other parties using any of these models (or using systems based on these models) or become users of the models, they should note that it is their responsibility to mitigate the risks arising from their use and, in any event, to comply with applicable regulations, including regulations regarding the use of Artificial Intelligence.
