This model was trained as part of the "Extractive QA Biomedicine" project developed during the 2022 Hackathon organized by SOMOS NLP.

Recent research has made available Spanish Language Models trained on Biomedical corpus. This project explores the use of these new models to generate extractive Question Answering models for Biomedicine, and compares their effectiveness with general masked language models.

The models trained during the Hackathon were:

This model is a fine-tuned version of PlanTL-GOB-ES/roberta-base-biomedical-clinical-es on the squad_es (v2) training dataset.

The hyperparameters were chosen based on those used in PlanTL-GOB-ES/roberta-base-bne-sqac, a spanish-based QA model trained on a dataset with SQUAD v1 fromat.

 --num_train_epochs 2
 --learning_rate 3e-5
 --weight_decay 0.01
 --max_seq_length 386
 --doc_stride 128 

Evaluated on the hackathon-pln-es/biomed_squad_es_v2 dev set.

Santiago Maximo: smaximo