The HealthHistoryRoBERTa-pt-ft was fine-tuned on the pre-trained model efbaro/HealthHistoryRoBERTa-pt and with patient data from health insurances organized in the form of historical sentences. The initial objective of the training was to predict hospitalizations, however, due to the possibility of applications in other tasks, we made these models available to the scientific community. This model was trained with Portuguese Health Insurance Data. There are also other training approaches that can be seen at:

The model was fine-tuned from 83,715 historical sentences from health insurance patients generated using the approach described in this paper Predicting Hospitalization from Health Insurance Data.

The model was fine-tuned on a GeForce NVIDIA RTX A5000 24GB GPU from laboratories of IT departament at UFPR (Federal University of Paraná).

We use a batch size of 16, a maximum sequence length of 512 tokens, accumulation steps of 4, number of epochs = 2 and a learning rate of 10−4 to fine-tune this model.

The training time was 3 hours 04 minutes per epoch.

Time to predict the first 500 sentences of dataset data_test_seed_pt_12.csv: 1.31 seconds

Time to predict the first 500 sentences + data tokenization of data_test_seed_pt_12.csv: 3.52 seconds

Predictions made with the maximum sentence length allowed by the models.

Load the model via the transformers library:

Refer to the original paper, Predicting Hospitalization with LLMs from Health Insurance Data

Refert to another article related to this research, Predicting Hospitalization from Health Insurance Data

Email: