gbert-large-text_parts is a sequence classification model in the scientific domain in German, finetuned from the model
gbert-large.
It was trained using a custom annotated dataset of around 4,800 training and 1,400 test examples containing introduction, main part and conclusion text sequences from scientific papers and theses in german.
The two German BERT models — gbert-base and gbert-large — were evaluated on the same text classification task. This repository contains the model and results for
gbert-large.
Despite its superior peak performance, gbert-large exhibited a consistent degradation in evaluation accuracy and an increase in loss over subsequent epochs.
This pattern is characteristic of overfitting, where the model fits the training data too closely and fails to generalize well to unseen data.
In contrast, gbert-base maintained more stable performance over time, making it a more robust choice in scenarios where computational resources or dataset size are limited.
Training was conducted using a standard Text classification objective. The model achieves an accuracy of approximately 78,5% on the evaluation set.
Here are the overall final metrics on the test dataset after 1 epoch of training:
