RobBERTje is a collection of distilled models based on
RobBERT. There are multiple models with different sizes and different training settings, which you can choose for your use-case.
We are also continuously working on releasing better-performing models, so watch
the repository for updates.
We calculated the
pseudo perplexity (PPPL) from
cite, which is a built-in metric in our distillation library. This metric gives an indication of how well the model captures the input distribution.
We also evaluated our models on sereral downstream tasks, just like the teacher model RobBERT. Since that evaluation, a
Dutch NLI task named SICK-NL was also released and we evaluated our models with it as well.
