DistilBERT model fine-tuned on 117567 English language tweets from a range of political agencies (EU_Commission, UN, OECD, IMF, ECB, Council of the European Union, UK government, Scottish government).
Fine-tuned with learning rate = 2e-5, 16-sample batches, chunk_size = 50 tokens, 100 epochs (early stopping patience = 3 epochs), 3 warmup epochs.
More details can be found at:
https://github.com/rbroc/eucomm-twitter.
No evaluation was performed, as fine-tuning was merely functional to providing checkpoints for a contextualized topic model.
