@article{Beltagy2020Longformer,
title={Longformer: The Long-Document Transformer},
author={Iz Beltagy and Matthew E. Peters and Arman Cohan},
journal={arXiv:2004.05150},
year={2020},
}
This model is fine-tuned on Longformer 4096 with political and world news from 2016 to 2018,
so it will probably identify all current news as fake since the world has changed a lot since then
But it performs well on more related news article:
Test accuracy: 0.9956
val_loss: 0.0003