SciLinkBERT: A Language Model for Understanding Scientific Texts with Citation Information
Introduction
SciLinkBERT is a BERT-based pre-trained language model specifically designed to enhance the understanding of scientific texts by incorporating citation information. This model is particularly useful in scientific domains, where understanding complex language and extracting meaningful information from citations is crucial.
[PDF]
References and Additional Resources
For more advanced fine-tuning and implementation details, you can refer to the following repositories:
LinkBERT: Provides an example of how citation links and other scientific data can be incorporated into BERT models.
SciDeBERTa-Fine-Tuning: This repository demonstrates fine-tuning approaches on scientific datasets which can be adapted for SciLinkBERT.
How to Cite
If you use SciLinkBERT in your research, please cite the following paper:
@article{Yu2025SciLinkBERT,
year = {2025},
title = {{Scilinkbert: a BERT-style language model for understanding scientific texts with citations}},
author = {Yu, Ju-Yeon and Yang, Donghun and Lee, Kyong-Ha},
journal = {The Journal of Supercomputing},
doi = {10.1007/s11227-025-07838-y},
pages = {1356},
number = {15},
volume = {81},
url={https://github.com/KISTI-KONI/SciLinkBERT},
}
Contributing
Contributions to SciLinkBERT are welcome! If you find any issues or have suggestions for improvements, feel free to open an issue or submit a pull request.
