This model is a continued pre-trained version of
google-bert/bert-base-uncasedThis BERT-based model has been continuously trained using the Masked Language Modeling (MLM) objective on 2 million samples from the English column of the
vi_pubmedThis model is developed as part of an undergraduate thesis to compare the performance of general language models with domain-specific biomedical language models for Clinical NER applications. It is an experimental model, and its performance may not be optimal. Therefore, I recommend using
biobert-v1.1 or
PubMedBERT for better results.
The model is trained on 2 million English text samples from
vi_pubmed, using a 90:10 split ratio for training and evaluation.
The training procedure follows the
tutorial provided by Hugging Face, with the batch size adjusted to 64. Training was conducted on an NVIDIA RTX 4070 GPU.
