You can deploy this model and start earning money today!

kimsiun/
kaers-bert-241101

Name: kaers-bert-241101
Availability: InStock

Views

No views yet

KAERS-BERT

Model Description

KAERS-BERT is a domain-specific Korean BERT model specialized for clinical text analysis, particularly for processing adverse drug event (ADE) narratives. It was developed by pretraining KoBERT (developed by SK Telecom) using 1.2 million ADE narratives reported through the Korea Adverse Event Reporting System (KAERS) between January 2015 and December 2019.

The model is specifically designed to handle clinical texts where code-switching between Korean and English is frequent, making it particularly effective for processing medical terms and abbreviations in a bilingual context.

Usage

You can load the model from HuggingFace Hub while using the local tokenizer:

python

from transformers import BertForPreTraining
from tokenization_kobert import KoBERTTokenizer

Load model from HuggingFace

model = BertForPreTraining.from_pretrained("kimsiun/kaers-bert-241101")

Load tokenizer from local file

tokenizer = KoBERTTokenizer.from_pretrained('skt/kobert-base-v1')

Key Features

Specialized in clinical and pharmaceutical domain text
Handles Korean-English code-switching common in medical texts
Optimized for processing adverse drug event narratives
Built upon KoBERT architecture with domain-specific pretraining

Training Data

The model was pretrained on:

1.2 million ADE narratives from KAERS
Training data specifically focused on 'disease history in detail' and 'adverse event in detail' sections
Masked language modeling with 15% token masking rate
Maximum sequence length of 200
Learning rate: 5×10^-5

Performance

The model demonstrated strong performance in various NLP tasks related to drug safety information extraction:

Named Entity Recognition (NER): 83.81% F1-score
Sentence Extraction: 76.62% F1-score
Relation Extraction: 64.37% F1-score (weighted)
Label Classification:
- 'Occurred' Label: 81.33% F1-score
- 'Concerned' Label: 77.62% F1-score

When applied to the KAERS database, the model achieved an average increase of 3.24% in data completeness for structured data fields.

Intended Use

This model is designed for:

Extracting drug safety information from clinical narratives
Processing Korean medical texts with English medical terminology
Supporting pharmacovigilance activities
Improving data quality in adverse event reporting systems

Limitations

The model is specifically trained on adverse event narratives and may not generalize well to other clinical domains
Performance may vary for texts significantly different from KAERS narratives
The model works best with Korean clinical texts containing English medical terminology

Citation

bibtex

@article{kim2023automatic,
  title={Automatic Extraction of Comprehensive Drug Safety Information from Adverse Drug Event Narratives in the Korea Adverse Event Reporting System Using Natural Language Processing Techniques},
  author={Kim, Siun and Kang, Taegwan and Chung, Tae Kyu and Choi, Yoona and Hong, YeSol and Jung, Kyomin and Lee, Howard},
  journal={Drug Safety},
  volume={46},
  pages={781--795},
  year={2023}