These BERT models are pre-trained on written Japanese (Wikipedia) and fine-tuned on Spoken Japanese.
We used CSJ and the Japanese diet record.
CSJ (Corpus of Spontaneous Japanese) is provided by NINJAL (
https://www.ninjal.ac.jp/).
We only provide model parameters. You have to download other config files to use these models.
-
1-6 layer-wise (Folder Name: models/1-6_layer-wise)
Fine-Tuned only 1st-6th layers in Encoder on CSJ.
-
TAPT512 60k (Folder Name: models/tapt512_60k)
Fine-Tuned on CSJ.
-
DAPT128-TAPT512 (Folder Name: models/dapt128-tap512)
Fine-Tuned on the diet record and CSJ.
These BERT models are pre-trained on written Japanese (Wikipedia) and fine-tuned on Spoken Japanese.
We used CSJ and the Japanese diet record.
CSJ (Corpus of Spontaneous Japanese) is provided by NINJAL (
https://www.ninjal.ac.jp/).
We only provide model parameters. You have to download other config files to use these models.
-
1-6 layer-wise (Folder Name: models/1-6_layer-wise)
Fine-Tuned only 1st-6th layers in Encoder on CSJ.
-
TAPT512 60k (Folder Name: models/tapt512_60k)
Fine-Tuned on CSJ.
-
DAPT128-TAPT512 (Folder Name: models/dapt128-tap512)
Fine-Tuned on the diet record and CSJ.
We continuously train the pre-trained Japanese BERT model (
cl-tohoku/bert-base-japanese-whole-word-masking; written BERT).
We use CSJ for the evaluation.
1@inproceedings{csjbert2021,
2 title = {CSJを用いた日本語話し言葉BERTの作成},
3 author = {勝又智 and 坂田大直},
4 booktitle = {言語処理学会第27回年次大会},
5 year = {2021},
6}
Use the code below to get started with the model.
