This model is a fine-tuned version of
chunwoolee0/mt5_small_bongsoo_en_ko
on the
bongsoo/news_talk_en_ko dataset.
It achieves the following results on the evaluation set:
mT5, a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset
covering 101 languages
You can use this model directly with a pipeline for translation language modeling:
1>>> from transformers import pipeline
2>>> translator = pipeline('translation', model='chunwoolee0/ke_t5_base_bongsoo_en_ko')
3
4>>> translator("Let us go for a walk after lunch.")
5[{'translation_text': '식당에 앉아서 밤에 갔다.'}]
6
7>>> translator("Skinner's reward is mostly eye-watering.")
8[{'translation_text': '벤더의 선물은 너무 마음이 쏠린다.'}]
The value of max_length is critical to the training. The usual value of 128 used for Indo-European languages causes a
greate trouble in gpu usage. Therefore it should be reduced to 64 in order to succeed.
Another problem comes from the usual split of data into 80% for train and 20% for validation.
By this, the evaluation
step takes too much time. Here 99% and 1% split is used without change in the evaluation.
The mT5 model of google cannot be used for Korean although it is trained over 101 languages. Finetuning
using very large data set such as bongsoo/news_talk_en_ko still yield garbage.
Since GPU memories allowed for free use in colab are greatly limited, repeated fine-tunings for the split datasets are performed
to obtain better results. Theoretically, this might give better results. But actual attempts fail to yield
better results. Instead, the results become worse. One should use other
models like the ke-t5 by KETI(한국전자연구원).
