Views
No views yet
If you want to import KoBERT tokenizer withAutoTokenizer, you should givetrust_remote_code=True.
1from transformers import AutoModel, AutoTokenizer
2
3model = AutoModel.from_pretrained("monologg/kobert-lm")
4tokenizer = AutoTokenizer.from_pretrained("monologg/kobert-lm", trust_remote_code=True)