Views
No views yet
If you want to import DistilKoBERT tokenizer withAutoTokenizer, you should givetrust_remote_code=True.
1from transformers import AutoModel, AutoTokenizer
2
3model = AutoModel.from_pretrained("monologg/distilkobert")
4tokenizer = AutoTokenizer.from_pretrained("monologg/distilkobert", trust_remote_code=True)