egonrp/

gpt2-small-portuguese

>>> from tokenizers import Tokenizer
>>> tokenizer = Tokenizer.from_pretrained("pierreguillou/gpt2-small-portuguese")
[2023-03-11T18:59:51Z ERROR cached_path::cache] ETAG fetch for https://huggingface.co/pierreguillou/gpt2-small-portuguese/resolve/main/tokenizer.json failed with fatal error
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception: Model "pierreguillou/gpt2-small-portuguese" on the Hub doesn't have a tokenizer

# Downloading the HuggingFace model:
$ python.exe
>>> import huggingface_hub
>>> huggingface_hub.snapshot_download(repo_id="pierreguillou/gpt2-small-portuguese", local_dir=r"C:\dev\pierre_gpt2_small_pt")

# Generating "tokenizer.json" file:
cd C:\dev\pierre_gpt2_small_pt
$ python.exe
>>> from transformers import AutoTokenizer
>>> tokenizer = AutoTokenizer.from_pretrained(r"C:\dev\pierre_gpt2_small_pt")
>>> tokenizer._tokenizer.save("tokenizer.json")