Views
No views yet
gpt_decoder_checkpoint.pt) and contains a Hugging Face compatible model and tokenizer so you can load it with transformers locally.config.json — model configuration (GPT-2 compatible fields: vocab_size, n_positions, n_ctx, n_embd, n_layer, n_head).pytorch_model.bin or tf_model.h5 / model.safetensors — model weights (PyTorch).tokenizer.json, vocab.json, merges.txt, tokenizer_config.json, special_tokens_map.json — tokenizer assets (GPT-2 tokenizer was reused and saved here).Note: If you have a different tokenizer used during training, replace the tokenizer files in this folder with your original tokenizer files for best results.
transformers:1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3tokenizer = AutoTokenizer.from_pretrained("./hf_model")
4model = AutoModelForCausalLM.from_pretrained("./hf_model")
5
6prompt = "Once upon a time"
7inputs = tokenizer(prompt, return_tensors="pt")
8outputs = model.generate(**inputs, max_new_tokens=50)
9print(tokenizer.decode(outputs[0], skip_special_tokens=True))app.py in the project is already configured to use model_name = "hf_model" (local folder). Run the app with the project's venv Python:1/path/to/your/project/.venv/bin/python app.py
2# then open http://127.0.0.1:7860 in your browserapp.py and change demo.launch() to demo.launch(share=True) and restart the process.huggingface_hub from Python (login required):1from huggingface_hub import create_repo, upload_folder
2create_repo("pragsyy1729/decoder_shakespeare", exist_ok=True)
3upload_folder(folder_path="hf_model", repo_id="pragsyy1729/decoder_shakespeare")git lfs and push the folder into a repository created on the Hub.GPT2LMHeadModel. While the script attempted to match shapes automatically, verify generation quality.model.save_pretrained() and tokenizer.save_pretrained() from the original training environment.