You can deploy this model and start earning money today!

HAR5HA-YELLELA/
luffy-gpt

Name: luffy-gpt
Availability: InStock

Views

No views yet

Luffy GPT

49.4M parameter GPT transformer trained from scratch on One Piece (Luffy) dialogue, fine-tuned with 5,004 SFT conversation pairs using style token conditioning and GPT-2 subword tokenizer.

Models

File	Params	Tokenizer	Description
`luffy_gpt.pth`	10.81M	char-level (96)	Base pretrained model (25k steps)
`luffy_gpt_finetuned.pth`	10.81M	char-level (96)	Fine-tuned with style tokens + SFT
`luffy_gpt_gpt2tok.pth`	49.39M	GPT-2 BPE (50,257)	Base pretrained with GPT-2 tokenizer (25k steps)
`luffy_gpt_gpt2tok_finetuned.pth`	49.39M	GPT-2 BPE (50,257)	Best model -- fine-tuned with SFT + EOS

Architecture

Decoder-only GPT built from scratch in PyTorch
n_embd=384, n_head=6, n_layer=6, context_size=256
KV cache, top-k/top-p sampling, temperature, repetition penalty
Style embedding for switching between corpus and conversation mode
EOS token for controlled generation stopping
Cosine LR decay with warmup, weight decay, early stopping

Try it out

This is not a perfect chatbot. It's a small model trained from scratch on limited data. Works best with direct questions.

Prompts that work well:

Who are you?
What is your dream?
Are you hungry?
Are you scared?
Who is Zoro?
Who is Shanks?
What happened to Ace?
Are you a hero?

Run locally

bash

# chat with best model (GPT-2 tokenizer)
python gpt.py --input dataset/processed/corpus_clean.txt \
  --eval luffy_gpt_gpt2tok_finetuned.pth --interactive --style 1 \
  --temperature 0.8 --top-k 50 --top-p 0.9 --repetition-penalty 1.2 \
  --gpt2-tokenizer

# pretrain from scratch
python gpt.py --input dataset/processed/corpus_clean.txt \
  --train luffy_gpt_gpt2tok.pth --epoch 25000 --gpt2-tokenizer \
  --warmup-steps 500 --weight-decay 0.1

# fine-tune
python gpt.py --input dataset/processed/corpus_clean.txt \
  --finetune luffy_gpt_gpt2tok_finetuned.pth \
  --sft-input dataset/luffy_sft.txt --pretrained luffy_gpt_gpt2tok.pth \
  --epoch 10000 --lr 5e-5 --warmup-steps 200 --weight-decay 0.1 \
  --gpt2-tokenizer

Sample outputs

You: who are you?
Luffy: I'm Monkey D. Luffy!

You: are you scared?
Luffy: Nope.

You: are you okay?
Luffy: Not really.

Tokenizer comparison

Tokenizer	Vocab	256 token context	Result
Character-level	96	~40 words	val loss 1.23, garbled SFT
Custom BPE (SentencePiece)	2,000	~125 words	overfitted, abandoned
GPT-2 BPE	50,257	~100 words	best chatbot responses

Built as part of the Build Fellowship program.

HAR5HA-YELLELA/luffy-gpt