Built as a learning project to understand LLM internals without HuggingFace Transformers. Architecture based on Attention Is All You Need (Vaswani et al., 2017)

Source code: https://github.com/conbopk/Yu-Gi-Oh-LLM

Model Architecture

Property	Value
Architecture	Decoder-only Transformer
Embedding dim	128
Context length	128 tokens
Attention heads	4
Feed-forward dim	1024
Decoder blocks	3
Total parameters	2,106,624
Positional encoding	Sinusoidal (fixed)
Activation	ReLU
Tokenizer	BPE (vocab size 4096)

Training Details

Property	Value
Dataset	hammadus/yugioh-full-card-database-index-august-1st-2025
Training hardware	NVIDIA RTX 4050 6GB VRAM
Framework	PyTorch (CUDA cu126)
Best val loss	1.2585
Epochs trained	1
Total steps	70,000
Optimizer	Adam (β₁=0.9, β₂=0.98, ε=1e-9)
LR schedule	Warmup (2000 steps) + Inverse Square Root Decay
Batch size	32

Sample Generated Cards

<card>
name: roar, the infer king
rarity: ultra rare
type: monster
sub_type: [spellcaster/effect]
attribute: fire
rank: level 8
attack: 2800
defense: 2800
description: cannot be normal summoned/set. must be special summoned by its
own effect. during your opponent's main phase (quick effect): you can target 1
"nekroz" monster you control; this turn, it can make a second attack during
each battle phase this turn. if this card attacks, it is changed to defense
position at the end of the battle phase.
</card>

<card>
name: gaia soul the fire kings
rarity: prismatic secret rare
type: spell
sub_type: quick-play
description: target 1 face-up monster your opponent controls (quick effect):
destroy it. you can only activate 1 "fire formation - tenki" per turn.
</card>

Usage

⚠️ This is a custom architecture. It cannot be loaded with AutoModel.from_pretrained(). You need the source code from the GitHub repo.

Step 1: Clone the repo

bash

git clone https://github.com/conbopk/Yu-Gi-Oh-LLM
cd Yu-Gi-Oh-LLM
uv sync

Step 2: Download the model weights

python

from huggingface_hub import hf_hub_download
 
# Download best_model.pt
model_path = hf_hub_download(
    repo_id="ThanhNguyen111/yu-gi-oh-llm",
    filename="best_model.pt",
    local_dir="model/hf/",
)
 
# Download tokenizer
tokenizer_path = hf_hub_download(
    repo_id="ThanhNguyen111/yu-gi-oh-llm",
    filename="tokenizer-bpe.json",
    local_dir="data/processed/yugioh/v001/",
)

Step 3: Update MODEL_PATH and run

# Edit scripts/inference.py: MODEL_PATH = "model/hf/best_model.pt"
uv run scripts/inference.py

# Open a booster pack
uv run scripts/open_pack.py

Files in this repo

File	Description
`best_model.pt`	Model weights + optimizer state + full config
`tokenizer-bpe.json`	BPE tokenizer (HuggingFace tokenizers format)
`config.json`	Model architecture config (human-readable)
`README.md`	This file
`INFERENCE.md`	Detailed inference guide

License

MIT — free to use, modify, distribute.