transformer
Objective
Recreate the decoder-only transformer from bottom up
Generate coherent WikiHow articles
Data
Wikihow corpus
Results
Sub par text generation results because of compute constraints (my potato laptop)
future improvements
Use GPU
Use BPE, Wordpiece etc for tokenization. The character level tokenization method is simplistic and fails to capture statistics of the corpus
