This model is a fine-tuned version of
gpt-j-6B-8bit on the
haiku dataset.
The model is a fine-tuned version of GPT-J-6B-8Bit for generation of
Haikus. The model, data and training procedure is inspired by a
blog post by Robert A. Gonsalves.
We used the same multitask training approach as in der post, but significantly extended the dataset (almost double the size of the original one). A prepared version of the dataset can be found
here.
The model is intended to generate Haikus. To do so, it was trained using a multitask learning approach (see
Caruana 1997) with the following four different tasks: :
We used a collection of existing haikus for training. Furthermore, all haikus were used in their graphemes version as well as a phonemes version. In addition, we extracted key word for all haikus using
KeyBERT and sorted out haikus with a low text quality according to the
GRUEN score.
