Based on GPT-J, this model attemps to model a conversation by rejecting utterances to switch to a different reply.

Reddit comment chains are used to model this, with the first instance being the highest scoring reply to the current comment, which we will randomly reject and go to the next highest comment. This repeats until we either run out of comments, or accept the current comment.

We sample the reply chain in this manner until we do not have any more replies to the current comment.

The input to the model will look something like this:

ExampleUser
Example Title
Example Comment <BACKGROUND_INDEX_TOKEN> Other_User
Edit: Second <REJECTED_UTTERANCE> First_User
First! <SELECTED_UTTERANCE> Other_User
😭 <SELECTED_UTTERANCE>