The time of the great dragon war is upon us! How many different fantasy novels? One hundred and seventeen you say?

Trained with full text windows, followed by completion, followed by ORPO, followed by one more epoch of the full text, rotated 1/4 in the window. That last train settled everything down and it seems quite coherent.

This is not a chat model, but intended for storymode or similar. No prompt, but start with a bit of story, or a name.

*** Prologue

The sun rose

Authors notes are highly effective. You can use an authors note of something like:

You have quite a cast of characters to draw from. Perhaps Perrin makes a stop by the Waystone Inn, or Zeddicus and Gandalf have a smoke together.

I usually use Min-P of 0.1, dynatemp between 0.5 and 2, and smoothing between 0.05 and 0.2.

To get rid of unwanted EOS's, I did the following...

import torch
result_dict : dict[str, torch.Tensor] = model.state_dict()
result_dict['lm_head.weight'][2] = 0
model.state_dict = lambda : result_dict

So now there are no EOS's at all, ever.