One arm of a study on teaching a language model to notice when its own conversation has collapsed into repetition, emit a special <end> token, and change the subject.

Version 4. A reliable detector (77-81%), built by cutting both classes from the same conversations. Its weakness is that its escapes propose nothing: 'let us talk about something else'.

Two things that will silently break this. <end> is a special token (id 151665), so decode with skip_special_tokens=False or the model fires and you see nothing. And feed it about 5 turns: it was trained on 5-turn windows, and a 16-turn window dropped firing to 0 of 17 conversations in our tests.

Full write-up, training recipe and limitations: center-for-humans-and-machines/end_token. Datasets: Farrous05/end_token_data_full.

Built at the Center for Humans and Machines, Max Planck Institute for Human Development.