dreamgen/

opus-llama-3-tokens-alt

This is alternative configuration for some special tokens for Opus Llama 3 models. This is needed for some backends because of the issues described here:

First, DreamGen Opus models use a variant of ChatML, so we rename <|start_header_id|> to <|im_start|> and <|eot_id|> to <|im_end|>. This is already done in the DreamGen Opus Llama 3 fp16 repos.

Then, in order to address the issues some backends are having with Llama 3's special tokens:

I consider using <|im_end|> as a stop token as suboptimal, as in multi-character scenarios I like to let he model generate multiple character messages at once, and this prevents that. But for now, until we can get custom stop strings with special tokens working, this is the best we have.