Further RL post-training on CRAM. Interesting results.
Most of the RL was focused on CJK -> English translation tasks.
Interestingly enough, for certain translation reward functions, using LLM-as-a-Judge with 7-9B to 20-40B judges made little diifference, but kept the strongest judge possible to be safe.
Contrastive ranking rewards seem very effective, and also act as a form of regularization (as generations are always compared to each other, rather than some abstract score in some model's imagination.)
Language switching does still seem to be an issue, need further reward shaping experiments.
