This is a bust, dont use it! I think,since it is a MoE model, the training was probably confusing, as some bits and pieces of reasoning and non-reasoning datasets went to places it probably should not have.
I am going to try to train the individual 12B models instead. Perhaps it will work better!
