This is an instruct model. It's to test out the effects of my dataset on the InternLM model if you just change everything to say "Llama" instead of "InternLM" in the config files.
I had a theory that a lot of these models are just Llama models with extraneous loaders. This is to test that theory.
Trained at rank 256 for 1 epoch at constant learning rate.
