Okay, so it's fuckin impossible to find a simple-ass utility to merge a LORA to a base model, or a few LORAs to a base model. mergekit has some code in there to handle loras but it seems like it's dead-ended or incomplete, it doesn't actually do anything. axolotl can do it, if you want to break your back dealing with axolotl's bullshit—hard dependencies on CUDA, weird deepspeed crap, funky handling of transformers device maps, being SLOW AS FUCK, and so on—so it made more sense to just... write something.
So,
here you go. Turns out durbin also wrote something not totally dissimilar but it's a royal pain in the ass to find anything durbin writes, because he doesn't put it anywhere people can find it.
Go to town, people.
