My instruct model has undergone DoRA SFT on my custom synthetic chat dataset, containing single and multi-round chats containing SFW, NSFW, and Toxic single and multi-round chats. This reinforces the model's uncensored compliance with all prompts and teaches it how to better fill roles assigned to it in the system prompt.
This model has yet to go through DPO preference training and may still have rough edges.
** Be aware that this adapter, when used without a system prompt to assign it a role may make up its own role. Meaning if you just say 'Hello' it could resond with 'Hello, how may I assist you?' or it might respond with something like "Hi, my name is Carol and I'm a librarian here to assist you with finding the book you're looking for."
