The initial results were a surprisingly coherent and functional model although I went ahead and gave it a fairly deep LoRA on 51 megabytes of raw text.
It responds well to Alpaca instruct style prompt formatting.
It can be a little rude at times and doesn't have Dendrite's ego and thirst for philosophical discussion but I feel that it's overall it's a much better general purpose model.
It does occasionally output grammatical errors during RP so might need a few more epochs to better fit the training data.
If you are role playing using the SillyTavern+SimpleProxy stack it does have a tendency to run away with a scene when using the verbose.mjs prompt format. The singleline.mjs format sometimes remedies this issue however it also causes some characters to give very short, dull replies. So achieving a balance might require a complete new custom prompt format.
use_cache was originally set to false when uploaded this has now been remedied. recommended edit or redownload config.
I have been asked about GPTQ for this model unfortunately there seems to be some weird vocabulary mismatch that causes GPTQ to corrupt the model. So the only way to run it in 4bit at the moment is to load the FP16 model in 4bit via transformers.
