Quantizations of https://huggingface.co/cgato/TheSpice-7b-FT-v0.3.1

Quants ending in "_X" are experimental quants. These quants are the same as normal quants, but their token embedding weights are set to Q8_0 except for Q6_K and Q8_0 which are set to F16. The change will make these experimental quants larger but in theory, should result in improved performance.

List of experimental quants:

The latest TheSpice, dipped in Mama Liz's LimaRP Oil. I've focused on making the model more flexible and provide a more unique experience. I'm still working on cleaning up my dataset, but I've shrunken it down a lot to focus on a "less is more" approach. This is ultimate a return to form of the way I used to train Thespis, with more of a focus on a small hand edited dataset.

Narration

If you request information on objects or characters in the scene, the model will narrate it to you. Most of the time, without moving the story forward.

If you're using Ooba in verbose mode as a server, you can check if you're console is logging something that looks like this.

{System Prompt}

Username: {Input}
BotName: {Response}
Username: {Input}
BotName: {Response}