A quick static GGUF made with llama.cpp b6691 by converting HF safetensors to F32 gguf then quantizing down to q4km.
I suggest using better quants, like imatrix quants if they're available.
Just a sneak peek of what I'm cooking in a little project called Vellum. This model was made to evaluate the quality of the CreativeGPT dataset, and how well Qwen3 trains on it. This is just one of many datasets that the final model will be trained on (which will also be using a different base model).
This got pretty good results compared to the regular instruct in my testing so thought I would share. I trained for 3 epochs, but both checkpoints at 2 epoch and 3 epoch were too overbaked. This checkpoint, at 1 epoch performed best.
I'm pretty surprised how decent this came out since Qwen models aren't that great at writing, especially at this size.
Usage
Use with thinking/chain-of-thought disabled. Use ChatML prompt format.
Qwen suggested sampler settings are recommended.
Temperature: 0.7
Top_P: 0.8
Top_K: 20
Min_P: 0
Quants
GGUFs
Special Thanks
Big thanks to everyone over at the KoboldAI discord. The members there have helped me a ton with various things over the long while I've been there.
