Quant's available from Bartowski <3: GGUF Single Quant from me: 4bpw Exl2
image/png
Uses Mistral Formatting, Text completion preset here
Notes: One off train most likely, this was done purely for internal testing purposes but seemed ok enough to release. I do not plan to offer any kind of extended support for using this model, so your mileage may vary depending on use and context size.
(Nemo 12B instruct as base)
200k randomized subset of GU_instruct-Remastered-1.1, with a splash of 25k hathor/poppy sauce, slow cooked for 3 epochs on medium heat.
