(i personally advise you to either go for F16, or Q8.)
Cascade0 DPO Instruct V2
Experimental DPO Pass to see the difference.
In some cases, during certain chats, the new DPO actually helps and makes the model feel more chat-y, sort of.
For eg, when asking 'How to make a salad', it responds:
image
Training info
Done in 2 weeks on a single RTX 4080, the raw pretraining only achieveing 40% of the max 12B tokens, due to potential overfill, reaching only 4.8B tokens.
Max context size is 1512.
SFT was done in two days, while DPO took only 1 day, on a RTX 3060M.
Other small models VS Cascade0-Series
All models are evaluated using LMEval Harness, on the same PC/Settings and GGUF with F16 Quant.
