This model finetuned from RWKV world 7B with context 32k, focus on multi turn coding.
Trainning details
4*A800 27hours with 1B tokens
image.png
datasets
mainly tiny codes and add a lots of long context multi turn datasets.
only finetuend in User: xxx\n\nAssistant: xxx\n format
Showcases
09713ffd8b5c21a525065a50964dd5f.jpg
other
if using RWKV runner to run this model, need to wait for updates in chat mode, as default chat using Question: xxx\n\nAnswer: xxx and have a default system prompt so far.