EXL3 FormatFor use with TabbyAPI (Slower on Ampere))
Prompting
Model has been tuned with the ChatML formatting. A typical input would look like this:
<|im_start|>user
Hi there!<|im_end|>
<|im_start|>assistant
Nice to meet you!<|im_end|>
<|im_start|>user
Can I ask a question?<|im_end|>
<|im_start|>assistant
Samplers
For testing of this model, I used Temp=1, 0.1 Min-P.
See Axolotl Config
https://files.catbox.moe/u9dakg.yml
Training
The training was done for 2 epoch using 8 x H200s GPUs graciously provided by Kalomaze for the fine-tuning of the model.
