Finetuned from model : unsloth/qwen2.5-7b-unsloth-bnb-4bit
Experiment
Instruction Task
The training effect is good, but the understanding of words with Cantonese cultural background is still insufficient
Chat Task
After many experiments, it was found that the model generation would have an echo effect after multiple rounds of training, and the model would repeatedly copy the user output as an answer
image/png
Analysis:
1.The dataset contains duplicate content (corrected, but there are still problems)
2.The data is not diverse enough, and simple word explanations cannot meet the training requirements of the dialogue task
