Based on my testing, here is what the model needs to be fine-tuned on to fix its mistakes:
Math & Chain-of-Thought: The model is extremely bad at math. It wasn't able to do a simple multiplication and it jumps to incorrect conclusions without showing correct reasoning. So, a dataset that introduces chain-of-thought would… See the full description on the dataset page:
https://huggingface.co/datasets/BhoomiPriya/Qwen-0.8B-Analysis.
