The evaluation of tiny-aya-base on Hausa mathematical reasoning reveals several systematic blind spots:
Instruction Non-Compliance: The model consistently fails to follow prompt instructions, whether given in Hausa or English. As a base model without instruction tuning, it treats prompts as text to continue rather than instructions to follow. The model's documentation mentions preference alignment, but this alone is… See the full description on the dataset page: https://huggingface.co/datasets/Al-Mahi/tiny-aya-blind-spots.
