Letters in Words Evaluation Dataset
"The strawberry question is pretty much the new Turing Test for future AI" BlakeSergin OP 3mo agohttps://
www.reddit.com/r/singularity/comments/1enqk04/how_many_rs_in_strawberry_why_is_this_a_very/
This dataset .json provides a simple yet effective way to assess the basic letter-counting abilities of Large Language Models (LLMs). (Try it on the new SmolLM2 models.) It consists of a set of questions designed to evaluate an LLM's capacity for:
Understanding… See the full description on the dataset page:
https://huggingface.co/datasets/MartialTerran/Eval_Counting_Letters_in_Words.
