The evaluation toolkit to be used is lmms-eval. This toolkit facilitates the evaluation of models across multiple tasks and languages.
To install lmms-eval, execute the following commands:
git clone
https://github.com/EvolvingLMMs-Lab/lmms-eval
cd lmms-eval
pip install -e .
For additional dependencies for models, please refer to the lmms-eval repository.
Copy the required MINT task files to the lmms-eval… See the full description on the dataset page:
https://huggingface.co/datasets/MBZUAI/MINT.
