A simple AI tester. It supports HF and GGUF formats. Open main.py; you'll figure it out in 5 minutes.
Dataset Path
DATASET_PATH = "gramm.jsonl" — path to your JSONL file containing questions and answers.
Model Path
MODEL_PATH = "model/" — folder with your local HuggingFace model or GGUF file.
Model Format Switch
USE_GGUF = False — set to True for GGUF models, False for HuggingFace Transformers.
Generation Temperature
TEMPERATURE = 0.6 — controls prediction randomness; lower values are more deterministic.
JSONL Format
Each line needs "q" for the question prompt and "a" for space-separated correct answers.
Multi-Answer Support
The "a" field accepts multiple answers separated by spaces, supporting questions with several correct options.
