An implementation for SKA-Bench: A Fine-Grained Benchmark for Evaluating Structured Knowledge Understanding of LLMs
conda create -n skabench python=3.9.0
conda activate skabench
pip install openai
pip install asyncio
pip install uvloop
For noisy robustness, order insensitivity and information integration testbeds, you can run:
python process_dataset.py --type KG --sequence random --scale 1k
NOTE:
Please write the… See the full description on the dataset page:
https://huggingface.co/datasets/zjukg/SKA-Bench.
