Resources and tools for evaluating the performance and behavior of web crawling systems.
Overview
CrawlEval provides a comprehensive suite of tools and datasets for evaluating web crawling systems, with a particular focus on HTML pattern extraction and content analysis. The project includes:
A curated dataset of web pages with ground truth patterns
Tools for fetching and analyzing web content
Evaluation metrics and benchmarking capabilities
Dataset… See the full description on the dataset page: https://huggingface.co/datasets/crawlab/crawleval.
