This workspace contains scripts for extracting, validating, and enriching
website URLs derived from the CWV Bench v0 dataset.
scripts/mine_cwv_urls.py: captures all known URLs per repo into
checkpoints/urls.ndjson.
scripts/extract_primary_urls.py: collapses each repo to a single canonical
URL, stored in checkpoints/primary_urls.ndjson.
scripts/enrich_primary_urls_with_github.py: backfills missing URLs… See the full description on the dataset page:
https://huggingface.co/datasets/Manaswi2006/Github_sites.