This dataset is the downloaded variant of Spawning/PD12M. More specifically, this dataset
is compatible with webdataset. It was made public after obtaining permission
from the original authors of the dataset.
You can use the following to explore the dataset with webdataset:
import webdataset as wds
dataset = (
wds.WebDataset(dataset_path… See the full description on the dataset page:
https://huggingface.co/datasets/Spawning/pd12m-full.
