非原製作者,僅搬移與封裝成 HF Dataset 格式方便使用。
此資料集下載約需要125GB(.parquet壓縮),對應悟道220G版本。
如果使用此資料集,請引用原作者:
@misc{ c6a3fe684227415a9db8e21bac4a15ab,
author = {Zhao Xue and Hanyu Zhao and Sha Yuan and Yequan Wang},
title = {{WuDaoCorpora Text}},
year = 2022,
month = dec,
publisher = {Science Data Bank},
version = {V1},
doi = {10.57760/sciencedb.o00126.00004},
url =
https://doi.org/10.57760/sciencedb.o00126.00004
}… See the full description on the dataset page:
https://huggingface.co/datasets/p208p2002/wudao.