Dataset Description
This is a VQA dataset on Document Images from DocVQA.
Load the dataset
import pandas as pd
import os
import sys
data_name = sys.argv[1]
df = pd.read_parquet(f"data/{data_name}/images.parquet", engine="pyarrow")
output_dir = f"data/{data_name}"
os.makedirs(f"{output_dir}/imgs", exist_ok=True)
for idx, row in df.iterrows():
img_bytes = row['image']['bytes']
output_path = os.path.join(output_dir, row["path"])
with open(output_path, "wb") as f:… See the full description on the dataset page:
https://huggingface.co/datasets/openbmb/EVisRAG-Test-DocVQA.
