You can deploy this model and start earning money today!

KETI-AIR/
kvqa

Views

No views yet

Visual question answering

VQA understands a provided image and if a person asks question about this, it provides an answer after analyzing (or reasoning) the image via natural language.

KVQA dataset

As part of T-Brain’s projects on social value, KVQA dataset, a Korean version of VQA dataset was created. KVQA dataset consists of photos taken by Korean visually impaired people, questions about the photos, and 10 answers from 10 distinct annotators for each question. Currently, it consists of 30,000 sets of images and questions, and 300,000 answers, but by the end of this year, we will increase the dataset size to 100,000 sets of images and questions, and 1 million answers. This dataset can be used only for educational and research purposes. Please refer to the attached license for more details. We hope that the KVQA dataset can simultaneously provide opportunities for the development of Korean VQA technology as well as creation of meaningful social value in Korean society.

You can download KVQA dataset via this link.

Evaluation

We measure the model's accuracy by using answers collected from 10 different people for each question. If the answer provided by a VQA model is equal to 3 or more answers from 10 annotators, it gets 100%; if less than 3, it gets a partial score proportionately. To be consistent with ‘human accuracies’, measured accuracies are averaged over all 10 choose 9 sets of human annotators. Please refer to VQA Evaluation which we follow.

Usage

python

from datasets import load_dataset

raw_datasets = load_dataset(
                "kvqa.py", 
                "default",
                cache_dir="huggingface_datasets", 
                data_dir="data",
                ignore_verifications=True,
            )

dataset_train = raw_datasets["train"]

for item in dataset_train:
    print(item)
    exit()

Data statistics

v1.0 (Jan. 2020)

	Overall (%)	Yes/no (%)	Number (%)	Etc (%)	Unanswerable (%)
# images	100,445 (100)	6,124 (6.10)	9,332 (9.29)	69,069 (68.76)	15,920 (15.85)
# questions	100,445 (100)	6,124 (6.10)	9,332 (9.29)	69,069 (68.76)	15,920 (15.85)
# answers	1,004,450 (100)	61,240 (6.10)	93,320 (9.29)	690,690 (68.76)	159,200 (15.85)

Data

Data field description

Name	Type	Description
VQA	`[dict]`	`list` of `dict` holding VQA data
+- image	`str`	filename of image
+- source	`str`	data source `["kvqa"
+- answers	`[dict]`	`list` of `dict` holding 10 answers
+--- answer	`str`	answer in `string`
+--- answer_confidence	`str`	`["yes"
+- question	`str`	question about the image
+- answerable	`int`	answerable? `[0
+- answer_type	`str`	answer type `["number"

Data example

json

[{
        "image": "KVQA_190712_00143.jpg",
        "source": "kvqa",
        "answers": [{
            "answer": "피아노",
            "answer_confidence": "yes

KETI-AIR/kvqa