VAQA: Visual Arabic Question Answering

Abstract Visual Question Answering (VQA) is the problem of automatically answering a natural language question about given image or video. Standard Arabic sixth most spoken around world. However, to best our knowledge, there are neither research attempts nor datasets for VQA in Arabic. In this paper, we generate first (VAQA) dataset, which fully generated. The dataset consists almost 138k Image-Question-Answer (IQA) triplets and specialized yes/no questions real-world images. A novel database schema an IQA ground-truth generation algorithm specially designed facilitate automatic VAQA creation. We propose Arabic-VQA system, where task formulated as binary classification problem. proposed system five modules, namely visual features extraction, pre-processing, textual feature fusion, answer prediction. Since it Arabic, investigate several approaches channel, identify effective pre-processing representation. For purpose, 24 models developed, two question-tokenization approaches, three word-embedding algorithms, four LSTM networks with different architectures investigated. comprehensive performance comparison conducted between all these on dataset. Experiments indicate that ranges from 80.8 84.9%, while utilizing Arabic-specified considering special case separating tool "Image missing" embedding words using fine-tuned Word2Vec AraVec2.0 have significantly improved performance. best-performing model treats separate token, embeds Skip-Gram model, extracts one-layer unidirectional LSTM. Further, compared related developed other popular language, their only according scope showing very comparable