文章基本信息

标题：A Novel Approach on Visual Question Answering by Parameter Prediction using Faster Region Based Convolutional Neural Network
本地全文：下载
作者：Sudan Jha ; Anirban Dey ; Raghvendra Kumar 等
期刊名称：International Journal of Interactive Multimedia and Artificial Intelligence
印刷版ISSN：1989-1660
出版年度：2019
卷号：5
期号：5
页码：30-37
DOI：10.9781/ijimai.2018.08.004
语种：English
出版社：ImaI-Software
摘要：Visual Question Answering (VQA) is a stimulating process in the ﬁeld of Natural Language Processing (NLP) and Computer Vision (CV). In this process machine can find an answer to a natural language question which is related to an image. Question can be open-ended or multiple choice. Datasets of VQA contain mainly three components; questions, images and answers. Researchers overcome the VQA problem with deep learning based architecture that jointly combines both of two networks i.e. Convolution Neural Network (CNN) for visual (image) representation and Recurrent Neural Network (RNN) with Long Short Time Memory (LSTM) for textual (question) representation and trained the combined network end to end to generate the answer. Those models are able to answer the common and simple questions that are directly related to the image’s content. But different types of questions need different level of understanding to produce correct answers. To solve this problem, we use faster Region based-CNN (R-CNN) for extracting image features with an extra fully connected layer whose weights are dynamically obtained by LSTMs cell according to the question. We claim in this paper that a single R-CNN architecture can solve the problems related to VQA by modifying weights in the parameter prediction layer. Authors trained the network end to end by Stochastic Gradient Descent (SGD) using pretrained faster R-CNN and LSTM and tested it on benchmark datasets of VQA.