DualNetDomain-InvariantNetwork.PDF

DualNet: Domain-Invariant Network for Visual Question Answering Kuniaki Saito, Andrew Shin, Yoshitaka Ushiku, Tatsuya Harada The University of Tokyo 7 Chome-3-1 Hongo, Bunkyo Tokyo 113-8654, Japan 7 1 0 2 Abstract out that building an ensemble of DualNets with varying di- mensions leads to even more superior performances, despite y Visual question answering (VQA) task not only bridges the a gap between images and language, but also requires that spe- feeding identical set of input features to all DualNet units. cific contents within the image are understood as indicated Another advantage of our DualNet is that it is applicable M by linguistic context of the question, in order to generate the to both real images and abstract scenes categories. So far, 4 accurate answers. Thus, it is critical to build an efficient em- it has widely been considered that successful methods for bedding of images and texts. We implement DualNet, which real images cannot be directly ported to abstract scenes do- ] fully takes advantage of discriminative power of both image main, as they have fundamentally different characteristics. V and textual features by separately performing two operations. In fact, applying the basic setting of fc7 features for images Building an ensemble of DualNet further boosts the perfor- and long short-term memory (LSTM) with one hidden layer C mance. Cont