MemexQA: Visual Memex Question Answering

08/04/2017
by   Lu Jiang, et al.
3

This paper proposes a new task, MemexQA: given a collection of photos or videos from a user, the goal is to automatically answer questions that help users recover their memory about events captured in the collection. Towards solving the task, we 1) present the MemexQA dataset, a large, realistic multimodal dataset consisting of real personal photos and crowd-sourced questions/answers, 2) propose MemexNet, a unified, end-to-end trainable network architecture for image, text and video question answering. Experimental results on the MemexQA dataset demonstrate that MemexNet outperforms strong baselines and yields the state-of-the-art on this novel and challenging task. The promising results on TextQA and VideoQA suggest MemexNet's efficacy and scalability across various QA tasks.

READ FULL TEXT

page 2

page 4

research
01/12/2020

Focal Visual-Text Attention for Memex Question Answering

Recent insights on language and vision with neural networks have been su...
research
09/09/2023

MMHQA-ICL: Multimodal In-context Learning for Hybrid Question Answering over Text, Tables and Images

In the real world, knowledge often exists in a multimodal and heterogene...
research
11/01/2020

CHIME: Cross-passage Hierarchical Memory Network for Generative Review Question Answering

We introduce CHIME, a cross-passage hierarchical memory network for ques...
research
11/14/2022

Multi-VQG: Generating Engaging Questions for Multiple Images

Generating engaging content has drawn much recent attention in the NLP c...
research
08/27/2019

Movie Plot Analysis via Turning Point Identification

According to screenwriting theory, turning points (e.g., change of plans...
research
07/03/2020

Eliminating Catastrophic Interference with Biased Competition

We present here a model to take advantage of the multi-task nature of co...
research
09/05/2018

TVQA: Localized, Compositional Video Question Answering

Recent years have witnessed an increasing interest in image-based questi...

Please sign up or login with your details

Forgot password? Click here to reset