Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog

02/01/2019
by   Zhe Gan, et al.
0

This paper presents Recurrent Dual Attention Network (ReDAN) for visual dialog, using multi-step reasoning to answer a series of questions about an image. In each turn of the dialog, ReDAN infers answers progressively through multiple steps. In each step, a recurrently-updated semantic representation of the (refined) query is used for iterative reasoning over both the image and previous dialog history. Experimental results on VisDial v1.0 dataset show that the proposed ReDAN model outperforms prior state-of-the-art approaches across multiple evaluation metrics. Visualization on the iterative reasoning process further demonstrates that ReDAN can locate context-relevant visual and textual clues leading to the correct answers step-by-step.

READ FULL TEXT

page 1

page 8

research
04/29/2020

Multi-View Attention Networks for Visual Dialog

Visual dialog is a challenging vision-language task in which a series of...
research
06/24/2023

Full Automation of Goal-driven LLM Dialog Threads with And-Or Recursors and Refiner Oracles

We automate deep step-by step reasoning in an LLM dialog thread by recur...
research
12/18/2019

DMRM: A Dual-channel Multi-hop Reasoning Model for Visual Dialog

Visual Dialog is a vision-language task that requires an AI agent to eng...
research
02/25/2019

Making History Matter: Gold-Critic Sequence Training for Visual Dialog

We study the multi-round response generation in visual dialog systems, w...
research
09/11/2019

Probabilistic framework for solving Visual Dialog

In this paper, we propose a probabilistic framework for solving the task...
research
04/12/2020

AMR Parsing via Graph-Sequence Iterative Inference

We propose a new end-to-end model that treats AMR parsing as a series of...
research
04/14/2020

DialGraph: Sparse Graph Learning Networks for Visual Dialog

Visual dialog is a task of answering a sequence of questions grounded in...

Please sign up or login with your details

Forgot password? Click here to reset