AiR: Attention with Reasoning Capability

07/28/2020
by   Shi Chen, et al.
0

While attention has been an increasingly popular component in deep neural networks to both interpret and boost performance of models, little work has examined how attention progresses to accomplish a task and whether it is reasonable. In this work, we propose an Attention with Reasoning capability (AiR) framework that uses attention to understand and improve the process leading to task outcomes. We first define an evaluation metric based on a sequence of atomic reasoning operations, enabling quantitative measurement of attention that considers the reasoning process. We then collect human eye-tracking and answer correctness data, and analyze various machine and human attentions on their reasoning capability and how they impact task performance. Furthermore, we propose a supervision method to jointly and progressively optimize attention, reasoning, and task performance so that models learn to look at regions of interests by following a reasoning process. We demonstrate the effectiveness of the proposed framework in analyzing and modeling attention with better reasoning capability and task performance. The code and data are available at https://github.com/szzexpoi/AiR

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 5

page 9

page 12

page 13

page 23

04/20/2022

Attention in Reasoning: Dataset, Analysis, and Modeling

While attention has been an increasingly popular component in deep neura...
03/11/2022

REX: Reasoning-aware and Grounded Explanation

Effectiveness and interpretability are two essential properties for trus...
02/26/2021

Where to look at the movies : Analyzing visual attention to understand movie editing

In the process of making a movie, directors constantly care about where ...
07/16/2020

LogiQA: A Challenge Dataset for Machine Reading Comprehension with Logical Reasoning

Machine reading is a fundamental task for testing the capability of natu...
09/21/2021

Unsupervised Abstract Reasoning for Raven's Problem Matrices

Raven's Progressive Matrices (RPM) is highly correlated with human intel...
05/28/2019

Learning Dynamics of Attention: Human Prior for Interpretable Machine Reasoning

Without relevant human priors, neural networks may learn uninterpretable...
02/27/2020

Understanding and Enhancing Mixed Sample Data Augmentation

Mixed Sample Data Augmentation (MSDA) has received increasing attention ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.