Improving Natural Language Processing Tasks with Human Gaze-Guided Neural Attention

10/15/2020
by   Ekta Sood, et al.
0

A lack of corpora has so far limited advances in integrating human gaze data as a supervisory signal in neural attention mechanisms for natural language processing(NLP). We propose a novel hybrid text saliency model (TSM) that, for the first time, combines a cognitive model of reading with explicit human gaze supervision in a single machine learning framework. On four different corpora we demonstrate that our hybrid TSM duration predictions are highly correlated with human gaze ground truth. We further propose a novel joint modelling approach to integrate TSM predictions into the attention layer of a network designed for a specific upstream NLP task without the need for any task-specific human gaze data. We demonstrate that our joint model outperforms the state of the art in paraphrase generation on the Quora Question Pairs corpus by more than 10 sentence compression on the challenging Google Sentence Compression corpus. As such, our work introduces a practical approach for bridging between data-driven and cognitive models and demonstrates a new way to integrate human gaze-guided neural attention into NLP tasks.

READ FULL TEXT

page 6

page 16

page 17

research
02/11/2023

Synthesizing Human Gaze Feedback for Improved NLP Performance

Integrating human feedback in models can improve the performance of natu...
research
12/21/2021

A Survey on Using Gaze Behaviour for Natural Language Processing

Gaze behaviour has been used as a way to gather cognitive information fo...
research
04/12/2016

Improving sentence compression by learning to predict gaze

We show how eye-tracking corpora can be used to improve sentence compres...
research
10/20/2020

Individual corpora predict fast memory retrieval during reading

The corpus, from which a predictive language model is trained, can be co...
research
10/10/2018

New Vistas to study Bhartrhari: Cognitive NLP

The Sanskrit grammatical tradition which has commenced with Panini's Ast...
research
11/09/2020

Generating Image Descriptions via Sequential Cross-Modal Alignment Guided by Human Gaze

When speakers describe an image, they tend to look at objects before men...
research
07/16/2021

Architectures of Meaning, A Systematic Corpus Analysis of NLP Systems

This paper proposes a novel statistical corpus analysis framework target...

Please sign up or login with your details

Forgot password? Click here to reset